ILLIXR

Illinois Extended Reality testbed or ILLIXR (pronounced like elixir) is the first open-source full-system Extended Reality (XR) testbed. It contains standalone state-of-the-art components representative of a generic XR workflow, as well as a runtime framework that integrates these components into an XR system. ILLIXR's runtime integration framework is modular, extensible, and OpenXR-compatible.

We use the term components and not kernels or computations because each component of ILLIXR is an entire application in itself, and consists of many kernels and computations. At the moment, ILLIXR interfaces the following state-of-the-art components, with periodic ILLIXR source snapshots hosted on our releases page:

• Perception

  1. Eye tracking **
  2. Scene reconstruction **
  3. Simultaneous Localization and Mapping **

• Visual

  1. Chromatic aberration correction
  2. Computational holography for adaptive multi-focal displays **
  3. Lens distortion correction
  4. Reprojection

• Aural

  1. Audio encoding
  2. Audio playback

(** Source is hosted in an external repository under the ILLIXR project.)

We plan on adding more components (e.g., graphics and multiple versions for individual components) and extending the runtime in the future. Our goal is not to create a commercial quality XR product for current hardware. Instead, the goal for ILLIXR is to advance computer architecture, systems, and hardware-software co-design research for XR by making available a full system and key state-of-the-art components of both modern and future XR applications.

Many of the current components of ILLIXR were developed by domain experts and obtained from publicly available repositories. They were modified for one or more of the following reasons: fixing compilation, adding features, removing extraneous code/dependencies. Each component not developed by us is available as a forked github repository for proper attribution to its authors.

Detailed descriptions of each component and our runtime, including performance and energy profiles, can be found in our paper.

Publications

We request that you cite our following paper (new version coming soon) when you use ILLIXR for a publication. We would also appreciate it if you send us a citation once your work has been published.

@misc{HuzaifaDesai2020,
    title={Exploring Extended Reality with ILLIXR: A new Playground for Architecture Research},
    author={Muhammad Huzaifa and Rishi Desai and Xutao Jiang and Joseph Ravichandran and Finn Sinclair and Sarita V. Adve},
    year={2020},
    eprint={2004.04643},
    primaryClass={cs.DC}
}

Getting Started and Documentation

For more information, see our Getting Started page.

Acknowledgements

Muhammad Huzaifa led the development of ILLIXR in Sarita Adve’s research group at the University of Illinois at Urbana-Champaign. Other major contributors include Rishi Desai, Samuel Grayson, Xutao Jiang, Ying Jing, Jae Lee, Fang Lu, Yihan Pang, Joseph Ravichandran, Finn Sinclair, Boyuan Tian, Henghzhi Yuan, and Jeffrey Zhang.

ILLIXR came together after many consultations with researchers and practitioners in many domains: audio, graphics, optics, robotics, signal processing, and extended reality systems. We are deeply grateful for all of these discussions and specifically to the following: Wei Cu, Aleksandra Faust, Liang Gao, Matt Horsnell, Amit Jindal, Steve LaValle, Steve Lovegrove, Andrew Maimone, Vegard Øye, Martin Persson, Archontis Politis, Eric Shaffer, Paris Smaragdis, Sachin Talathi, and Chris Widdowson.

Our OpenXR implementation is derived from Monado. We are particularly thankful to Jakob Bornecrantz and Ryan Pavlik.

The development of ILLIXR was supported by the Applications Driving Architectures (ADA) Research Center (a JUMP Center co-sponsored by SRC and DARPA), the Center for Future Architectures Research (C-FAR, a STARnet research center), a Semiconductor Research Corporation program sponsored by MARCO and DARPA, and by a Google Faculty Research Award. The development of ILLIXR was also aided by generous hardware and software donations from ARM and NVIDIA. Facebook Reality Labs provided the OpenEDS Semantic Segmentation Dataset.

Wesley Darvin came up with the name for ILLIXR.

Licensing Structure

ILLIXR is available as open-source software under the permissive University of Illinois/NCSA Open Source License. As mentioned above, ILLIXR largely consists of components developed by domain experts and modified for the purposes of inclusion in ILLIXR. However, ILLIXR does contain software developed solely by us. The NCSA license is limited to only this software. The external libraries and softwares included in ILLIXR each have their own licenses and must be used according to those licenses:

• ElasticFusion \ ElasticFusion license

• GTSAM \ Simplified BSD License

• HOTlab \ GNU Lesser General Public License v3.0

• Kimera-VIO \ Simplified BSD License

• libspatialaudio \ GNU Lesser General Public License v2.1

• Monado \ Boost Software License 1.0

• moodycamel::ConcurrentQueue \ Simplified BSD License

• Open-VINS \ GNU General Public License v3.0

• RITnet \ MIT License

Note that ILLIXR's extensibility allows the source to be configured and compiled using only permissively licensed software. For more details on configuring ILLIXR, see the ILLIXR Plugins and Writing your Plugin pages.

Get in Touch

Whether you are a computer architect, a systems person, an XR application developer, or just anyone interested in XR, we would love to hear your feedback on ILLIXR! ILLIXR is a living testbed and we would like to both refine existing components and add new ones. We believe ILLIXR has the opportunity to drive future computer architecture and systems research for XR, and can benefit from contributions from other researchers and organizations. If you would like to be a part of this effort, please contact us at illixr at cs dot illinois dot edu or visit us on Gitter or just send us a pull request!