{"id":383246,"date":"2017-05-12T08:44:20","date_gmt":"2017-05-12T15:44:20","guid":{"rendered":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/?post_type=msr-project&p=383246"},"modified":"2017-06-15T08:57:26","modified_gmt":"2017-06-15T15:57:26","slug":"holographic-near-eye-displays-virtual-augmented-reality","status":"publish","type":"msr-project","link":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/holographic-near-eye-displays-virtual-augmented-reality\/","title":{"rendered":"Holographic Near-Eye Displays for Virtual and Augmented Reality"},"content":{"rendered":"

Summary<\/h1>\n

In this project, we explore how digital<\/strong> holography<\/strong> can be used to build novel near-eye displays for virtual\u00a0and\u00a0mixed (or augmented)\u00a0reality.\u00a0\u00a0We experiment with true, phase-only <\/strong>holograms in which the image is formed by the interference of laser light<\/strong>. We address some of the known limitations of digital holograms and demonstrate how holography can add powerful, new features<\/strong> to near-eye displays: per-pixel focus control<\/strong>, vision correction<\/strong>, and unpresented combinations of form factor and field of view<\/strong>.\u00a0\u00a0See the\u00a0technical paper<\/a> and video (opens in new tab)<\/span><\/a>\u00a0for limitations, additional details, and a description of future work. Also see our\u00a0blog post<\/a> on the project.<\/p>\n

Note that this Microsoft Research project is not necessarily indicative of any Microsoft product roadmap, but relates to basic research around holographic displays.<\/em><\/p>\n

Addressing the Limitations of Holograms<\/h1>\n

Image quality<\/h2>\n

Digital holograms are often associated with noisy<\/em>, low contrast<\/em>, low resolution<\/em>, and mono color<\/em> imagery. We demonstrate how high contrast<\/strong>, high resolution<\/strong>, and full color <\/strong>digital holograms can be\u00a0formed using existing hardware devices, yielding images that more closely match the quality of conventional displays. The following images are photographs\u00a0taken of a prototype holographic near-eye display. The field of view is 70 degrees horizontally.<\/p>\n

\"\"<\/p>\n

<\/h2>\n

Computation Speed<\/h2>\n

Due to the complex required calculations, digital holograms are also often associated with slow<\/em>, off-line<\/em> calculations. We propose\u00a0the use of eye tracked approximate holograms\u00a0that\u00a0have\u00a0correct image focus and\u00a0best image quality where the user is looking<\/strong>.\u00a0Combined with GPU-accelerated algorithms, we demonstrate real-time <\/strong>hologram generation at rates of 90-260 Hz<\/strong> on a desktop GPU (NVIDIA GeForce GTX 980 TI).<\/p>\n

 <\/p>\n

Adding\u00a0a Powerful Feature Set to\u00a0Near-Eye Display<\/h1>\n

<\/h2>\n

Focus Control<\/h2>\n

The ability to change the image focus <\/em>is desirable in a near-eye display: it addresses the accommodation-convergence conflict, allows image focus to match eye focus in a see-through display, and adds realism. Unlike vari-focal, multi-focal, and light field displays, a holographic display is able to provide per-pixel<\/strong> focus control<\/strong> with virtually no discretization<\/strong>, enabling smooth and natural focal cues to imagery. Below, we demonstrate the ability to display holograms with per-pixel focus control with high resolution and\u00a0image quality on a prototype display.\u00a0Note\u00a0that the region of camera focus (the dragon’s chest)\u00a0<\/em>is in focus, while closer\u00a0regions (e.g. the head)\u00a0<\/em> farther regions (e.g. the\u00a0main body and tail)<\/em> are out of focus.\u00a0The inset images show various regions of the dragon’s body when they are brought in and out of focus;\u00a0see the video for an animated change of focus.\u00a0(This result was calculated offline.)<\/em><\/p>\n

\"\"<\/p>\n

\u00a0<\/em><\/p>\n

Vision Correction<\/h2>\n

Another\u00a0desirable quality of a display\u00a0is vision correction, <\/em>the ability to fix defects in the user’s vision. Such a display allows a user to view the display without their glasses<\/strong>. We demonstrate that holographic displays are capable of a powerful vision correction capability: they can correct simple vision problems such as near- <\/em>and far- sightedness <\/em>as well as higher order vision problems like astigmatism<\/em>. In the image below we demonstrate the ability of a prototype holographic display to correct astigmatism. In\u00a0the left image<\/em>, for reference, we show a image on a prototype display for a user with normal vision. In the center image<\/em>, the display is viewed with astigmatic vision by the addition of a cylindrical lens in front of the camera; <\/em>note that lines that are primarily horizontal are quite blurred in the vertical direction. Finally, in the right image<\/em>, the display is viewed with astigmatic vision, but with vision correction applied in the hologram. The image looks virtually the same as the display viewed with normal vision.<\/p>\n

\"\"<\/p>\n

<\/h2>\n

Aberration correction<\/h2>\n

Holographic displays are also able to provide aberration correction, <\/em>the ability to\u00a0encode\u00a0optical\u00a0corrections\u00a0<\/strong>to the display optics in software. Such a capability allows the use of simpler optics and enables new optical architectures, as we describe in the next section. The optical corrective capability of holograms are extremely powerful<\/strong> and allow arbitrary optical corrections on a per-pixel basis. This is akin to using an independently customized, complex lens<\/em> to form each point in the image, and where each of these lenses acts independently yet can overlap<\/em> with its neighbors. In the image below, we show images from an optical see-through <\/em>holographic display in a challenging off-axis <\/em>optical configuration. In the top left image<\/em>, we\u00a0show a hologram with no aberration correction applied; the off-axis optics\u00a0result in severe astigmatism so that no part of the image is\u00a0both in proper horizontal and vertical focus. In the top right image<\/em>, we show a hologram with aberration correction applied — note that the lines are sharp in both directions over the whole field of view. Finally, in the bottom image<\/em>, we demonstrate the ability to display complex, full color, optical see-through holograms with good image quality after the application of aberration correction.<\/p>\n

 <\/p>\n

\"\"<\/p>\n

<\/h2>\n

Form Factor<\/h2>\n

One of the most important considerations for a near-eye display is form factor, especially for see-through,\u00a0mixed reality devices. Lightweight, eyeglasses-like<\/em> displays are needed to facilitate viewer comfort and all-day use. Using a powerful wavefront correction ability, we demonstrate that holographic near-eye displays\u00a0enable a combination of form factor and field of view<\/strong> that have been inaccessible through conventional means. As a first step, we demonstrate a prototype near-eye\u00a0holographic display in a sunglasses-like form factor<\/em> with a wide 80 degree<\/em> horizontal field of view (left image<\/em>). The display\u00a0uses a thin and highly transparent holographic optical element<\/em> as a combiner, cut in the shape of an eyeglass lens, allowing an optical see-through capability<\/strong> (center image). <\/em>Even\u00a0with these miniaturized optics, the display can resolve pixel-scale details over the whole field of view (right image) —\u00a0<\/em>note in the image that\u00a0the line width of the text and the spacing between the dot of the letter i<\/em> are only\u00a0one pixel. We find promise in this early prototype but note that\u00a0the display is\u00a0monoscopic and the driving electronics are external; significant challenges remain to build a practical stereo display.<\/p>\n

 <\/p>\n

\"\"<\/p>\n

 <\/p>\n

Video<\/h2>\n