Well probably not literally. But close!Virtual reality (VR) is no longer the sci-fi dream promised to many for decades, but never quite delivered. In fact, 2016 could even be the year of VR. CNN Money, among others, named it one of its only two biggest tech trends this year<a href="https://www.creaform3d.com/blog/virtual-reality-is-at-our-doorstep/#_ftn1" name="_ftnref1" target="_blank">[1]</a>. &nbsp;VR a slow start, but a really fast pickup!<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1592469622183-girl-testing-occulus-rift-virtual-reality-headset.jpg" style="width: 788px;" class="fr-fic fr-dib">VirtualReality is nothing new, of course, but let’s just say it has seen its shares of… false starts. In the early 1990s, the technology was already touted as “the next technology revolution,” as this (now quite funny in hindsight) <a href="https://www.youtube.com/watch?v=rVn3H93Ysag" target="_blank" title="Virtual Reality 1991">1991 American prime time TV report</a> attests. But it just never happened. Why? Prohibitive costs (the 1990 <a href="https://www.youtube.com/watch?v=ACeoMNux_AU" target="_blank" title="Virtual Reality from 1990, Jaron Lanier, Eye phones">EyePhone</a> VR headset device could cost up to $500K), clunky technology, idea way ahead of its time? Probably a mix of all that.What changed everything in Virtual reality, obviously, was Oculus. The company, founded in 2012, launched a Kickstarter campaign shortly after to fund its Oculus Rift VR headset. It garnered about $2.5 million in financing, as well as a lot of attention. Enough for Facebook to acquire the company in 2014, for $2 billion. After much speculation, Oculus VR development kits and prototypes, the “consumer version” of the virtual reality headset <a href="https://www.oculus.com/en-us/" target="_blank">Oculus Rift</a> is finally launching this spring.Explains Linda Jacobson, founding editor at Wired Magazine and virtual reality expert: “What’s new is this particular set of products at a new price point, as well as the availability of new people and new talent who are looking at it<a href="https://www.creaform3d.com/blog/virtual-reality-is-at-our-doorstep/#_ftn2" name="_ftnref2" target="_blank">[2]</a>.” In short, the current VR revival could be attributed to both its more “democratic” (relatively speaking) price, as well as to the presence of a whole new generation looking at it with fresh eyes. What the new buzz hints, is that maybe this time, VR could become a genuine mainstream thing.<h3>A multiplayer game…</h3><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1592469745491-virtual-reality-headset.jpg" class="fr-fic fr-dib" style="width: 788px;">Oculus is far from the only player in this VR arena; other major companies are currently bringing their own devices forward. HTC just released its more upscale <a href="https://www.htcvive.com/ca/" target="_blank">HTC Vive</a>, while Sony plans to launch its <a href="https://www.playstation.com/en-us/explore/playstation-vr/" target="_blank" title="PlayStation®VR">Play Station VR</a> later this year. A Canadian company is also vying for market share: startup VRvana, from Montreal, is expected to ship its <a href="https://www.vrvana.com/license-totem-technology/" target="_blank" title="Totem technology">Totem</a> sometime this summer. Competition is heating up and users are likely to benefit from that in the mid-term.<h3>…And a perfectly suited 3D scanning application</h3>Sure, first and foremost, Virtual Reality seems to be destined to go hand in hand with the gaming industry. Gaming of all kinds will be its most expected application. But it doesn’t stop there! Far from it. According to Tech Republic<a href="https://www.creaform3d.com/blog/virtual-reality-is-at-our-doorstep/#_ftn3" name="_ftnref3" target="_blank">[3]</a>, it’s easy “to see VR establishing itself more firmly in industries like healthcare, education, tourism, real estate, and marketing, where the uses are more practical than fantastical. Marketing in particular could end up having a big impact on the way many people might first encounter virtual reality through experiences like traveling installations or branded apps. And for agencies, Virtual Reality is already becoming a point of interest”. All of these applications potentially broaden the reach of VR in the next few years, while opening up many opportunities.What this exciting new trend means, of course, is that the need for quality 3D content has never been this high. To feed this explosion of VR devices and experiences, 3D scanning, along with a “3D vision”, is going to be key. And we at Creaform, along with our&nbsp;<a href="https://www.goscan3d.com/en" target="_blank">Go!SCAN 3D scanner</a>, are&nbsp;<a href="https://www.goscan3d.com/en/applications/serious-gaming-virtual-reality" target="_blank" title="Serious Gaming & Virtual Reality">perfectly ready for the ride</a>.We can’t wait to see how this unfolds.

Virtual reality (VR) is no longer the sci-fi dream promised to many for decades, but never quite delivered. In fact, 2016 could even be the year of VR. CNN Money, among others, named it one of its only two biggest tech trends this year

Virtual Reality Is At Our Doorstep!

<h1 data-selectable-paragraph="">Optics</h1>Before looking at (pun intended) the fundamentals of optics, it is important to understand the basic properties of the human eye.<h2 data-selectable-paragraph="">Basic properties of the human eye</h2>Field of View (FOV) It is defined as the total angular size of the image visible to both the eyes. On an average, the horizontal binocular FOV is 200 deg out of which 120 deg is a binocular overlap. The binocular overlap is especially important for stereopsis and other depth cues discussed further. The vertical FOV is approximately 130 deg.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574118639-1+GAdxViPw-qozgCxibr5CHQ.jpeg" style="width: 411px;" class="fr-fic fr-dib">Inter-pupillary distance (IPD) As the name suggests, it is the distance between the pupils of the eyes and is an extremely important consideration for binocular viewing systems. This distance varies from person to person, by gender and ethnicity. An inaccurate IPD consideration can result in poor eye-lens alignment, image distortion, strain on the eyes and headache. Mean IPD for adults is around 63mm with majority in the 50&ndash;75mm range. The minimum IPD for children is around 40mm.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574150453-1+CgdRmFurVj4pnkvTZ2_HqA.jpeg" style="width: 386px;" class="fr-fic fr-dib"><section>Eye relief This is the distance from the cornea of the eye to the surface of the first optical element. It defines the distance at which the user can obtain full viewing angles. This is an important consideration especially for people who wear corrective lenses or spectacles. Eye relief for spectacles is approximately 12mm. Enabling users to adjust the eye relief is extremely important for head mounted displays.Exit pupil This is the diameter of light transmitted to the eye by an optical system.Eye box This is the volume within which users can place their pupils to experience the visuals wholly.<hr><h2 data-selectable-paragraph="">Ocularity</h2>Ocularity is the measure of the number of eyes needed to see something. Since most of us have a maximum of two eyes, the limit of ocularity in smart displays is likewise.Monocular display This display provides a single channel for viewing often through a small display element and lens. The channel is positioned in front of one eye where the user is free to view the real world completely through another eye. Monocular displays are often used as information displays due to their small form factor. However, these type of displays provide no stereo depth cues and often result in very low contrast. Google Glass is a type of monocular display.Biocular This type provides a single viewing channel to both eyes by means of internal reflections. Biocular displays lack stereopsis and are suitable for close proximity tasks.Binocular Each eye gets a separate view in these type of displays creating a stereoscopic view. These display types provide the most depth cues and a sense of immersion however, they are the heaviest, most complex and computationally intensive displays.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574196324-1+Arh2uQ4zZKNjd4qDqtftGw.jpeg" style="width: 766px;" class="fr-fic fr-dib"><h2 data-selectable-paragraph="">Optical architectures</h2>Optics in smart glasses serve three main purposes:<ul><li data-selectable-paragraph="">Collimation of light such that the image appears at a greater distance than it&rsquo;s physical distance.</li><li data-selectable-paragraph="">Magnification of the display image to make it appear larger than it&rsquo;s actual size.</li><li data-selectable-paragraph="">Relaying of light patterns to the viewers eyes.</li></ul><h2 data-selectable-paragraph="">Distortion</h2>There are two primary optical design systems, or architectures for AR and VR displays: pupil forming and non-pupil forming.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574226538-1+qYs6v5YGAvOli3CLkHp-8w.jpeg" style="width: 387px;" class="fr-fic fr-dib">Non-pupil forming architecture These consist of a single lens and are commonly seen in popular immersive displays such as HTC Vive, Oculus Rift and Sony PSVR. This type of architecture uses a single magnifier to directly collimate light from the display panel.Pupil forming architectures The non-pupil forming architectures result in lighter and more compact designs with a large eye box, however they create a significant distortion when bending the light field. This effect is known as pincushion distortion. In pupil forming architectures, another lens that produces a barrel distortion is used to nullify the effect. These are often used in non-immersive type displays such as Microsoft&rsquo;s Hololens and Google glass.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574249873-1+LCbiIff3SAG5fTdRsK2SWA.jpeg" style="width: 610px;" class="fr-fic fr-dib"><h2 data-selectable-paragraph="">Waveguides</h2>A waveguide as the name suggests is a physical structure in optics that guides a light wave to the user&rsquo;s eye. This is done by means of internal reflection and the contraption controls the movement of light between entry and exit. There are four types of waveguides used in industry:Holographic waveguide This is a fairly simple type of wave-guide with optical elements like lenses used for in-coupling (entry) and out-coupling (exit) through a series of internal reflections. This type of waveguide is used in Sony&rsquo;s Smart Eyeglass displays.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574269214-1+OhVeiQf8Vamg86d3IeY_iw.jpeg" style="width: 575px;" class="fr-fic fr-dib">Diffractive waveguide Precise surface relief gratings are used to achieve internal reflections for a seamless overlay of 3D graphics through the display. These waveguides are used in a number of Vuzix displays and Microsoft&rsquo;s Hololens.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574290271-1+rBmrm_e4MLllgW9dNmbWdA.jpeg" style="width: 503px;" class="fr-fic fr-dib">Polarized waveguide Light enters the waveguide and through a series of internal reflection on a partially reflective polarized surface. Selected light waves cancel out (polarization) exiting into the viewer&rsquo;s eye. The method is used by the Lumus DK-50 AR glasses.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574314743-1+qxRuMvKlRjfDz-J-OJ87yw.jpeg" style="width: 471px;" class="fr-fic fr-dib">Reflective waveguide This is similar to the holographic waveguide in which a single planar light guide is used with one or more semi-reflective mirrors. This type of waveguide can be seen in Epson&rsquo;s Moverio as well as Google Glass.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574344210-1+5anYIQsARNckMe2wbXGy9g.jpeg" style="width: 478px;" class="fr-fic fr-dib"><hr><h1 data-selectable-paragraph=""> </h1><h1 data-selectable-paragraph="">Display technologies</h1><h2 data-selectable-paragraph="">Display types</h2>Fully immersive These are standard fully immersive virtual reality displays. These stereoscopic displays are combined with sensors to track position and orientation. They completely block the user&rsquo;s view of the outside world like in the book &lsquo;Ready Player One&rsquo;.Optical see through In Optical see through glasses, the user views reality directly through optical elements such as holographic wave guides and other systems that enable graphical overlay on the real world. Microsoft&rsquo;s Hololens, Magic Leap One and the Google Glass are recent examples of optical see through smart glasses.Video see through With these type of smart glasses, the user views reality that is first captured by one or two cameras mounted on the display. These camera views are then combined with computer generated imagery for the user to see. The HTC Vive VR headset has an inbuilt camera which is often used for creating AR experiences on the device.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574377342-1+YFYHXnON9H_IMUsyG7SgRQ.jpeg" style="width: 766px;" class="fr-fic fr-dib"><h2 data-selectable-paragraph="">Imaging technologies</h2>Imaging and display technologies have improved greatly in the past few decades. High end CRTs have been mostly replaced by four key imaging technologies: Liquid Crystal Displays (LCD) LCDs are common in high definition televisions and have been used in ARVR displays since 1980s. This display type consists of an array of cells containing liquid crystal molecules sandwiched between two polarizing sheets. This contraption rests between thin glass substrates printed with millions of transistors. For colored LCDs, an additional substrate containing red, green and blue filters is positioned over each cell of the substrate. A single RGB liquid crystal cell is called a subpixel. Three subpixels form one pixel. A electric current is passed through the glass substrates. Varying the current allows the LCD to modulate the passage of light to create a precise color. If all subpixels are fully open, it creates a white light. Liquid crystal cells do not emit their own light and require backlighting. The liquid crystal cells can only vary the passage of light to create the desired color and subsequently an image.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574432301-1+9vG6865Nf6odfuEe2XY8aw.jpeg" style="width: 300px;" class="fr-fic fr-dib">Organic Light Emitting Diode (OLED) This display technology is based on organic (carbon and hydrogen bonded) materials that emit light when an electric current is applied. This is a solid-state display technology where energy passed through the organic sheet is released in the form of light, also known as electroluminescence. Colors can be controlled by carefully crating organic emission, however most manufacturers add red, green and blue films in the OLED stack. There are two types of OLED panels:<ol><li data-selectable-paragraph="">Passive Matrix OLED (PMOLED): Like CRTs, this display type consists of a complex electronic grid to sequentially control individual pixels in each row. It does not contain storage capacitors making update rates slow and a high power consumption to maintain a pixel&rsquo;s state. These are mainly used for simple character and iconic displays. </li><li data-selectable-paragraph="">Active Matrix OLED (AMOLED): Unlike PMOLEDs, AMOLEDs consist of a thin transistor layer that contains a storage capacitor to maintain each subpixel&rsquo;s state providing greater control over individual pixels. In case on AMOLEDs, individual pixels can be completely switched off enabling deeper blacks and higher contrast. These are most suitable display types for near-eye virtual and augmented reality devices.</li></ol><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574474643-1+2XVFSENmFYaIADzTMvSHQg.jpeg" style="width: 627px;" class="fr-fic fr-dib">OLEDs and AMOLEDs in particular are far superior to LCDs. Their construction is relatively simpler and they can be extremely thin since there is no need of external backlighting. In addition to this they consume significantly less power, have faster refresh rates, high contrast, great color reproduction and higher resolutions. Most fully immersive HMDs utilize this technology.Digital Light Projector (DLP) Microdisplay Originally developed by Texas Instruments, DLP chip is also referred to as a Digital micromirror device (DMD). The display consists of about 2 million individually controlled micromirrors each which can be used to represent a single pixel. Each of these micromirrors measure approximately 5.4 microns. What is interesting about these displays is that the retina of the eye itself serves as a display surface. RGB light is reflected on these micromirrors which tilt towards and away from the light source. As each micromirror can be reoriented in either direction thousands of times in a second, varying the reflected color can produce different shades of light on the retina.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574499211-1+GVbgkSYCxhd-SGJoXH636g.jpeg" style="width: 766px;" class="fr-fic fr-dib">DLP Microdisplays are one of the fastest display technologies in existence. Their speed of color refresh, low latency, low power consumption and extreme high resolutions (0.3 inch array diagonal enables a 1280 x 720 image) make them ideal candidates for building head-mounted displays.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574516762-1+oHOQFhzooGQFxhGdaXHdNw.jpeg"><figcaption data-selectable-paragraph="">Scale of DLP Microdisplays</figcaption> Liquid Crystal on Silicon (LCoS) Microdisplay LCoS Displays lie somewhere in between LCD and DLP displays. LCD is a transmissive technology where the image is generated and transmitted to the user while DLP is a reflective technology where individual subpixels are reflected through micromirrors. Light source is passed onto a reflective surface. As the light reflects, it passes through a series of sub filters that modulate the light intensity and color. Similar to DLP displays, their small size enables considerable flexibility when integrating with small form factor devices. Microsoft&rsquo;s Hololens, Google Glass and even the Magic Leap One uses an implementation of LCoS Microdisplays.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1566574547646-1+1__J0RNbQgXuDhPwu6R2Ew.jpeg"><figcaption data-selectable-paragraph="">LCoS modulates light from an external source to produce the image</figcaption>Given the extreme resolutions of display technologies in development, it is almost certain that flat panel based HMDs might become a thing of the past for AR devices.<section>References: 1. <a href="https://amzn.to/2C7k3c3" target="_blank">Augmented Human &mdash; Helen Papagiannis</a> 2. <a href="https://amzn.to/2C6XDHR" target="_blank">Practical Augmented Reality &mdash; Steve Aukstakalnis</a></section><hr></section>

Head-mounted devices for virtual and augmented reality come in different shapes and sizes from the minimal Google Glass to the fully immersive HTC vive. At their core, head-mounted displays consist of two primary structural elements, optics and image displays.

Fundamentals of display technologies for AR and VR

Head-mounted devices for virtual and augmented reality come in different shapes and sizes from the minimal Google Glass to the fully immersive HTC vive. At it’s core, head-mounted displays (HMDs) consist of two primary structural elements: optics and image displays.

Display technologies for Augmented and Virtual Reality

Creating and modifying a virtual reality environment just got a lot easier thanks to software being released today by Imverse, an EPFL spin-off. The secret behind Imverse&rsquo;s program, which works much like a photo editor, is a three-dimensional rendering engine based on 3D pixels called voxels. The rendering engine can be used for other virtual reality applications as well, such as depicting real people. The startup&rsquo;s initial target market is the movie and video game industry.&nbsp;<iframe src="https://www.youtube.com/embed/dCYUgPfcBTc?&wmode=opaque" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 788px; height: 520px;"></iframe>Wearing a virtual reality headset and using manual controllers, you can use Imverse&rsquo;s new software to select tools that let you bring depth, cut, paste, paint and zoom in and out &ndash; much like in a photo editor. Your virtual environment can therefore respond to your creative flourishes, even from a 2D or 360 photo: in an instant, your surroundings can change as you push back walls, create space here and there, add furniture and experiment with colors. The beta version of this software program will be available online starting today.<h2>Creating a 3D scenario ten times faster</h2>Imverse&rsquo;s time-saving technology could come in handy in a wide range of fields, such as real estate, architects, graphic designers, decorators and photgraphers &ndash; engineers could even use it for certain kinds of modeling. But the EPFL spin-off is initially setting its sights on the movie, medias and video game industry, where its software could be used for such tasks as sketching out scenarios in 3D, previewing movie sets and how they could change during a film and mapping out the locations of cameras and actors. &ldquo;Producers can use it to take a photo of a real set, convert the 360 degrees image into a 3D environment and then, using virtual reality, edit it in order to visualize the scene before they film it. And they&rsquo;ll be able to do these things ten times faster than with conventional programs,&rdquo; says Javier Bello, the startup&rsquo;s co-founder and CEO.Setting its sights on the movie industry from the start may seem ambitious, but the company has done its homework. Imverse&rsquo;s founders have already begun feeling out the industry and making contacts, including at Sundance &ndash; the largest independent film festival in the US &ndash; early this year, and then later at the Cannes Film Festival. &ldquo;Our first target market is the entertainment industry, because people who work there are already familiar with the types of software used for visual effects and motion capture. They can quickly grasp how our program can simplify their work and save them time,&rdquo; adds Bello. What&rsquo;s more, going after the movie and game industry will put them into contact with heavyweights like Intel, Microsoft and Oculus.<h2>A voxel-based, high-potential technology</h2>Imverse is also aiming high for another reason: the technology on which its software is based can be used for many other applications. The company&rsquo;s powerful voxel-based rendering engine obviates the need for those notorious polygon grids so familiar to video-game and animated-film makers. Voxels are not new, but until recently they required too much computing power to be feasible for real-time rendering.&ldquo;This technology was developed over the past 12 years at EPFL&rsquo;s Laboratory of Cognitive Neuroscience, for use in neurology research. It lets us create smarter tools, processes and approaches for connecting hardware and software,&rdquo; says Bello, a virtual reality expert. Bello, together with colleague and co-founder Robin Mange, sees broader implications &ndash; and thus a particularly bright future &ndash; for their technology. &ldquo;And not just in virtual reality &ndash; we started with that field to show the high level of precision it can attain,&rdquo; he adds.<h2>Shaking a friend&rsquo;s hand in the real and virtual worlds at the same time</h2>The startup is also hard at work on bringing real-time representations of users&rsquo; bodies &ndash; and of other people around &ndash; into the virtual reality environment. Their mixed-reality technology has already been used by an artist to make a short film called&nbsp;<a href="https://vimeo.com/256589715" rel="noopener" target="_blank">Elastic Time</a>. They showcased their system at Sundance, giving industry professionals a chance to try it out. &ldquo;The ability for people to interact in real time in both the real and virtual worlds will lead to ever more possibilities,&rdquo; concludes Bello.

Creating and modifying a virtual reality environment just got a lot easier thanks to software being released today by Imverse, an EPFL spin-off. The secret behind Imverse’s program, which works much like a photo editor, is a three-dimensional rendering engine based on 3D pixels called voxels.

Modifying a virtual environment in just a few clicks

Researchers at the <a href="http://seas.harvard.edu/" target="_blank">Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS)</a> have developed a flat optical component that is simultaneously a metalens, a microscope objective that can resolve details smaller than a wavelength of light, and an optical vortex and hologram generator. Each functionality is controlled by a different wavelength of light.&ldquo;The breakthrough of this new flat optical device is that it can radically change its function based on the wavelength of light it reflects,&rdquo; said <a href="https://www.seas.harvard.edu/directory/capasso" target="_blank">Federico Capasso</a>, the Robert Wallace Professor of Applied Physics at SEAS and senior author of the research. &ldquo;By tying functionality to wavelength, we have opened up a whole range of new possibilities for metasurfaces.&rdquo;The research was published in <a href="https://pubs.acs.org/doi/10.1021/acs.nanolett.7b05458" target="_blank">Nano Letters</a>.&ldquo;In this research, we decoupled functions at different wavelengths,&rdquo; said Zhujun Shi, first author of the paper and graduate student at SEAS. &nbsp;&ldquo;Compared to previous flat optical devices, this device has an extra degree of freedom that you can tune at different wavelengths. For example, at one color, this lens behaves like a traditional metalens but at another wavelength, it generates a vortex beam.&rdquo;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1577104033804-shi_stedWEB.jpg">Wavelength-controlled beam generator, light is focused (l = 0) at red and green, and forms a donut shape at blue (Capasso Lab/Harvard SEAS)&nbsp;<a href="https://otd.harvard.edu/" target="_blank">The Harvard Office of Technology Development</a> has protected the intellectual property relating to this project and is exploring commercialization opportunities.The lens builds on previous technology developed in the Capasso Lab, which used different polarized light to change the function of a lens. But since there are only two forms of circularly polarized light &mdash; &nbsp;clockwise or counterclockwise &mdash; the researchers could embed only two different functions in the metasurface.&ldquo;By controlling the device function with wavelength, rather than polarization that is bound to two states, we have dramatically increased the information capacity of the lens,&rdquo; said Mohammadreza Khorasaninejad, co-first author of the paper and former postdoctoral fellow in the Capasso Lab. &ldquo;With this technology, we demonstrated an achromatic metalens in blue, green, yellow and red wavelengths, two beam generators, and a full-color hologram.&rdquo;By tying functionality to wavelength, we have opened up a whole range of new possibilities for metasurfaces.While this is not the first lens to tie function to wavelength, it is the most efficient. Previous wavelength-dependent metalenses encoded different functions in different areas of the surface; for example, red light would be focused in one quadrant and blue light in another.With this technology, Shi and the rest of the team engineered the individual nanoscale optical elements to embed functionality at the local level, across the entire lens.&ldquo;By encoding everything locally, in a single layer, we improved efficiency from 8 percent demonstrated in previous wavelength-dependent metasurfaces to more than 30 percent,&rdquo; said Yao-Wei Huang, co-first author of the paper and postdoctoral fellow at SEAS.Next, the team aims to improve that efficiency even further and develop a transmitting, rather than reflective lens.

Schematic of multiwavelength achromatic metalens. This multifunctional metalens can focus four different wavelengths of light (R, Y, G, and B) at the same distance and generate wavelength-controlled vortex beams (B,C). (Image courtesy of the Capasso Lab/Harvard SEAS)

Flat optical device has applications in augmented and virtual reality, ultra-high resolution microscopy, holography

Multifunctional metalens unlocks with light

A new camera that builds on technology first described by Stanford researchers more than 20 years ago could generate the kind of information-rich images that robots need to navigate the world.This camera, which generates a four dimensional image, can also capture nearly 140 degrees of information.&ldquo;We want to consider what would be the right camera for a robot that drives or delivers packages by air. We&rsquo;re great at making cameras for humans but do robots need to see the way humans do? Probably not,&rdquo; said&nbsp;Donald Dansereau, a postdoctoral fellow in electrical engineering.With robotics in mind, Dansereau and <a href="https://profiles.stanford.edu/gordon-wetzstein" target="_blank">Gordon Wetzstein</a>, assistant professor of electrical engineering, along with colleagues from the University of California, San Diego have created the first-ever single-lens, wide field of view, light field camera, which they are <a href="https://mandrillapp.com/track/click/30855492/www.computationalimaging.org?p=eyJzIjoiaWtSRzRKNlNpUjFTX0NsNWt5T1ZaWXNxNVNBIiwidiI6MSwicCI6IntcInVcIjozMDg1NTQ5MixcInZcIjoxLFwidXJsXCI6XCJodHRwOlxcXC9cXFwvd3d3LmNvbXB1dGF0aW9uYWxpbWFnaW5nLm9yZ1xcXC93cC1jb250ZW50XFxcL3VwbG9hZHNcXFwvMjAxN1xcXC8wNFxcXC9MRk1vbm9jZW50cmljLnBkZlwiLFwiaWRcIjpcIjFkYjRkMmJhZmRhMzQ1Zjg4MGQ0NWUyMzFhZGFjY2Y3XCIsXCJ1cmxfaWRzXCI6W1wiNTM0YmE4ZWJlZGQxYmQwYzg3ZjY2OWI0Y2UzZmY1YTM5YjkxODE2OFwiXX0ifQ" rel="nofollow" target="_blank">presenting</a> at the computer vision conference CVPR 2017 on July 23.As technology stands now, robots have to move around, gathering different perspectives, if they want to understand certain aspects of their environment, such as movement and material composition of different objects. This camera could allow them to gather much the same information in a single image. The researchers also see this being used in autonomous vehicles and augmented and virtual reality technologies.&ldquo;It&rsquo;s at the core of our field of computational photography,&rdquo; said Wetzstein. &rdquo;It&rsquo;s a convergence of algorithms and optics that&rsquo;s facilitating unprecedented imaging systems.&rdquo;<h2>From a peephole to a window</h2>The difference between looking through a normal camera and the new design is like the difference between looking through a peephole and a window, the scientists said.<blockquote>&ldquo;A 2D photo is like a peephole because you can&rsquo;t move your head around to gain more information about depth, translucency or light scattering,&rdquo; Dansereau said. </blockquote>&ldquo;Looking through a window, you can move and, as a result, identify features like shape, transparency and shininess,&rdquo;&nbsp;Dansereau said.That additional information comes from a type of photography called light field photography, <a href="https://mandrillapp.com/track/click/30855492/graphics.stanford.edu?p=eyJzIjoiOEhYRHAwTERPVTh2YnZrdTF5enFqVkhEOE1jIiwidiI6MSwicCI6IntcInVcIjozMDg1NTQ5MixcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL2dyYXBoaWNzLnN0YW5mb3JkLmVkdVxcXC9wYXBlcnNcXFwvbGlnaHRcXFwvbGlnaHQtbG9yZXMtY29ycmVjdGVkLnBkZlwiLFwiaWRcIjpcIjFkYjRkMmJhZmRhMzQ1Zjg4MGQ0NWUyMzFhZGFjY2Y3XCIsXCJ1cmxfaWRzXCI6W1wiMDBjOTc1NWI4N2JjZGU4MzE4YzlkZjBhYWExZTRmODEwNWFmZWZjMFwiXX0ifQ" target="_blank">first described</a> in 1996 by Stanford professors <a href="https://mandrillapp.com/track/click/30855492/profiles.stanford.edu?p=eyJzIjoiZUthNldVYm43UlNpTGRfb3hSR1dXQU44Y013IiwidiI6MSwicCI6IntcInVcIjozMDg1NTQ5MixcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3Byb2ZpbGVzLnN0YW5mb3JkLmVkdVxcXC9tYXJjLWxldm95XCIsXCJpZFwiOlwiMWRiNGQyYmFmZGEzNDVmODgwZDQ1ZTIzMWFkYWNjZjdcIixcInVybF9pZHNcIjpbXCIzNjA1OTMyNjc0ZDU4YTgyMTAwYTk1Mzg2OGE2MjE1ZTBhZDBiMTQwXCJdfSJ9" target="_blank">Marc Levoy</a> and <a href="https://mandrillapp.com/track/click/30855492/profiles.stanford.edu?p=eyJzIjoiNWpkcVJVS3pnSjRVeUJsTmtQMDZJYWl1aF80IiwidiI6MSwicCI6IntcInVcIjozMDg1NTQ5MixcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3Byb2ZpbGVzLnN0YW5mb3JkLmVkdVxcXC9wYXRyaWNrLWhhbnJhaGFuXCIsXCJpZFwiOlwiMWRiNGQyYmFmZGEzNDVmODgwZDQ1ZTIzMWFkYWNjZjdcIixcInVybF9pZHNcIjpbXCI4OTExMmJiZGMyZWQ1Mzc3NzQ2ZDBiNGNlYzdkZDAxNzAyZTkzM2JmXCJdfSJ9" target="_blank">Pat Hanrahan</a>. Light field photography captures the same image as a conventional 2D camera plus information about the direction and distance of the light hitting the lens, creating what&rsquo;s known as a 4D image. A well-known feature of light field photography is that it allows users to refocus images after they are taken because the images include information about the light position and direction. Robots might use this to see through rain and other things that could obscure their vision.The extremely wide field of view, which encompasses nearly a third of the circle around the camera, comes from a specially designed spherical lens. However, this lens also produced a significant hurdle: how to translate a spherical image onto a flat sensor. Previous approaches to solving this problem had been heavy and error prone, but combining the optics and fabrication expertise of UCSD and the signal processing and algorithmic expertise of <a href="https://mandrillapp.com/track/click/30855492/www.computationalimaging.org?p=eyJzIjoiTEdXQjBaYm96OHkwMGpsckRFc0lvRW5jMi04IiwidiI6MSwicCI6IntcInVcIjozMDg1NTQ5MixcInZcIjoxLFwidXJsXCI6XCJodHRwOlxcXC9cXFwvd3d3LmNvbXB1dGF0aW9uYWxpbWFnaW5nLm9yZ1xcXC9cIixcImlkXCI6XCIxZGI0ZDJiYWZkYTM0NWY4ODBkNDVlMjMxYWRhY2NmN1wiLFwidXJsX2lkc1wiOltcIjY5Y2VhNjUyYmY0YTU1YTU3ZmQ3M2JhZDgzOTYwMTk3OGQ3MGFkNjBcIl19In0" rel="nofollow" target="_blank">Wetzstein&rsquo;s lab</a> resulted in a digital solution to this problem that not only leads to the creation of these extra-wide images but enhances them.<h2>Robotics up close</h2>This camera system&rsquo;s wide field of view, detailed depth information and potential compact size are all desirable features for imaging systems incorporated in wearables, robotics, autonomous vehicles and augmented and virtual reality.&ldquo;It could enable various types of artificially intelligent technology to understand how far away objects are, whether they&rsquo;re moving and what they&rsquo;ve made of,&rdquo; said Wetzstein. &ldquo;This system could be helpful in any situation where you have limited space and you want the computer to understand the entire world around it.&rdquo;Although it can also work like a conventional camera at far distances, this camera is designed to improve close-up images. Examples where it would be particularly useful include robots that have to navigate through small areas, landing drones and self-driving cars. As part of an augmented or virtual reality system, its depth information could result in more seamless renderings of real scenes and support better integration between those scenes and virtual components.The camera is currently a proof-of-concept and the team is planning to create a compact prototype next. That version would hopefully be small enough and light enough to test on a robot. A camera that humans could wear could be soon to follow.

The extremely wide field of view comes from a specially designed spherical lens. I Photo courtesy of Donald Dansereau and Gordon Wetzstein

A team of engineers developed a 4D camera with an extra-wide field of view, which they believe will be better for close-up robotic vision and augmented reality.

virtual reality (VR)

ORGANIZATIONS. SHAPING THE INDUSTRY.

Oversight

Immersal

Hololight

Keysight Design Engineering Software

Premo Group

Latest Posts

Virtual Reality Is At Our Doorstep!

Fundamentals of display technologies for AR and VR

Display technologies for Augmented and Virtual Reality

Modifying a virtual environment in just a few clicks

Multifunctional metalens unlocks with light

A new kind of camera could improve robot vision and virtual reality