<p>Engineers build the world. But at what cost to the world?</p><p><a href="https://massless.dev" rel="noopener noreferrer" target="_blank"><em>Massless&nbsp;</em></a>offers a framework to accelerate innovation and decelerate climate change <em>at the same time</em>. We don&#39;t all need to hide in a cave. Quite the opposite. We need to be leveraging every technology available to change our approach to product design, reconfigure our global manufacturing supply chains, and engage the power of consumer choice and education to change the culture of commerce.</p><p>Engineers have tremendous leverage in our race to curb the most disastrous impacts of climate change. Obviously, a young engineer can go work for a solar company, or a battery startup, or design the next generation of carbon capture. What is less obvious is that engineers can have a tremendous impact on climate by re-imagining how <em>anything</em> is built. E=mc<sup>2</sup>. Mass is energy. Energy is mass. We need to be applying the principles of sustainability to <em>every</em> design. The planet cannot wait.</p><p>The <a href="https://massless.dev" rel="noopener noreferrer" target="_blank"><em>massless mission</em></a> is to reduce global energy usage by 25% by the year 2050, leveraging the power of distributed digital manufacturing. This is a mammoth target, and it will require deep changes to our entire global manufacturing ecosystem, but it is possible. We selected this target based on modeling done in a <a href="https://www.sciencedirect.com/science/article/pii/S0301421517306997" rel="noopener noreferrer" target="_blank">2018 research paper published in Energy Policy</a>, where the authors show different scenarios for how digital manufacturing can impact global energy usage. In the different scenarios, they model the impact to be between 5-27% by 2050. <a href="https://massless.dev" rel="noopener noreferrer" target="_blank"><em>massless</em></a> is the mission to achieve the top end of that range. To do that, we need to change the culture.</p><p>In this article, I am going to outline the four massless principles that every engineer can begin to apply immediately to any and every project that they are currently working on.<br><strong><br>1. Massless.&nbsp;</strong></p><p>Thanks to Moore&#39;s Law, compute power is cheap, abundant and energy-efficient. Bill Gross calls this &quot;bits over atoms&quot;. If you don&#39;t need to make a physical object, don&#39;t. Leverage the power of advanced simulation and augmented reality to work out the kinks before you build. polySpectra has created a free tool, <a href="https://ar.polyspectra.com" rel="noopener noreferrer" target="_blank"><strong>polySpectra AR</strong></a>, that allows anyone to instantly view their CAD files in augmented reality, in their own workspace on their own phone or tablet, without needing to download any app or install any special software. Our hope is that designers and engineers will use this tool for their &quot;looks like&quot; prototypes, which would have otherwise ended up in a landfill.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 414px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NjU0MzA2MTkyLWFyLWJyYWNrZXQtb24tZGVzazEyMDBfNjMwLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 412px;" class="fr-fic fr-dib"><span class="fr-inner">Massless prototyping, courtesy https://ar.polyspectra.com</span></span></span><strong><br>2. Less mass.&nbsp;</strong></p><p>30% of US energy is used to move people and stuff around. According to the US Department of Energy, reducing the weight of a car by 10% can save roughly 7% on fuel. Engineers need to be leveraging new advancements in topology optimization, lightweighting and advanced materials to create the lightest weight component that will successfully meet the design criteria. To help engineers with lightweighting, we recently created <a href="https://www.youtube.com/watch?v=wJGujqEB8Pw" rel="noopener noreferrer" target="_blank">a tutorial on lightweighting for resin printing</a>, which includes a free nTopology notebook for any engineer to apply to their project.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/wJGujqEB8Pw?&wmode=opaque&rel=0" allowfullscreen="" class="fr-draggable" width="640" height="360" frameborder="0"></iframe></span><strong><br>3. Less waste.&nbsp;</strong></p><p>polySpectra&#39;s DLP-based additive manufacturing process uses about <a href="https://medium.com/polyspectra-wavefront/why-3d-printing-is-a-critical-part-of-the-battle-against-climate-change-726e493a3f58" rel="noopener noreferrer" target="_blank">50x less energy than an injection molding press</a>. Additive manufacturing is capable of printing the precise mass, in the precise geometry, and in the precise quantity needed - with little to no waste. Most consumers are unaware of the carbon footprint of the products that they buy, these wasteful externalities are not visible or rationally priced. Today, it is too cheap for companies to create tons of trash. We have become addicted to the &quot;infinite shelf space&quot; of online retail. But the planet is not infinite. What if the product didn&#39;t exist until it was ordered? And then only the number of units that were ordered were produced? Consumers can still experience abundance, choice, prosperity - and we can do it without wrecking the planet.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NjU1MDk1MTI4LXJlc2luLXByaW50LXdpZGUtc21hbGxlci5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 543px;" class="fr-fic fr-dib"></p><p><strong>4. Less movement.&nbsp;</strong></p><p>I&#39;ll say it again: 30% of US energy is used to move people and stuff around. Digital manufacturing can have a tremendous energy impact by providing a lower carbon footprint alternative to traditional manufacturing. It can have a huge energy impact by enabling the next generation of light-weight transportation vehicles. But the really massive impact comes from &quot;bits over atoms&quot; - instead of sending components halfway around the world 4 times before the product is fully assembled (<em>hint: your smartphone is made this way</em>), we will send files over the internet, to be digitally manufactured regionally or even locally. Even if we could just shift our manufacturing ecosystem to have products manufactured on the <em>same continent</em> as their end-user, we could massively cut the energy intensity of our built world.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/y3qfeoqErtY?&feature=emb_title&wmode=opaque&rel=0" allowfullscreen="" class="fr-draggable" width="640" height="360" frameborder="0"></iframe></span><br></p><p>To achieve the <a href="https://massless.dev" rel="noopener noreferrer" target="_blank"><em>massless mission</em></a>, we will need every tool in the toolbox: augmented reality, design for additive manufacturing, topology optimization, education, customer co-creation, metamaterials, virtual inventories, advanced automation...the possibilities are endless. We will also need as many innovators as possible.<br><br>We sincerely hope that you will join us on this mission. We certainly can&#39;t do it alone. As engineers - you have much more power than you might imagine. You are building the world. Are you going to build it with intention? Could you bring the massless mindset into your workflow today?</p><p>Go massless today =&gt; <a href="https://massless.dev" id="isPasted" rel="noopener noreferrer" target="_blank">https://massless.dev</a><br><br><br><br></p>

1. massless 2. less mass 3. less waste 4. less movement

Engineers have tremendous leverage in humanity's race to curb the most disastrous impacts of climate change. Massless offers a framework to accelerate innovation and decelerate climate change at the same time.

Engineering with a Massless Mindset

<p id="isPasted">David Shapira and Kordian Caplazi have three things in common: they are young, have an ETH degree in their pockets and knew before they graduated that they wanted to become self-employed. With the founding of Rimon Technologies in August 2020, they have fulfilled their dream of starting their own company The start-up is so successful that four more members joined the team last year.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 766px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ1NDM5MzI5ODI3LWltYWdlLmltYWdlZm9ybWF0LmZ1bGx3aWR0aC44MDgwNTU0NDMuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">What Lukas Roder, David Shapira and Kordian Caplazi like about their daily work is that they can help shape the future of augmented reality with their ideas.. (Image: Annick Ramp / ETH Zurich)&nbsp;</span></span></span></p><p>Rimon develops virtual user manuals for industrial companies to support technicians and frontline workers frontline workers when commissioning new equipment or repairing it. Instead of working their way through a tedious paper-based handbook, users wear augmented reality glasses that guide them step by step through complex instructions, providing them with the information they need exactly when and where they need it.</p><p>Augmented reality should not be confused with virtual reality, which involves becoming completely immersed in a virtual world. AR glasses, in contrast, provide a clear view of the real environment and overlay it with digital elements. In the case of augmented user manuals, the glasses project explanatory videos, interactive text fields or even 3D machine models into the field of view.</p><h2>Out of the lab and into the field</h2><p>“Our customers often only understand exactly what we do when we put AR glasses on them,” says Shapira, laughing. The “Coffee Case”, a step-by-step guide to preparing coffee with a barista machine, is a fitting choice for a demonstration. The 29-year-old originally developed this user manual as a semester project back in early 2019.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/flZFF6wkEfY?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 768px; height: 435px;"></iframe></span><em>The “Coffee Case” has proven as a good choice for sales pitch demonstrations. Everyday work for the young entrepreneurs involves designing similar AR user manuals for specific machines belonging to their business clients. (Video: Rimon Technologies)&nbsp;</em></p><p id="isPasted">Shapira first became acquainted with AR technology and its possible applications in an elective lecture during his mechanical engineering studies. “I was fascinated by how AR glasses could present information in a different way,” he says. Caplazi, who also attended the lecture, adds: “Because it’s a young technology, there was a lot of space for new ideas.” It soon became clear to both of them that they wanted to take a closer look at AR in their Master’s theses.</p><p>Shapira came to his project through a happy coincidence, having got into conversation with the technical director of the locking systems company Dormakaba, who was visiting the mechanical engineering laboratories at ETH. The Master student talked about the possible AR applications he saw for the industrial company. Dormakaba was convinced, and Shapira set about demonstrating in a practical study the potential of the new technology in Dormakaba’s operations.</p><h2><strong>Emergency Covid solution proves successful</strong></h2><p>For his study, Shapira designed a virtual instruction manual that guides users through the assembly of a security door system for commercial buildings. “The door system was a good case study because it was being installed only rarely at the time,” he explains. Training service technicians to do this work is correspondingly time-consuming. The plan was to compare the difference in the technicians’ performance following conventional training and instruction via AR glasses.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 766px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1645439693218-image-1.imageformat.slideshow.1966286893.jpg" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">Lukas Roder demonstrates how the virtual instruction manual works. The AR glasses project the image directly onto the retina. (Image: Annick Ramp / ETH Zurich)</span></span></span></p><p id="isPasted">But as with so much other practical work, Covid threw a spanner in the works here too, and Shapira was unable to perform the planned study with industry employees during lockdown. The emergency solution was to use mechanical engineering students. With no prior knowledge of the security door system or how to use the 3D glasses, all of the students completed the assembly process with no mistakes. “I never expected that,” says Shapira.</p><p>At the same time, his fellow student Caplazi was focusing his Master’s thesis on the way in which hand and eye movements can be used to predict subsequent actions of AR users. This would make it possible to warn them of dangerous situations in good time. Shapira and Caplazi exchanged ideas and experiences and continued to experiment industriously in their free time, both becoming increasingly convinced that AR instructions were an area with a great deal of potential. In August 2020, their start-up was founded.</p><h2><strong>A diverse customer base</strong></h2><p>Today, a team of six works on further developing the AR user manuals – two team members focus exclusively on programming, while a designer takes care of the visual aspects. The aim is to design a program interface that guides users through the process as intuitively as possible. “Unlike social media apps, there aren’t any best-practice examples for AR instructions as yet,” says Caplazi.</p><p>The start-up’s customer base is diverse, incorporating companies from heating technology, energy technology and even kitchen companies, from international corporations to small SMEs. Rimon works closely with industry representatives on a project basis to find the appropriate application for each company. “We’ve seen in practice that companies often have similar needs,” says Roder.</p><blockquote>&nbsp;"We have enough scientific knowledge. Now it’s a matter of finding useful applications.” - <cite>Lukas Roder, young entrepreneur</cite>&nbsp; &nbsp;</blockquote><p>The food technology graduate, who brings experience from a previous start-up, is active primarily in sales. Like probably many others, he first considered going on to complete a doctorate after graduating, says the 28-year-old. In the field of augmented reality, however, it quickly became clear to Roder that: “We have enough scientific knowledge. Now it’s a matter of finding useful applications.”</p><p>AR instructions bring particular added value to contracts from abroad and training, both made more difficult by the pandemic. If a machine breaks down at a customer’s premises abroad, the company can send AR glasses instead of a technician, thereby not only offering a faster service but also cutting costs. Training can now be provided regardless of location, or for one employee at a time.</p><h2>Rapid developments in technology</h2><p>The start-up is currently working on a project with a heating technology company, in which the use of AR glasses as a training tool offers even greater potential. The switch to renewable energies and heating systems means that the company will have to retrain large parts of its workforce in a short time. “Seventy percent of the service technicians have been trained to work with fossil-fuel heating systems,” explains Caplazi. AR instructions help the company keep pace with technological change.</p><p>Rimon faces tough competition – tech giants such as Microsoft are also working on AR instructions for various uses. “Right now, the whole industry is trying to find out which AR applications work,” says Roder. As a result, the ETH start-up is currently concentrating on solutions for the mechanical engineering industry, where the two company founders already have plenty of expertise. Next, the young entrepreneurs plan to launch a self-service platform that will enable companies to create AR instructions themselves.</p>

Clicking on a button is intuitive but requires accuracy. (Image: Annick Ramp / ETH Zurich)

No sooner had David Shapira and Kordian Caplazi finished their studies than they set about creating a start-​up. Rimon develops virtual user manuals for industrial companies, enabling users to quickly and intuitively learn how to operate complex machines with the aid of AR glasses.

ETH graduates reinvent the user manual

<p id="isPasted">Accessibility is an undeniably important subject, yet one which is all-too often overlooked in the development of new technologies. Despite the march of progress, the majority of legally blind individuals rely on a simple white cane for navigation — something researchers Manuel Zahn and Armaghan Ahmad Khan are looking to change.</p><p>The pair, working in the Center for Digital Technology and Management at the Technical University of Munich and in partnership with FRAMOS GmbH, have developed a system which combines a depth-sensing 3D camera with a wearable haptic feedback sleeve — providing, they say, navigation and obstacle avoidance capabilities entirely hands-free, and at a fidelity and range which far exceeds a cane.</p><h1>Computer vision for the blind</h1><p>The fully-wearable system proposed by Zahn and Khan is built around an Intel RealSense D415 depth-sensing camera, housed in a 3D-printed pair of glasses worn over the user’s eyes. This is linked to a single-board computer, an AAEON Up Board, which handles processing the data from the camera, and a battery pack for portability.</p><p>The RealSense Camera is designed around an active infrared stereo imaging system, projecting beams of invisible infrared light and measuring the depth of its surroundings. Compared with a system based on visible light cameras, it has one major advantage: It can operate in absolute darkness, allowing its user to navigate without difficulty.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 300px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261002540-blind2.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="Two images: the left image shows an Intel RealSense depth-sensing camera mounted in a 3D printed pair of glasses, in place of the lenses; the right image shows a person wearing the camera system."><span class="fr-inner">An off-the-shelf depth-sensing camera combined with a 3D printed frame acts as the input to a hands-free navigation system for the blind.<br></span></span></span></p><p>Determining the depth of the wearer’s surroundings is only part of the system’s function, however: That information has to be communicated to the user in some way. Eschewing spoken or otherwise audible feedback, as likely to take away from the user’s ability to easily hear their surroundings, the researchers developed a wearable haptic feedback sleeve made from 25 vibration motors sewn onto fabric and linked to an Adafruit Feather 32U4 microcontroller development board.</p><p>With this, the system is complete: A depth-sensing camera which can map the wearer’s surroundings onto the 25-“pixel” haptic feedback system, providing a physical response when obstacles are near.</p><h1>Novel navigation</h1><p>Exactly what feedback is provided depends on where the user is navigating, the pair explain. For indoor navigation, the system provides vibration of nearby obstacles and walls — the location of the vibration mapped to the direction of the obstacle and the intensity mapped to its proximity. This way, a user can navigate with only the wearable system.</p><p>An outdoor mode switches from a standalone navigation device to an accessory, designed for use with the traditional cane: The obstacle detection range is extended to beyond two meters (around six and a half feet), providing information about obstacles and other objects outside the range of a traditional cane.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 300px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261012136-blind3.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="A picture of a custom-built haptic feedback sleeve, comprised of a grid of 25 vibration motors sewn onto fabric. Wires are seen leading to a control box and a battery pack."><span class="fr-inner">Designed to keep the user's hands free, this haptic feedback sleeve proved easy to use.<br></span></span></span></p><p>To prove the concept, the pair set up a navigation route which included walkways, doors, and various obstacles — and which had to be traversed in absolute darkness. Impressively, after a period of familiarization with the vibration feedback system, the five volunteers were able to complete the obstacle course perfectly on their very first try.</p><p>In subsequent attempts, the time taken to traverse the course reduced — showing increasing familiarity with and trust in the system: “Overall the averages for the runs decreased from 320 seconds in the first run to 148 seconds in the third run, two weeks later,” the researchers note — while admitting that one participant did become disoriented for a short period in their last run with the system.</p><p>The results encourage, with Zahn and Khan pointing to the system’s low cost and lightweight nature as making it of interest for future development — as too is the fact it does not interfere with the wearer’s hearing and leaves both hands free, either for using a traditional navigation aid like cane or for any other task.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 300px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1644261019783-blind4.jpg" style="width: 300px;" class="fr-fic fr-dib" alt="A picture of a narrow hall with a grid of colored boxes overlaid; an image to the right shows how each color maps to a strength of vibration on the haptic feedback sleeve."><span class="fr-inner">Varying strength and positional haptic feedback turn obstacles into a feeling on the wearer's arm.<br></span></span></span></p><p>“Further developments could include the implementation of an object recognition, based on the RGB [visible light] image of the camera,” the pair add. “This would allow the system to artificially generate bordered paths, solving the problem of orientation in a large open space.”</p><p>The only sticking point: Intel, the manufacturer of the camera platform used for the prototype, <a href="https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive" rel="noopener noreferrer" target="_blank">recently confirmed</a> it was to discontinue its RealSense platform — though, at the time of writing, it continued to provide the RealSense D415 for sale.</p><p>Zahn and Khan's paper is available <a href="https://arxiv.org/abs/2201.04453" rel="noopener noreferrer" target="_blank">on Cornell's arXiv.org preprint server</a>.</p><h1>References</h1><p>Manuel Zahn, Armaghan Ahmad Khan: <cite>Obstacle avoidance for blind people using a 3D camera and a haptic feedback sleeve</cite>. DOI <a href="https://arxiv.org/abs/2201.04453" target="_blank">arXiv:2201.04453</a> [cs.HC].</p><p>Dylan Martin, CRN: <em>Why Pat Gelsinger Decided to Wind Down Intel's RealSense Unit: Exclusive.&nbsp;</em><a href="https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive" id="isPasted" rel="noopener noreferrer" target="_blank">https://www.crn.com/news/components-peripherals/why-pat-gelsinger-decided-to-wind-down-intel-s-realsense-unit-exclusive</a></p>

Using a depth-sensing camera, this wearable navigation platform can alert its wearer to obstacles via haptic feedback.

Built around a custom-made haptic feedback sleeve and an Intel RealSense camera in a 3D-printed glasses housing, this navigation system lets its user find their way even in absolute darkness.

A wearable navigation system, featuring a 3D camera, offers a hands-free navigation solution for the blind

<section id="isPasted"><p>Have we entered the “Metaverse?” Beyond its very sci-fi origins, the word has become a catchall term for the way virtual reality and alternate reality tech have started to permeate everyday life, from advertising to social media to commerce.</p><p>One person on the ground floor of that 3D revolution is Alon Grinshpoon MS’18, who founded his cloud-based alternate reality/virtual reality startup shortly after he graduated from Columbia Engineering. With co-founder Ben Padazur and his twin brother and fellow Columbian Koren by his side, Grinshpoon’s product, which supplies back-end support and software for a variety of virtual reality and alternate reality projects, has been picking up steam, and recently gained close to $4 million in capital through a number of high profile investors.</p><p>For Grinshpoon, we have just begun to tap the true potential of what the technology has to offer. Metaverse, as a concept, got a huge shot in the arm once Facebook became Meta and announced its new focus on AR. The way Grinshpoon sees it, more and more professionals in every industry are trying to leverage 3D content, from multidimensional assets like fully interactable ATMs for banking apps, to crypto items like Non Fungible Tokens (NFTs), which can be represented as 3D objects in a digital space.</p><p>Scroll down to read more about how Grinshpoon’s project got started, what challenges his startup faced, advice for those seeking the same path, and where he sees the burgeoning “Metaverse” will head next.</p></section><section><h2>How Grinshpoon and echo3D got started</h2><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/Woju7XvrWlc?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 769px; height: 432px;"></iframe></span><em>Grinshpoon explains the origins of echo3D&nbsp;</em></p><section id="isPasted"><h3>What is echo3D?</h3><p>Echo3D is a cloud platform for 3D applications, which basically means that we provide tools and infrastructure for developers and companies who want to build games, AR or VR applications, metaverse apps, or any 3D experience they can think of. Very similar to how if you’re building a website, you store images and videos on some remote cloud. And with us, you store 3D models, animations, interactive content, and then you stream that to game engines and mobile devices and web browsers all over the world.</p><h3>Who I partnered with</h3><p>My cohort, actually, included my twin brother, Koren, who went to Columbia Law, so go Lions. It’s amazing, get a twin everyone while you can. We definitely have that cool combo of a techie founder and a non-techie founder. And later after that, we also got another co-founder, Ben Padazur, our CTO and highly technical software engineer. So it’s been a really, really great ride.</p><h3>My average day</h3><p>Usually, I wake up early, which I hate because my hobby is sleeping, and then I look at my phone where I’m usually bombarded with emails and messages from the night before. I usually go through them and clear them before I head to the office, or head to the computer from working from home. Usually I have multiple meetings throughout the day, whether it’s with our investors, or if it’s with potential clients. Being a founder, I would say, is really difficult. There’s a lot of sitting in front of your computer alone, basically engaging with your inbox to make stuff happen. But it's so rewarding, because I love this space so much. I went through Columbia Engineering, I studied VR, this is what I love.</p></section><section><h2>Where is the AR and VR industry heading?</h2><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/iXFkNgyQ5yw?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 765px; height: 428px;"></iframe></span><em>Grinshpoon reflects on the AR/VR market at large&nbsp;</em></p><section id="isPasted"><h3>The burgeoning market of Non-fungible Tokens</h3><p>We started early before NFTs were all the rage and before a lot of the buzzwords were there. But the cool thing about our product, and what we saw in the market, is that suddenly all these terms, all these kinds of buzzwords converged. We have clients who build in the NFT marketplace and are using echo3D as their backbone. That is something that two years ago we wouldn’t have even imagined. E-commerce has become such a cool case for using AR and VR, but suddenly we have something like NFTs, or kinds of digital assets and digital goods that obviously can benefit from AR and VR. One of our clients has a NFT marketplace for digital trading cards. They upload those 2D images and they’re converted into 3D automatically on echo3D. They did eight successful sold out drops, and the last three they did with echo3D’s AR platform sold in 30 minutes.</p><h3>Facebook’s transition into “Meta”</h3><p>My phone was burning when Meta did that announcement, because everyone is going to be super excited about this—suddenly it’s going to go mainstream. But I think, again, this is something that people are really confronted with and need to be forward thinking of, like, “how do we do 3D? How do we inject 3D content to my application?” We see that in our product as well. We get between 100 to 200 people registered every week to build applications. And everyone who’s reading this can be one of those people who literally just uses echo to build an app in 15 minutes or less.</p><h3>What all this means for echo3D</h3><p>Now we’re looking at how to leverage echo3D. Today, you can drag and drop 3D assets, and then BAM, stream them to your camera, just like that. And that’s a big, big benefit that suddenly that hardware is there. And not only that, it’s there, It’s in everyone’s pocket. One of the things that we pride ourselves in is that the time is really minimum for you to build an application. It can literally be less than 15 minutes. So I definitely encourage people to just start playing around with it, brainstorming how you can bring content into your phone, into your headset eventually and then into your industry at large.</p></section><section><h2>What challenges did echo3D face?</h2><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/ti0Mxxr13no?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 768px; height: 430px;"></iframe></span><em>Grinshpoon discusses some of the challenges of entrepreneurship&nbsp;</em></p><section id="isPasted"><h3>The simple difficulties you don’t anticipate</h3><p>I have a few challenges that can come to mind. Hiring is definitely one. We need to get to the point that we need to find the right people and make sure that they have the right DNA to join the company, have a cultural fit. But also, how do you assess their skills? Let’s say we just hired a VP of sales. I’m not a sales expert, so how do you assess those kinds of skills to make sure that the person you’re in front of is an actual good fit? The good thing is that through TechStars, and through all these entrepreneurial programs I went through the Columbia Startup Lab, you have this kind of array of mentors that you can ask. Another challenge, it sounds funny to say, but just taxes. I mean, how do you run a company? I was fortunate enough, again, to have a twin brother who went to Columbia Law who’s my co-founder, so he handles all the operations on that end, and he’s doing a phenomenal job.</p><h3>Resources at Columbia</h3><p>Columbia Startup Lab, which our offices are located and we’re in right now, is an amazing space for entrepreneurs that’s highly subsidized. You can get like really cool office space, inspired by We Work-level amenities, just some really, really cool stuff that really helped us because it’s not just a space, they have amazing mentors and resources and workshops that help you scale your company and they bring in Columbia staff from Columbia Business school and engineering to help you build your startup. Columbia Engineering Entrepreneurship is doing phenomenal work in supporting startups like ourselves, whether it’s through legal advice or mentorship, or one of the things I love to go back in the day, were the Third Thursdays which were an entrepreneurial meetup where you can come on, meet fellow like-minded founders, and basically engage with the Columbia community to help grow your startup.</p></section><section><h2>Advice for getting started</h2><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/lwC9b8EzbzM?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 768px; height: 434px;"></iframe></span><em>Grinshpoon offers tips for aspiring entrepreneurs&nbsp;</em></p><h3 id="isPasted">Use the entrepreneurial resources at your disposal</h3><p>Use programs like the Columbia Startup Lab because it gives you this automatic network and puts you in kind of the space with like-minded founders that are literally in the same boat as you are and are confronted with the same issues. So if you have a problem, you can literally ask someone next to you, how did you solve it? Some of the problems obviously are recurring between startups. The engineering school has all these reunions for alumni that I went through and reconnected with peers. Now when I engage peers I can say, “Oh, these are potential hires.”</p><h3>Find investors who care about your vision</h3><p>Another thing that is really important is founder investor fit. Basically, with the people you’re bringing in are supporters, you should make sure they actually believe in your vision, that you’re happy that they’re on your cap table because you want them on this journey. And it’s more than just like “oh, I just got money from investors.” I love our investors, like Conway Ventures, Reimagined Ventures, Space Capital, and the angels that we have from GitHub, Facebook and Data Dogs. They’re all amazing and they’re all subscribing to our vision and where we want to go.</p></section></section></section></section>

AR startup innovator Alon Grinshpoon MS’18 says it’s time to think beyond typical AR and VR tech and consider the full 3D experience.

Is "Metaverse" More Than the Latest Buzzword?

<p>This article was discussed in our <a data-saferedirecturl="https://www.google.com/url?q=https://www.wevolver.com/profile/the.next.byte&source=gmail&ust=1645483330039000&usg=AOvVaw3G1InVzNPFrh6EQN0Bd-pi" href="https://www.wevolver.com/profile/the.next.byte" id="isPasted" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/5141d285-e7f7-4062-83fb-4e9e29407a6d?dark=false" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></p><p>The full article will continue below.</p><hr><p>This article was first published on <a href="https://news.mit.edu/2022/invisible-labels-identify-track-objects-0128" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>.</p><hr><p id="isPasted">If you download music online, you can get accompanying information embedded into the digital file that might tell you the name of the song, its genre, the featured artists on a given track, the composer, and the producer. Similarly, if you download a digital photo, you can obtain information that may include the time, date, and location at which the picture was taken. That led Mustafa Doga Dogan to wonder whether engineers could do something similar for physical objects. “That way,” he mused, “we could inform ourselves faster and more reliably while walking around in a store or museum or library.”</p><p>The idea, at first, was a bit abstract for Dogan, a 4th-year PhD student in the MIT Department of Electrical Engineering and Computer Science. But his thinking solidified in the latter part of 2020 when he heard about a new smartphone model with a camera that utilizes the infrared (IR) range of the electromagnetic spectrum that the naked eye can’t perceive. IR light, moreover, has a unique ability to see through certain materials that are opaque to visible light. It occurred to Dogan that this feature, in particular, could be useful.</p><p>The concept he has since come up with — while working with colleagues at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) and a research scientist at Facebook — is called <a href="https://groups.csail.mit.edu/hcie/files/research-projects/infraredtags/2022-CHI-InfraredTags-paper.pdf" target="_blank">InfraredTags</a>. In place of the standard barcodes affixed to products, which may be removed or detached or become otherwise unreadable over time, these tags are unobtrusive (due to the fact that they are invisible) and far more durable, given that they’re embedded within the interior of objects fabricated on standard 3D printers.</p><p><a target="_blank"></a></p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/Tjs1zH7IMxE?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 767px; height: 430px;" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></p><p id="isPasted">Last year, Dogan spent a couple of months trying to find a suitable variety of plastic that IR light can pass through. It would have to come in the form of a filament spool specifically designed for 3D printers. After an extensive search, he came across customized plastic filaments made by a small German company that seemed promising. He then used a spectrophotometer at an MIT materials science lab to analyze a sample, where he discovered that it was opaque to visible light but transparent or translucent to IR light — just the properties he was seeking.</p><p>The next step was to experiment with techniques for making tags on a printer. One option was to produce the code by carving out tiny air gaps — proxies for zeroes and ones — in a layer of plastic. Another option, assuming an available printer could handle it, would be to use two kinds of plastic, one that transmits IR light and the other — upon which the code is inscribed — that is opaque. The dual material approach is preferable, when possible, because it can provide a clearer contrast and thus could be more easily read with an IR camera.</p><p>The tags themselves could consist of familiar barcodes, which present information in a linear, one-dimensional format. Two-dimensional options — such as square QR codes (commonly used, for instance, on return labels) and so-called ArUco (fiducial) markers — can potentially pack more information into the same area. The MIT team has developed a software “user interface” that specifies exactly what the tag should look like and where it should appear within a particular object. Multiple tags could be placed throughout the same object, in fact, making it easy to access information in the event that views from certain angles are obstructed.</p><p>“InfraredTags is a really clever, useful, and accessible approach to embedding information into objects,” comments Fraser Anderson, a senior principal research scientist at the Autodesk Technology Center in Toronto, Ontario. “I can easily imagine a future where you can point a standard camera at any object and it would give you information about that object — where it was manufactured, the materials used, or repair instructions — and you wouldn't even have to search for a barcode.”</p><p>Dogan and his collaborators have created several prototypes along these lines, including mugs with bar codes engraved inside the container walls, beneath a 1-millimeter plastic shell, which can be read by IR cameras. They’ve also fabricated a Wi-Fi router prototype with invisible tags that reveal the network name or password, depending on the perspective it’s viewed from. They’ve made a cheap video game controller, shaped like a wheel, that is completely passive, with no electronic components at all. It just has a barcode (ArUco marker) inside. A player simply turns the wheel, clockwise or counterclockwise, and an inexpensive ($20) IR camera can then determine its orientation in space.</p><p>In the future, if tags like these become widespread, people could use their cellphones to turn lights on and off, control the volume of a speaker, or regulate the temperature on a thermostat. Dogan and his colleagues are looking into the possibility of adding IR cameras to augmented reality headsets. He imagines walking around a supermarket, someday, wearing such headsets and instantly getting information about the products around him — how many calories are in an individual serving, and what are some recipes for preparing it?</p><p>Kaan Akşit, an associate professor of computer science at University College London, sees great potential for this technology. “The labeling and tagging industry is a vast part of our day-to-day lives,” Akşit says. “Everything we buy from grocery stores to pieces to be replaced in our devices (e.g., batteries, circuits, computers, car parts) must be identified and tracked correctly. Doga’s work addresses these issues by providing an invisible tagging system that is mostly protected against the sands of time.” And as futuristic notions like the metaverse become part of our reality, Akşit adds, “Doga’s tagging and labeling mechanism can help us bring a digital copy of items with us as we explore three-dimensional virtual environments.”</p><p>The <a href="https://groups.csail.mit.edu/hcie/files/research-projects/infraredtags/2022-CHI-InfraredTags-paper.pdf" target="_blank">paper</a>, “InfraredTags: Embedding Invisible AR Markers and Barcodes into Objects Using Low-Cost Infrared-Based 3D Printing and Imaging Tools,” is being presented at the ACM CHI Conference on Human Factors in Computing Systems, in New Orleans this spring, and will be published in the conference proceedings.</p><p>Dogan’s coauthors on this paper are Ahmad Taka, Michael Lu, Yunyi Zhu, Akshat Kumar, and Stefanie Mueller of MIT CSAIL; and Aakar Gupta of Facebook Reality Labs in Redmond, Washington.</p><hr><p>Reprinted with permission of <a href="https://news.mit.edu/2022/invisible-labels-identify-track-objects-0128" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>"</p>

MIT scientists built a user interface that facilitates the integration of common tags (QR codes or ArUco markers used for augmented reality) with the object geometry to make them 3D printable as InfraredTags. Credits: Photos courtesy of MIT CSAIL.

An MIT team develops 3D-printed tags to classify and store data on physical objects.

Invisible machine-readable labels that identify and track objects

<p id="isPasted">These days, people stare at their screens ever more frequently – especially since the onset of the coronavirus pandemic. Conferences, meetings and discussions with work colleagues all take place via video call. If the big tech companies have their way, such encounters will become an immersive experience in the so-called metaverse as early as next year, thanks to 3D glasses and specialised computer programs.</p><p>The key to permitting a natural user experience in VR and AR applications is to create what are known as avatars, which are computer-generated, dynamic representations of people. The more realistic the avatars’ appearance and behaviour, the more likely it is that people will gain a sense of real social interaction.</p><p>However, modelling a human being in detail and in motion is a task that continues to challenge the developers of these applications. Today’s graphics programs can already create photorealistic, static avatars. But to animate a smiling face, say, graphic designers have to manually edit almost every single image on the computer to correct nuances such as wrinkles and shadows.</p><p>Researchers led by Otmar Hilliges, Professor of Computer Science at ETH Zurich, showed how to do this more easily at the <a href="https://openaccess.thecvf.com/content/ICCV2021/html/Chen_SNARF_Differentiable_Forward_Skinning_for_Animating_Non-Rigid_Neural_Implicit_Shapes_ICCV_2021_paper.html" target="_blank">International Conference on Computer Visioncall_made</a> in October 2021. Instead of modelling every last detail, they use intelligent algorithms that learn to automatically render animated avatars in every conceivable pose by observing 3D images of humans in just a few poses.</p><h2>Computer model can even handle handsprings</h2><p>Computer programs that use artificial intelligence to create lifelike virtual humans have existed only for a few years. These programs learn to realistically depict different body positions using 3D scans of a real person, which are recorded beforehand using a complex camera system.</p><p>The AI algorithms process the scans by measuring countless points inside and outside the person’s body to define its contours as a mathematical function. In this way, the algorithms build a template shape of the person. To move the avatar to new poses, the algorithms learn to memorize the path from the moving pose back to the template.</p><p>However, for extreme poses outside the known repertoire of movements, such algorithms do not have the knowledge and predict wrong paths, leading to clearly visible artifacts: the arms might be detached from the body or the joints located in the wrong place. This is why today’s models are trained on as many different poses as possible – which entails a huge effort for 3D scanning and requires enormous computing power.</p><p>To date, AI avatars are hardly applicable, especially for interactive applications. “It’s impractical to capture the entire possible repertoire of movements,” says Xu Chen, a doctoral student and lead author of the study.</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1643193965433-image.imageformat.twocolumn.248606704.gif" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-img-caption fr-fic fr-dib" style="width: 300px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1643193946604-image.imageformat.twocolumn.1792160336.gif" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Graphics: Xu Chen/ETH Zurich&nbsp;</span></span></span></p><p id="isPasted">The new method Chen has developed takes the opposite approach: the model computes the path from the template to the moving poses. Since this means the calculations always have the same starting point, it lets the intelligent algorithms learn better how to generalise movements.</p><p>Indeed, for the first time, it puts such a computer model in a position to easily represent new motion patterns as well. It can even produce acrobatic movements such as a somersault or a back bridge.</p><h2>Any number of new faces from just one image</h2><p>The new full-body avatars can’t yet be personalised; the representations are limited to the person scanned in the original 3D images. Chen and his colleagues would like to further develop their computer model so that it can create new identities at will.</p><p>Marcel Bühler, another doctoral student in Hillige’s group, has already found a solution for personalising avatar faces and changing them as desired. Like Chen in his full-body models, Bühler used intelligent algorithms to create new animated faces from a combination of a 3D face model and a large collection of portrait photos.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 766px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1643194439353-ezgif-7-c725b9e1b0.gif" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">Graphics: Marcel Bühler/ETH Zurich&nbsp;</span></span></span></p><p id="isPasted">While previous computer programs already provided good animations of faces from the front, Bühler’s model can also realistically represent faces from the side as well as from above and below.</p><h2>A close look can unmask deepfakes</h2><p>Is there a danger that the new technology will soon allow even more realistic deepfake videos to circulate, for example to fake a speech by an important politician? “Deepfake videos are still far from perfect,” Bühler says. Most computer programs, he points out, achieve good results only for a particular setting. For example, the new face model can’t yet realistically represent details such as hair.</p><p>“Anyone who looks closely will still find artifacts,” Bühler says. He thinks it is more important to keep the public informed and aware of the current state of affairs. Making the research on 3D rendering techniques, as well as their vulnerabilities, publicly available could help cybersecurity experts more easily detect deepfake videos on the web, he adds.</p><p>For interactive virtual reality applications, the work of these ETH researchers represents enormous progress. It’s quite possible that tech companies like Facebook and Microsoft will implement the two doctoral students’ newly developed techniques in their avatars.</p><hr><h2 id="isPasted">Reference</h2><p>Chen X, Zheng Y, Black M, Hilliges O, Geiger A. <a href="https://openaccess.thecvf.com/content/ICCV2021/html/Chen_SNARF_Differentiable_Forward_Skinning_for_Animating_Non-Rigid_Neural_Implicit_Shapes_ICCV_2021_paper.html" target="_blank">SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescall_made</a>. International Conference on Computer Vision (ICCV), published online on October 11th, 2021.</p><p>Bühler M, Meka A, Li G, Beeler T, Hilliges O. <a href="https://openaccess.thecvf.com/content/ICCV2021/html/Buhler_VariTex_Variational_Neural_Face_Textures_ICCV_2021_paper.html" target="_blank">VariTex: Variational Neural Face Texturescall_made</a>. International Conference on Computer Vision (ICCV), published online on October 11th, 2021.</p>

The new AI-​based approach can create realistic virtual humans that perform never-​before-seen moves such as wild dancing. (Graphic: Xu Chen/ETH Zürich)

Soon, internet users will be able to meet each other in cyberspace as animated 3D avatars. Researchers at ETH Zurich have developed new algorithms for creating virtual humans much more easily.

One step closer to lifelike avatars

<p>The global <a href="https://www.marketsandmarkets.com/Market-Reports/digital-twin-market-225269522.html" id="isPasted" rel="noopener" target="_blank">digital twin market size was valued</a> at $3.1 billion in 2020 and is projected to reach $48.2 billion by 2026. It is expected to grow at a CAGR of 58% over the forecast period.&nbsp;</p><p>According to <a href="https://www.prnewswire.com/news-releases/the-future-of-the-digital-twins-industry-to-2025-in-manufacturing-smart-cities-automotive-healthcare-and-transport-301028858.html" rel="noopener" target="_blank">Research &amp; Markets</a>, the key factors driving the adoption of digital twins in organizations today are:</p><ul><li>The evolution of artificial intelligence (AI) technologies, data science, and the industrial <a href="https://www.rinf.tech/how-to-capture-industrial-iot-iiot-value-through-rd-innovation/" rel="noopener noreferrer" target="_blank">Internet of Things (IIoT)</a>,</li><li>The rapid development of digital health and manufacturing solutions,</li><li>Increased demand for predictive maintenance and real-time monitoring data,</li><li>The need to reduce time to market and improve decision making,</li><li>The need for production optimization and cost savings, etc.</li></ul><p>Speaking of the constraints holding back the massive adoption of digital twin technology, data security due to the widespread use of IoT and Cloud platforms is one of the main ones.</p><p>With the proliferation of cyberattacks on critical Cloud infrastructures over the past decade, cybersecurity has become a major concern for industrial automation users and vendors. As such, the growing threat to the security of data connected to the Cloud is expected to be a major impediment to the growth of the digital twin market.</p><p>Another major challenge on the path to efficiently designing and deploying digital twins is the lack of awareness of the cost-benefit of implementing digital twins.</p><p>Despite the barriers, 31% of companies surveyed by <a href="https://www.gartner.com/en/newsroom/press-releases/2020-10-29-gartner-survey-reveals-47-percent-of-organizations-will-increase-investments-in-iot-despite-the-impact-of-covid-19-" rel="noopener" target="_blank">Gartner</a> in 2020 said they were already using digital twins to improve the safety of their employees or customers by implementing remote asset monitoring, control, and tracking.&nbsp;</p><p>We at <a href="https://www.rinf.tech" rel="noopener noreferrer" target="_blank">rinf.tech</a> have been experimenting with remote asset control and the digital twin technology for quite a while in our R&amp;D Center. Now we'd like to present to you one of the outcomes – our proof-of-concept robotic arm's digital twin solution based on AR (HoloLens).</p>

This is a demo of the robotic arm's digital twin PoC project created by the rinf.tech R&D Division.

HoloLens-enabled robotic arm's digital twin

<p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/lq3fW7Z8p2c?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 767px; height: 430px;"></iframe></span></p><p id="isPasted">Read the&nbsp;<a href="https://longread.epfl.ch/en/dossier/virup/" target="_blank">longread version</a> of this article.</p><p>You’re floating in space, just above the Earth. The International Space Station is an arm’s length away. You twist your head around only to see the moon, a tiny circle, far off in the distance. You can’t help but think that this is probably what an astronaut would see during a space-walk.</p><p>This is the beginning of a journey into outer-space, in a virtual environment developed by EPFL scientists.</p><p>Now, for the very first time, you can enter the most comprehensive virtual universe based on the latest astrophysical and cosmological data, thanks to powerful, open-source software developed at EPFL’s Laboratory of Astrophysics (LASTRO). The software is called VIRUP, for Virtual Reality Universe Project, and a&nbsp;<a href="http://go.epfl.ch/virup" target="_blank">first beta version</a> is being released today.</p><p>“You can navigate through the most detailed map of the universe from the comfort of your own home,” explains Jean-Paul Kneib, director of LASTRO. “It’s the chance to travel through space, through time, and discover the universe.”</p><h2><strong>The VIRUP challenge : visualizing terabytes of data at once</strong></h2><p>Astronomers and astrophysicists are collecting data about billions of celestial objects in the night sky with the help of telescopes here on Earth and in space. There are already decades of observational data. Even greater amounts of data are expected in the near future.</p><p>In order to get visual representations of the vast amounts of data, like a movie, it’s standard practice to pre-render specific sequences. But what about creating a visual representation of the data in real-time, as if you were there, an observer at an arbitrary point in space and time? This is what astrophysicist Yves Revaz of LASTRO set out to do with the VIRUP, with the help of LASTRO software engineer Florian Cabot, and it meant rendering terabytes of data at 90 frames per second. The latter constraint is imposed by the virtual reality environment, for a fully immersive and smooth experience.</p><p>“Visualization of astrophysical data is much more accessible than showing graphs and figures, it helps to develop intuition of complex phenomena,” explains Revaz. “VIRUP is precisely a way of making all of our astrophysical data accessible to everyone, and this will become even more important as we build bigger telescopes like the&nbsp;<a href="https://www.skatelescope.org/" rel="noopener" target="_blank">Square Kilometer Array</a> that will generate tremendous amounts of data.”</p><h2><strong>Astrophysical, cosmological data and simulations</strong></h2><p>For the moment, VIRUP can already visualize data from over 8 databases bundled together. The&nbsp;<a href="https://www.sdss.org/" rel="noopener" target="_blank">Sloan Digital Sky Survey</a> consists of over 50 million galaxies and 300 million objects in general. The&nbsp;<a href="https://sci.esa.int/web/gaia" rel="noopener" target="_blank">Gaia data</a> of the Milky Way Galaxy consists of 1.5 billion light sources. The&nbsp;<a href="https://www.cosmos.esa.int/web/planck" rel="noopener" target="_blank">Planck mission</a> involves a satellite which measures the universe’s first light after the Big Bang called the cosmic microwave background radiation. There’s also the&nbsp;<a href="http://www.openexoplanetcatalogue.com/" rel="noopener" target="_blank">Open Exoplanet Catalog</a> which aggregates various sources of exoplanet data. Other databases include a repertoire of over 3000 satellites orbiting the Earth, as well as various skins and textures to render the objects.</p><p>VIRUP also renders data sets of contemporary and scientifically robust simulations based on research. You can watch the Milky Way Galaxy and its future collision with the Andromeda galaxy, our galactic neighbor also known as M31. You can also see huge portions of the cosmic web – the filamentary large-scale structures that extend across the universe – coming into existence over billions of years, based on simulations from a data set called&nbsp;<a href="https://www.tng-project.org/" rel="noopener" target="_blank">IllustrisTNG</a> which consists of 30 billion simulated particles. A major challenge is ensuring smooth transition from one database to the next.</p><p>“We considered using existing graphics engines for visualizing the data, but in the end, I developed one specifically for the project. It’s flexible, we can add more data as it becomes available, and it’s tailored to astronomy,” explains Cabot. “For this first release of VIRUP, I have focused on rendering static data, so interacting with the data is still a bit rough and the rendering of simulations can’t yet happen in real-time for example.”</p><p>Of course, it’s only possible to navigate through the data and simulations imported into VIRUP. You can visit the 4500 discovered exoplanets so far, for instance, but the way they look are artist impressions inferred from observation. You can also navigate through the 50 million galaxies measured so far by the Sloan Digital Sky Survey, but the actual data has limited resolution and this limits how much detail can be shown in its virtual representation. That being said, there is still a tremendous amount of data that can be explored with the help of VIRUP. Some of the next steps could be to include databases of objects in our solar system like all of the asteroids, or various other objects in the galaxy like nebulae and pulsars.</p><h2><strong>Flexible immersive virtual environment</strong></h2><p>For the fully immersive, 3D, 360 experience, you’d need a pair of VR glasses and a computer for running the VIRUP engine, plus storage space to store a selection of astrophysical and cosmological data.</p><p>VIRUP is also capable of building a virtual universe in other VR environments, like a dome which is especially useful for venues like planetariums, panaromas,&nbsp;<a href="https://en.wikipedia.org/wiki/Cave_automatic_virtual_environment" rel="noopener" target="_blank">caves</a> and half-caves. The open-software’s transition from the rather personal and isolated experience of VR goggles to the collective, theatrical experience offered by domes and caves, became possible thanks to a collaboration between LASTRO scientists and researchers at EPFL’s&nbsp;<a href="https://www.epfl.ch/labs/emplus/" target="_blank">Laboratory for Experimental Museology</a> (eM+), and funded by&nbsp;<a href="file:///Volumes/GoogleDrive/tmp/mozilla_revaz0/Link%20to%20Seed%20Funding%20page" target="_blank">EPFL seed funding</a> for fostering interdisciplinary projects.</p><p>“It’s about data discovery. The immersive system means that you can embody the data which has a profound effect on how you actually perceive the data,” says artist Sarah Kenderdine who leads eM+.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/I2Mcg2aUdNw?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 768px; height: 432px;"></iframe></span></p><h2 id="isPasted"><strong>A journey though the universe – a short movie</strong></h2><p>With the release of VIRUP comes a short movie entitled&nbsp;<a href="https://www.youtube.com/watch?v=EMfSDXyfBXw" rel="noopener" target="_blank">“Archaeology of Light”</a><a href="http://go.epfl.ch/ArchaeologyofLight" target="_blank">,</a> one possible journey through the virtual universe made possible thanks to the open software.</p><p>The 20 minute movie starts from Earth, and charts out a voyage throughout the various scales of the universe, from our solar system, to the Milky Way, all the way to the cosmic web and the relic light of the Big Bang.</p><p>If you’re impatient to see the movie, you will be able to watch it in<a href="https://youtu.be/I2Mcg2aUdNw" rel="noopener" target="_blank">&nbsp;2D</a>, in 360° and in stereo 180° on&nbsp;<a href="https://www.youtube.com/channel/UCFcfHd-8g-8JY29Jd5R0eKw" rel="noopener" target="_blank">YouTube</a>. If you have access to a VR environment, you can immerse yourself in “Archaeology of Light”.</p><p>For the dome experience, the movie will be showcased at EPFL’s next exhibit, Cosmos Archaeology: Explorations in Space and Time, which opens on 21 April 2022 at <a href="https://epfl-pavilions.ch/" rel="noopener" target="_blank">EPFL Pavilions</a>. A preliminary version of the movie was shown at the Synra Dome of the Science Museum of Tokyo in September, thanks to support from the Swiss Embassy in Tokyo. VIRUP will be presented this month at an exhibit, in Dubai, as part of <a href="http://www.epflvirtualspacetour.com/" rel="noopener" target="_blank">EPFL’s Virtual Space Tour</a>.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/5D-7hszUhzM?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 766px; height: 441px;"></iframe></span></p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable fr-active" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/VGA4NrqqYiU?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 766px; height: 421px;"></iframe></span><br></p>

Have you ever wanted to explore outer-space? Now you can, without leaving Earth, thanks to powerful, open-source beta software VIRUP that builds – in real-time – a virtual universe based on the most detailed contemporary astrophysical and cosmological data.

Explore the universe with virtual reality

<p><em>Together with Ansys, a developer of simulation software, we are creating <a href="https://www.wevolver.com/article/decarbonization-sustainability-and-innovations-in-transportation-and-mobility" target="_blank">a series of content</a> about the technologies that drive innovation. This article was written by engineer&nbsp;</em><a href="https://www.wevolver.com/profile/nicolas.sarmiento.sierra" rel="noopener noreferrer" target="_blank"><em>Nicolas Sierra Sarmiento</em></a><em>&nbsp;with editing from Wevolver.&nbsp;</em></p><h1 dir="ltr" id="isPasted">Simulation Demands Intuitive 3D Visualizations</h1><p dir="ltr" id="isPasted">Simulation technology made up of various software packages enable designers and engineers to build, test and analyze objects, systems and ideas. Simulation software has exponentially improved over the last decade, and there is now a growing need for supporting technologies to better leverage the potential of simulation. This article examines the use of emerging visualization technologies that will enable faster and more accurate simulation solutions. Additionally, it discusses the rise of cloud computing that will be necessary to deliver seamless interactive experiences.&nbsp;</p><p dir="ltr">The rapid evolution of simulation technologies promotes innovation that elevates how people interact with the detailed models and data that simulation delivers. As innovation in the simulation sector experiences continual dramatic changes, mechanisms to support demanding technical requirements are also required to provide better and faster data processing, tracking, measuring, and mapping of environments.&nbsp;</p><p dir="ltr">An answer to these challenges is Assistive Technologies (AT). ATs are innovations developed to support or enhance the performance and technical capabilities of the equipment used for the design, acceleration of data processing, and/or rendering of 3D and virtual environments. Because higher computational performance is necessary to complete these and other tasks, computing-based ATs are becoming essential to bring simulation technologies to more applications and devices without worrying about software and hardware limits. ATs include virtual reality (VR) and augmented reality (AR), as well as other emerging hardware such as 3D-enabled tablets.</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib fr-draggable" contenteditable="false" draggable="false" style="width: 766px;"><span class="fr-img-wrap"><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjI4MTQ3NDc2MzYyLXhyLWV4cG8taXBEaE9RNWd0RWstdW5zcGxhc2guanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner" contenteditable="true"><span contenteditable="true" id="isPasted">Figure 01: VR in use in research and design.</span></span></p></span></span></div><h2 dir="ltr" id="isPasted">Basics of Virtual Reality</h2><p dir="ltr">In principle, VR is a computer-generated digital setting experienced through sensory stimuli observed in 2D or 3D<sup>[1]</sup>. VR offers an interactive and multi-sensorial experience of a fictitious world in an enclosed environment where the user determines how the simulated world interacts. Users typically experience the simulated environments by putting on immersive <a href="https://www.wevolver.com/article/ferrari-innovating-and-winning-with-simulation-technology" rel="noopener noreferrer" target="_blank">head-mounted displays</a> (HMDs).&nbsp;</p><p dir="ltr">The user sends information into the system, and the feedback returns to the user from the system with particular stimuli. Data is collected from the user, such as eye position, user location, hand movements, etc., and transmitted to the application to update geometry, user inputs, and other simulation features (Figure 02). The rendering transforms the computer format to a user format delivering visual, auditory, and haptic outputs.&nbsp;</p><div class="fr-img-space-wrap"><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib" style="width: 766px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjI4MTQ4NDk0MjAxLTEtMDEgZWRpdGVkLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Figure 02: A typical VR system [1]</span></span></span></div></div><h2 dir="ltr" id="isPasted">Basics of Augmented Reality</h2><p dir="ltr">In contrast to VR, where an environment is fictitious, AR is a set of technologies that helps see the real world differently. With the aid of computer-made images, a processor, and remote (cloud contained or downloaded directly through the internet) or local data sources, a device can augment the real world by superimposing virtual objects that can be seen or perceived in real-time<sup>[2]</sup>. AR can be created anytime, anywhere using either fixed (e.g., rooms) or movable systems (phones). Together with cloud computing, a real-time AR simulation can be experienced on low-compute environments and devices.</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib fr-draggable" contenteditable="false" draggable="false" style="width: 766px;"><span class="fr-img-wrap"><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjI4MTQ4NTY2MzI1LUFSLUNGRC1waG90by1hdC1OTVcyMDE3LTEwMjR4NjQwLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner" contenteditable="true">Figure 03: Using AR to present CFD results is especially effective when trying to communicate insights from simulations to a non-technical audience. Image credit: LEAP/Monash Motorsports.</span></p></span></span></div><h3 dir="ltr" id="isPasted">The next generation of VR/AR solutions</h3><p dir="ltr">Ansys has developed a physics-based and real-time solution called <a href="https://www.ansys.com/products/optics-vr/ansys-vrxperience-perceived-quality" rel="noopener noreferrer" target="_blank"></a><a href="https://www.ansys.com/products/optics-vr/ansys-vrxperience-perceived-quality" rel="noopener noreferrer" target="_blank">VRXPERIENCE Perceived Quality</a> for design evaluation by using VR technologies that allow interactive tracing and configuration of prototype features while aiming for significant accuracy. Prototypes can be significantly improved by having access to predictive and validating options that can help control the impacts of light, color, and material variation. This enables engineers and designers to make better decisions through virtual prototypes that impact product design optimization.&nbsp;</p><p dir="ltr" id="isPasted">The Ansys VRXPERIENCE allows users to take CAD formats and optimize models through light, color, and material libraries. Based on the physics of light, accurate rendering can be delivered due to the optical propagation capabilities that can be manipulated in 360° sets. This service enables scalable rendering completed with calculations designed to find the best options between accuracy and interactivity. HMD-based visualization can guide users to efficiently improve their designs through different simulations. As a result, productivity and prototype quality are considerably increased from the moment VR is used for prototyping.</p><h1 dir="ltr" id="isPasted">The Lume Pad tablet</h1><p dir="ltr">To date, most devices that are used to generate 3D images are based on stereoscopy. This technique creates images with the major depth on a flat screen but does not provide images of a vividness that can replicate what the human eye can see, which is one of the factors severely limiting the optical delivery of VR/AR services. Lightfield technology is one solution to this challenge. The technology collects the radiance of light rays coming from different directions. It provides elegant and intense visual experiences that take advantage of angular information obtained at different distances and positions in the scene<sup>[4]</sup>.&nbsp;</p><p>California-based company <a href="https://www.leiainc.com/" rel="noopener noreferrer" target="_blank">Leia</a> has built upon Lighfield’s characteristics and developed a patented technology called Diffractive Lightfield Backlighting (DLB™) that uses finely tuned nano-structures to project a dense Lightfield from a transparent, edge-lit substrate. The Lume Pad is Leia’s keystone product that is capable of sending variable amounts of light to contrasting directions, resulting in content that can be examined in high fidelity 3D without the use of glasses or eye-tracking.&nbsp;</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib fr-draggable" contenteditable="false" draggable="false" style="width: 766px;"><span class="fr-img-wrap"><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1629880263984-lumepad-car-on.png" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner" contenteditable="true">Figure 04: Leia’s Lume Pad.&nbsp;</span></p></span></span></div><p dir="ltr" id="isPasted">With the DLB™, the tablet can use optimal resolutions (2D resolution 2560×1440) where about 16 images are sent to the display simultaneously (the resolution of each Lightfield view is 640x400). In addition to the DLB™ technology, Leia uses other software and hardware necessary to bring about the total performance of the Lume Pad. In particular: nanotechnology, manufacturing, video processing tools, and the development of its LF content. The Lume Pad is helping advance the way simulation can be experienced and potentially broadening the way simulation is delivered to different applications. One exciting example is the possibility for DLB to offer a much more interactive and intuitive way to view and understand the digital twins of smart buildings or public infrastructure, opening this critical tool to more users.</p><h2 dir="ltr" id="isPasted">Technological Convergence to deliver the future of simulation</h2><p dir="ltr" id="isPasted">Graphic processing units (GPUs) perform in parallel to central processing units (CPUs) to enhance visualization in 2D surfaces<sup>[5]</sup>. They take CPU’s off-loaded functions with a high degree of parallelism, for example, running several mathematical functions or data points simultaneously. High-performance computing (HPC) or inclusive deep learning are technologies taking advantage of modern GPUs. But, traditional GPUs might not be enough to withstand the requirements of new simulation tools. Common restraint factors are equipment availability (including price), processor performance, data storage, growing digital collaborative work, and running times for data processing. For high-end 3D visualizations, other types of infrastructure are required.</p><p dir="ltr">In the case of data processing, cloud-based GPUs offer a great solution to deal with high loads of information and stream visualizations back to remote devices in real-time. An example of cloud-based immersive streaming is NVIDIA’s CloudXR, which delivers graphics-intensive VR/AR experiences across radio signal or ethernet. Simulation focussed companies such as Ansys are integrating NVIDIA cloud services into some products.</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib fr-draggable" contenteditable="false" draggable="false" style="width: 766px;"><span class="fr-img-wrap"><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjI4NzUzMDM1NDQ3LXByb3Zpei12aXN1YWwtY2xvdWR4ci1zdGFjay11cGRhdGUucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner" contenteditable="true">Figure 05: NVIDIA CloudXR Architecture.</span></p></span></span></div><p dir="ltr" id="isPasted">In the search to improve graphical processing, Ansys uses cloud-based GPU virtualization to accelerate their simulation and rendering. When run in a virtualized environment, <a href="https://developer.nvidia.com/nvidia-cloudxr-sdk" rel="noopener noreferrer" target="_blank">NVIDIA CloudXR</a> leverages NVIDIA RTX Virtual Workstation (RTX vWS) to stream fully accelerated immersive graphics to a remote device such as an HMD or the Lume Pad. By running their simulations remotely on powerful GPUs and streaming them with CloudXR and RTX vWS, Ansys users are able to visualize their designs and interact with them, without interrupting their simulation.&nbsp;</p><h1 dir="ltr">Conclusion</h1><p dir="ltr">As simulation for design, monitoring, testing, and more becomes more and more integrated into all industries, assistive technologies that enable us to engage more richly with simulation will continue to appear. With the aid of these technologies, simulation will continue to expand as a method of design, research, monitoring, and testing. &nbsp;</p><h2 dir="ltr">References</h2><p dir="ltr">1. &nbsp; &nbsp; Jerald J. The VR Book: Human-Centered Design for Virtual Reality. 1st ed. Özsu MT, Hart JC, editors. Morgan &amp; Claypool Publishers All; 2016. 636 p.&nbsp;</p><p dir="ltr">2. &nbsp; &nbsp; Furth B, Carmigniani J, Hugues O, Fuchs P, Nannipieri O, Kalkofen D, et al. Handbook of Augmented Reality. Furth B, editor. Boca Raton, Florida: Springer New York Dordrecht Heidelberg London; 2011. 769 p.&nbsp;</p><p dir="ltr">3. &nbsp; &nbsp; Ansys. VRXPERIENCE Perceived Quality. © 2020 ANSYS, Inc. All Rights Reserved.&nbsp;</p><p dir="ltr">4. &nbsp; &nbsp; Mahmoudpour S, Schelkens P. On the performance of objective quality metrics for lightfields. Signal Process Image Commun [Internet]. 2021;93(January):116179. Available from: https://doi.org/10.1016/j.image.2021.116179</p><p dir="ltr">5. &nbsp; &nbsp; Edition S, Parker M. Digital Signal Processing 101. Second Edi. Jonathan Simpson; 2017. 411 p.&nbsp;</p>

vr/ar