<a href="https://www.ubisoft.com/" rel="noopener noreferrer" target="_blank">Ubisoft</a> Annecy delivered the ultimate winter sports game with 2016&rsquo;s Steep, a sublime simulation that recreated the wide-open slopes of the Alps that players could explore on skis or snowboards, or even by paragliding or soaring with a wingsuit. Whether you wanted a quiet solo jaunt down the side of a mountain or a raucous race against online rivals, it was all there.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxODA0ODIxNzA1LXN0YXJ0LTExLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">It was the first wholly original project from Ubisoft Annecy, the global publisher&rsquo;s French studio, and after shipping extensive add-on content, including collaborations with the 2018 Winter Olympics and ESPN&rsquo;s X Games, the team turned its attention towards the next big thing.This next big thing is&nbsp;<a href="https://www.ubisoft.com/en-gb/game/riders-republic" rel="noopener noreferrer" target="_blank">Riders Republic</a>, a multi-disciplinary sports sandbox game that features skis, snowboards, snowmobiles alongside wingsuit, and mountain biking in a vast world that mashes up various American landmarks, including Yosemite Valley and Grand Teton. It&rsquo;s also punched up with a lot more personality and flair and takes a massively multiplayer approach with dozens of players competing together at once.&nbsp;For the Riders Republic, Ubisoft Annecy wanted to tap into the latest technology to deliver precise and realistic-looking tricks and animations. That&rsquo;s why the studio turned to the <a href="https://www.xsens.com/products/mvn-animate" rel="noopener noreferrer" target="_blank">Xsens MVN</a> inertial&nbsp;<a href="https://www.xsens.com/motion-capture" rel="noopener noreferrer" target="_blank">motion capture system</a>, with robust suits that could be easily transported to the slopes to capture professional athletes and translate that action into the game.&nbsp;<h3>Capturing the pros</h3>Ubisoft Annecy had previously worked with optical motion capture for indoor shoots, but camera-based systems aren&rsquo;t ideal when it comes to real-world and large-volume capture demands. When it was time to work with professional skiers and snowboarders on-location to capture tricks and actions for the Riders Republic, the studio turned to Xsens. The suit&rsquo;s durable sensors can work anywhere.&nbsp;<iframe width="640" height="360" src="https://www.youtube.com/embed/xapNAdQcPVg?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe>&quot;We started using Xsens in 2018 because we wanted another option. It&#39;s easier to use in specific conditions, like the sports we have in Riders Republic,&quot; explains Julien Beilles, Lead Animator at Ubisoft Annecy. &quot;The biggest point for us in using Xsens was being able to use it on the snow. That was the first time we used it for that. It&#39;s really hard to do optical motion capture on the snow, so we chose Xsens for that.&quot;&nbsp;Another reason Julien selected the Xsens suit for the job was the ability to set up and get ready to capture without needing to bring an additional team to manage technical support. There&rsquo;s significantly less fuss and setup with the Xsens MVN inertial motion capture suits, which the skiers and snowboarders could wear beneath their usual gear and perform unabated. Clean data can be captured anywhere, even using a laptop.&ldquo;It was quite easy, I would say,&rdquo; he explains. &ldquo;For optical capture, we used to have technical directors with us, and they are in charge of most of the technical stuff. With Xsens, we are able to go on a shoot without any TDs with us, and we ended up shooting by ourselves with a couple of animators. In the end, we managed to do what we wanted with a small team, so it went great.&rdquo;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxODA0OTc0NTY4LVJpZGVycyBSZXB1YmxpYyAyLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">Although Annecy has ample slopes to choose from, the team opted to shoot at a covered, real-snow, ski facility in Landgraaf in the Netherlands to ensure consistent conditions and access to electricity. It was a five-day shoot that included pro athletes, and while their availability was limited, the rapid setup and recording with the Xsens suits meant that no time was wasted as they captured every takeoff, landing, and rail trick they could.&ldquo;We shot pro athletes that had very limited time to offer in their schedule, so Xsens sounded like a better solution for us to go with,&rdquo; says Julien. &ldquo;In the end, I think we made the right choice.&rdquo;<h3>Just like real life</h3>Julien&#39;s team used the clean inertial motion capture data to translate the realistic dynamics of various tricks and movements and combine that with keyframing to input into the game&rsquo;s animation system. The Xsens suits not only helped deliver on the impressive realism of the real-world actions but also significantly cut down the amount of work that would&rsquo;ve been done if they had fully keyframed the animations.&ldquo;When capturing full rotations, Xsens was stellar,&rdquo; Julien affirms. &ldquo;It&rsquo;s a strong point of the system. We captured a lot of different rotations in the air: spins, flips, and combined rotations.&rdquo;Ubisoft Annecy continued to use the Xsens suits for other purposes in the game following that initial shoot, including prototyping animations for other activities and developing the fun emotes that players can have their characters perform. The studio has two suits that it turns to regularly, as they don&rsquo;t have to book a mocap session: the technology is always ready.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1651805676381-Riders+Republic+3.png" style="width: 300px;" class="fr-fic fr-dib">&ldquo;One of the big advantages of Xsens for all of the shooting and prototyping we did is that you can hit record and let it roll, and you have all of the data,&rdquo; says Julien. &ldquo;For the shooting in Landgraaf, we were able to capture data across the whole snow park, and we were surprised because it was like 50 meters long or more. That was one of the big advantages. Xsens is also much faster to use for prototyping than some tools, and we use it a lot for that.&rdquo;Xsens is Ubisoft&rsquo;s global partner for inertial motion capture technology, and the durable MVN suits have been used at other Ubisoft studios across an array of blockbuster video game projects. Ubisoft Annecy plans to support Riders Republic extensively after its release, as it did with Steep.&ldquo;It&rsquo;s really useful to have Xsens in the studio when you don&rsquo;t have your own mocap studio,&rdquo; Julien concludes.

with Xsens MVN suits

Ubisoft hits the slopes for the Riders Republic

<a href="https://www.twentyonepilots.com/" rel="noopener noreferrer" target="_blank">21 Pilots</a> have performed for thousands of fans at the Grammys, Wembley Arena and Reading festival, but they have never had an experience quite like taking to the virtual stage in <a href="https://www.roblox.com/" rel="noopener noreferrer" target="_blank">Roblox</a> and their 35 million daily users. As the media and technology industries attempt to find their &lsquo;<a href="https://www.xsens.com/blog/creating-the-metaverse-with-motion-capture" rel="noopener noreferrer" target="_blank">Metaverse</a>&rsquo;, the digital world of Roblox provides a window into what the concert-going experience might look like in the next internet&rsquo;s next stage.Virtual concerts aren&rsquo;t new, but previous renditions have had their share of challenges, with the biggest hurdle being trying to encapsulate the energy of an in-person show in the digital realm. Despite this, it was essential for 21 Pilots that the emotion of their performance transcended across into the virtual space. This is why Actor Capture and <a href="https://www.gsu.edu/" rel="noopener noreferrer" target="_blank">GSU</a>&rsquo;s <a href="https://cmii.gsu.edu/" rel="noopener noreferrer" target="_blank">CMII</a> Studios sought out Xsens, implementing our inertia-based technology to bring the concert experience into players&rsquo; homes.We sat down with long-time Xsens collaborator, and <a href="https://actorcapture.com/" rel="noopener noreferrer" target="_blank">Actor Capture</a> Technical Director, James Martin, to discuss how the project came about, and he was quick to point out how essential Xsens technology was in capturing the 21 Pilots&rsquo; performance.&ldquo;It is essential for drummers and specifically for musical performances to utilize inertia based and furthermore, a hand solution that allows you to get the grip or things like notation on a fretboard or on a keyboard. It&#39;s very, very, very important. All of these things are prevalent and this performance.&rdquo;<h3><iframe width="640" height="360" src="https://www.youtube.com/embed/HOo2NbSRxjA?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe></h3><h3>Making use of Xsens&rsquo; tools&nbsp;</h3>Lil Nas X&rsquo;s Roblox show was a rousing success for Symbol Zero, but with their next virtual concert, they wanted to take things to the next stage, and engage the player at an unprecedented level. Having built up an in-depth understanding of Xsens through his work on past projects &lsquo;Suicide Squad 2&rsquo; and Cardi B&rsquo;s &lsquo;<a href="https://www.xsens.com/cases/how-cardi-b-created-a-video-game-with-xsens" rel="noopener noreferrer" target="_blank">Cardi Tries</a>&rsquo;, James knew that our technology was the best way to address the challenges 21 Pilot&rsquo;s virtual performance might have brought.&ldquo;We knew that the work would have over 15 minutes of animation. We also knew that they were very interested in doing face bonding and hands because instruments would be involved. They (Symbol Zero) weren&#39;t just looking for somebody who was versed in the moves and the Xsens workflow. They were looking for somebody who could utilize things like the Manus gloves that now carry the timecode over, which is very important to get the finger articulation. And then, of course, a face solution that was robust enough to get all of the lyric performance.&rdquo;Capturing the articulation of the performance was an important factor for Roblox and 21 Pilots. The band wanted to carry across the emotions of a concert to the virtual space, so it was essential the right tools and methods were chosen. MVN Link suits were worn by the band as they performed, while <a href="https://www.xsens.com/products/xsens-gloves-by-manus" rel="noopener noreferrer" target="_blank">Xsens Gloves by Manus</a> were also used to ensure hand movements were accurately captured. Alongside this, James made use of the iPhone&rsquo;s ARkit to track the face movements of 21 Pilots.<h3>The Metaverse experience</h3>In the virtual spaces that will make up the metaverse, it will be the individualities that the user will seek out to make themselves stand apart. For now, these come in the form of digital clothes, or NFT art, but as the user base grows, so does the adoption of aspects of reality. This is where the 21 Pilots Roblox concert might serve as a preview for the role Xsens can play in the metaverse. During the recording process, it was clear that there was value attributed to the movements of 21 Pilots. It was essential for the player that the way the drums or the guitar was played, matched that of the band. It&rsquo;s not out of the question that the metaverse will see similar demand for realistic, and unique avatar movements, which is where technology like the Manus gloves will come into play.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUwNDM4NzI2MzAwLTIxcGlsb3Rfc2NyZWVuc2hvdF8wMDHljovnvKkucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><h3>Recording the performance</h3>Whether the stage is real or virtual, 21 Pilots have always sought to have a connection with their loyal fan base. As explained by Mark Eshleman, the band&rsquo;s Creative Director, they wanted <a href="https://variety.com/2021/digital/news/twenty-one-pilots-21-roblox-concert-experience-1235058749/" rel="noopener noreferrer" target="_blank">&ldquo;to honor the band&rsquo;s fans, its lore, and history to make sure this experience aligns with what people can expect from a show.&rdquo;</a>The question is no longer whether performance can be captured for the virtual space, but more so, can the virtual concert capture the sophistication and emotion that comes with an in-person show? Thanks to Xsens&rsquo; inertia-based solutions, James was able to address these concerns and created an environment that allowed 21 Pilots to perform just as they would in any arena around the world.&ldquo;In the behind-the-scenes, you&#39;ll see that we had all their instruments in the CMII Studios stage, and we didn&rsquo;t hinder their performance. That&#39;s really the true magic of inertia-based solutions; we bring a solution to a performer and meet them in their own native environment.&rdquo;The success of the metaverse will be dictated by the ease of adoption and use. It will be the technology, like the Xsens tools that allowed James and Actor Capture to record the essential, intricate details of performing, that will be necessary for the metaverses&rsquo; ability to maintain a sustained suspension of disbelief.<h3>Engaging the player</h3>Hosting a concert within the world of Roblox allowed 21 Pilots new opportunities to interact with their audience in a way only possible in the virtual space. It was important for Symbol Zero that the player remained a part of the performance, rather than being a passive spectator.&ldquo;Where at first you&#39;re just a captive audience member just watching a performance, the interactivity that is offered through Roblox engages the game player. In the opening track that they utilized, one of the options was that the player would go up and down alongside the hook of the song, and if the player did not jump when Tyler said the lyric &lsquo;up&rsquo;, then they would fall through the floor. It&#39;s smaller interactivity that makes the game player have a little bit more of an intimate experience with the performance.&rdquo;Users will venture into virtual worlds in the metaverse for an enhanced experience of their reality, not just a 1-1 depiction of their reality. Roblox implemented quests, rewards and virtual merchandise into the 21 Pilots concert, not just as a way to sustain and market the performance but to keep players engaged in a way that differs from an in-person experience.

with Xsens MVN animate

How Xsens technology brought the 21 Pilots virtual concert to life in the Roblox Metaverse

&#39;What are we actually adding to a live performance?&#39; asked Ben Kee, founder, creative director, and executive producer of <a href="https://www.anomalyst.studio/" rel="noopener noreferrer" target="_blank">Anomalyst Studio</a> in Singapore, as his team embarked on the mixed reality live performance of ShiGGa Shay at the opening ceremony of the <a href="https://www.globalesports.org/post/singapore-2021-global-esports-games-breaks-the-meta-with-opening-ceremony" rel="noopener noreferrer" target="_blank">Global Esports Games 2021</a>. Mixed reality performance is a hybrid between virtual production and traditional, real-time production, and was brought to life in this show with <a href="https://www.xsens.com/motion-capture" rel="noopener noreferrer" target="_blank">motion capture</a> technology powered by Xsens. Kee explains how people are increasingly demanding that the virtual concert is not just a digital imitation, but an experience in its own right.<h3>How gaming facilitated a Promising Pivot into virtual production</h3>&ldquo;Anomalyst is a multimedia company that mainly produces content for live events. Over the last two years, with the COVID-19 pandemic, there have been no live events.&rdquo; Finding a solution to this problem, Anomalyst Studio had to change focus. &ldquo;We have had to pivot, in a strange way, into virtual production. We started with green screen solutions, virtual backdrops and experimenting with Unreal Engine and other game engines to create content.&rdquo;<a href="https://www.xsens.com/motion-capture" rel="noopener noreferrer" target="_blank">Motion capture</a> was already in Anomalyst Studio&#39;s pipeline when they decided to shift their focus to virtual event production. &ldquo;Our partners and friends are performers and musicians who have been looking to stream their performances, so they would still be able to work during the pandemic,&rdquo; said Kee. &ldquo;The demand for virtual productions gave us the idea of improving the way we live stream performances using a games engine, a performer, and motion capture.&rdquo; In creating the mixed reality performance of ShiGGa Shay at the Global Esports Games 2021, the studio had to think on its feet to bring the performance together, as the new format meant there were new challenges in meeting their objectives for the opening ceremony.<iframe width="640" height="360" src="https://www.youtube.com/embed/borCiOzRfew?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe>ShiGGa Shay performs to his song, uTopia reimagined: chasing.<h3 id="isPasted">The highs and lows of being first</h3>To be the first to produce a mixed reality performance live was a feat itself, but to be the first in an industry that&#39;s growing as fast as Esports was phenomenal. In 2021, the global worldwide esports audience reached 474 million people and is estimated to reach 577 million by 2024. Anomalyst themselves were effectively performing to a large audience. Being the pioneer brings its own set of challenges. &ldquo;Knowing how to plan was difficult, but thankfully, our clients gave us creative freedom.&rdquo; From the timing and setup of virtual assets to real logistical issues, Anomalyst Studio took up the challenge: &ldquo;A virtually produced performance was a new creation for us and our client&rdquo;, explains Kee. &ldquo;Our setup on-site, at the Global Esports Games, was only three days before the event when we got entry to the event grounds. The content for the show could not be made before seeing the space at the event grounds. We had to run and test the performance to make sure it worked with the rest of the show and the requirements of our clients.&rdquo;Kee and his team, whilst challenged by the logistics, were also inspired by their own high expectations, as well as their creativity: &ldquo;Anomalyst has the objective of ensuring that we are working beyond just the expectations of the content produced.&rdquo; For example, Kee wanted to have virtual assets that added to the vibe and movement for the performance, so he &ldquo;enlisted a colleague who is a dancer and pre-recorded some of the dance moves to use as an additional asset in the environment. I think we just decided what felt right and looked at our toolset and saw what we could build and what we were able to pull through with the dancers.&rdquo; While the logistics were challenging for Anomalyst Studio, the team had the help of <a href="https://www.xsens.com/products/mvn-animate" rel="noopener noreferrer" target="_blank">Xsens suits</a> which they had specifically chosen for their reliability, to bring the mixed reality performance to life.<h3>Xsens&#39;s role in realizing live movement in the virtual world</h3>The creative direction of the Anomalyst team and their experience in event planning and execution made the project feasible, but Kee highlights that the technology they opted for was important in realizing the movements they imagined. Movella&#39;s mission to bring meaning to movement aligned with the energy that Kee&#39;s team wanted to bring; the technology served as a tool to create digital movement that imitated the natural flow of the dancers: &ldquo;Xsens captures motion in a way that gets the vibrancy of the movements across.&rdquo;&ldquo;The Xsens suit has the feature that allows the performer to jump off the ground without having their legs clipped into the ground after they land&rdquo;, elaborates Kee. &ldquo;There is less sliding. The suit can convey the power and agility of each movement. Having tested other suits, we needed a suit that was a lot easier to set up because we were creating this illusion for an in-venue event. I think Xsens came across as a superior motion capture solution that is more of an inside-out tracking rather than outside-in like others available. &rdquo;The Xsens suit was key in projecting the energy of the mixed reality performances across to viewers at the Global Esports Games. The performance has amassed attention for its unique format and ability to engage viewers. For Anomalyst, the attention created around the event has confirmed the power of mixed reality performances to share ideas.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1647609857715-ShiGGa+Shay+-2.JPG" style="width: 300px;" class="fr-fic fr-dib"><h3 id="isPasted">The art of storytelling through virtual production</h3>&ldquo;I believe mixed reality can be a storytelling mechanism, which I think a lot of traditional show directors and filmmakers are looking into right now.&rdquo; shares Kee. The digitisation of live events is bright, but the first examples being set will be key to maintaining and driving interest: &ldquo;We were able to capture the attention of the national broadcaster in Singapore, they were very interested in visiting us and understanding the mixed reality performances we&#39;re able to create. People are excited about these technologies, it&#39;s just that they don&#39;t yet know what&rsquo;s been done,&rdquo; explains Kee.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ3NjA5MjEzMjQ1LVNoaUdHYSBTaGF5IC0zLkpQRyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">

An Interview with with Ben Kee, Founder Anomalyst Studio

Live Movement in a Virtual World with Xsens x Anomalyst Studio

3D Positional Aiding and Object Tracking complete MVN Animate, revolutionizing inertial motion capture <iframe src="https://www.youtube.com/embed/pno_NuhXzKg?&amp;t=28s&amp;wmode=opaque" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 788px; height: 466px;"></iframe> &nbsp;We proudly present the release of Xsens MVN Animate 2020 – a major update to complete Xsens’ inertial motion capture solution for studio professionals. With this development, <a href="https://www.xsens.com/products/mvn-animate" rel=" noopener" target="_blank">MVN Animate</a> can use positioning data from HTC Vive and fuse it with Xsens motion capture data for persons and objects.The new release also enables massive motion capture shoots with the new Remote On-Body Recording feature and deeper third-party hardware integration with the new Network sync. Process exact positions in a spaceThe new 3D Positional Aiding functionality is embedded within the MVN Animate engine, allowing users to process exact positional data. The Positional Aiding tracks positioning using HTC VIVE lighthouses, the leading positional tracking technology. The exact positional data from VIVE is then processed within MVN Animate software and fused with body motion capture data to present one clear picture. This new development unlocks new opportunities for studios – two or more people can be measured in a space at the exact same time, meaning groups of actors in a scene can be recorded with their relative positioning tracked. Track Objects in the FieldUsers can place sensors on objects within a space –&nbsp;such as an actor’s prop – and process the objects’ exact location in relation to an actor. When recording the body motion data of a group of people, animators can plot the position of the objects, contextualizing the motion and positional data within the environment. Remote On-Body RecordingIn any location, users can track the body motion of an unlimited number of actors simultaneously. Large, group animations can now be completed accurately with ease, and Xsens automatically transfers your data, negating the need to manually download it after the session. Synchronous motion and facial captureUsing other motion technology alongside MVN Animate has never been easier. With the new Network Synchronization feature, other tracking devices, such as <a href="https://www.xsens.com/watch-webinar/Xsens-and-Faceware" rel=" noopener" target="_blank">Faceware’s</a> and <a href="https://www.xsens.com/watch-webinar/xsens-full-body-plus-Dynamixyz-Facial-MoCap" rel=" noopener" target="_blank">Dynamixyz</a> facial capture technology can be managed from one interface. Just press record on MVN Animate to begin capturing from all devices simultaneously. International usabilityThis release includes the addition of Japanese and Korean languages, alongside the Chinese and English language options, expanding global usability.MVN Animate 2020 now offers animators around the world one of the most comprehensive multi-sensor motion data integration systems, demonstrating a degree of precision and ease-of-use that takes motion capture to the next level.&nbsp;Xsens MVN Animate 2020Learn more about <a href="https://www.xsens.com/products/mvn-animate" rel="noopener noreferrer" target="_blank">Xsens MVN Animate 2020 here</a>.

entertainment industry

Latest Posts

Ubisoft hits the slopes for the Riders Republic

How Xsens technology brought the 21 Pilots virtual concert to life in the Roblox Metaverse

Live Movement in a Virtual World with Xsens x Anomalyst Studio

MVN Animate 2020, revolutionizing inertial motion capture