Sony announces the development of a spatialized content creation system

Sony Corporation announces the development of a spatial content creation system, featuring an XR display headset with high-quality 4K OLED microdisplays, a natural vision function and a pair of controllers optimized for precise pointing and intuitive interaction with 3D objects. The system is designed to help creators produce sophisticated 3D content. Sony is planning partnerships with developers of various 3D creation software, notably in the entertainment and industrial sectors. At the launch, Sony will partner exclusively with Siemens, a leader in industrial technologies, to launch a new immersive design and collaborative engineering solution using software from the Siemens Xcelerator open digital business platform.

The system will be available in 2024. Sony and its partners will present further details at a later date, including specifications, launch dates and regions, pricing, sales channels and software compatibilities.

This spatialized content creation system, featuring an XR headset with 4K OLED microdisplays and Sony’s proprietary display technologies, delivers real-time, realistic, high-definition rendering with textures, 3D objects and human character facial expressions. In addition to the ability to see through the headset and spatial position detection thanks to a total of six cameras and sensors, the system features a ring controller that allows users to intuitively manipulate objects in virtual spaces, as well as a pointing controller that enables precise designation: users can thus create in a virtual space with controllers and keyboards, without removing the display headset.

Designers will not only be able to view full-size 3D models in an Extended Reality (XR) environment thanks to the high-definition display, but also create and modify 3D models. Integration with third-party 3D authoring applications also enables remote, real-time preview from one location to another, making this equipment an intuitive, immersive tool for the entire spatialized content creation process.

The device is based on a Snapdragon XR2 Gen 2 platform, the latest XR processor from Qualcomm Technologies, Inc. which unleashes the capabilities of 4K OLED microdisplays to deliver stunning image quality and give creators a high-performance platform for their creative workflow.

Sony offers technologies that improve the workflow and quality of spatial content, while simplifying the use of 3D CGI in a variety of creative fields. The ” mocopi ” mobile motion capture system, with small, lightweight sensors and a dedicated smartphone app, tracks full-body movements using Sony’s exclusive technologies. The Spatial Reality Display system offers extremely realistic three-dimensional content without the need for special glasses or virtual reality (VR) headsets, simplifying the creation and verification of spatialized content. By introducing this new system and collaborating with 3D creation software developers, Sony aims to empower spatial content creators to transcend the separation between the physical and virtual worlds, in order to deliver more immersive creative experiences.

System used to create 3D computer-generated images

System used for film production

” For many, the metaverse is a place to escape the real world. But for Siemens and our customers, the industrial metaverse is where you create the real world! It’s where we study climate change, globalization, digitization. It’s where we go to transform the industries that form the backbone of our economies,” explains Cedrik Neike, Member of the Management Board of Siemens AG and CEO of Siemens Digital Industries. ” In association with Sony, we are presenting the latest brick in our industrial metaverse: immersive engineering. It provides an environment for experiencing the realities of physics, without the limitations of time. The industrial metaverse, enhanced by immersive engineering, brings real and digital worlds together as never before: it will affect the way our customers work, in depth and on a daily basis.”

” We designed our Snapdragon XR2 Gen 2 to handle 4K definition display, state-of-the-art graphics and unmatched performance to enable creators to build innovative spatialized content that will transform the industry for the better ,” says Said Bakadir, Senior Director of Product Management at Qualcomm Technologies, Inc. “It’s exciting to see Sony enter the enterprise world with its XR hardware and use Snapdragon XR technologies to unleash more realistic, detailed and accurate virtual or mixed reality (MR/VR) experiences. This will help developers and creators showcase an even more immersive future. ”

Key features

High-quality XR experience for creative uses

The display headset features large (1.3″ type), high-definition 4K OLED microdisplays, with an extended gamut covering up to 96% of the DCI-P3 color space. Realistic, detailed rendering of the textures of 3D objects and the facial expressions of human characters means creators can follow the entire process, from modeling to final checks, without ever taking off the headset.

By supportingsplit rendering, which divides the workload between the computers and the headset, the system is able to deliver stable, high-definition rendering of large 3D models.

With this collaboration with Qualcomm Technologies, Inc. Sony is among the first users of their latest Snapdragon XR2 Gen 2 platform. The processor unleashes the power of 4K OLED microdisplays to deliver stunning image quality, while the AI built into the processor enhances user and volume tracking for smoother XR experiences: the physical and digital worlds merge seamlessly, making it easy and intuitive to navigate creative spaces in extended reality.

Controllers that deliver an intuitive experience of spatialized content creation

The system incorporates a ring controller, enabling users to intuitively manipulate objects in virtual space. It also features a pointing controller for stable, precise designation in virtual spaces, whose shape and button layout are optimized for efficient, precise use. By holding the pointing controller in the dominant hand and attaching the ring controller to the fingers of the secondary hand, designers can model 3D objects using both controllers and a keyboard, while wearing the display headset.

Intuitive object manipulation using the controller

The controller enables precise pointing in spaces

Designed for comfort and smooth XR creative experiences

The display headset offers the stability and comfort required for prolonged creative use: its center of gravity, design and the materials of the pads in direct contact with the head have been precisely studied. What’s more, the display’s flip-up mechanism enables designers to move easily and quickly from virtual to physical space, without having to recalibrate the device by taking it off and putting it back on again.

ideal balance for prolonged creative use

Display lift mechanism

Immersive creative experience thanks to integration with 3D design software

Sony is seeking partnerships with various 3D production software developers, both in entertainment and industry. At launch, Sony will have an exclusive partnership with Siemens to bring innovative immersive engineering capabilities to the manufacturing industry through the integration of the new NX Immersive Designer software, a product engineering solution that is part of the Siemens Xcelerator industrial software portfolio. The integration of powerful hardware and software will empower designers in the fields of industrial design and product engineering with more immersive design, verification and collaboration tools.