Yuma Wakahara has received B.E. and M.E. degrees of Science degrees from Department of Chemistry at Keio University in 2014 and 2016. He joi...read more

Yuma Wakahara has received B.E. and M.E. degrees of Science degrees from Department of Chemistry at Keio University in 2014 and 2016. He joined NHK in 2016. Since 2019, he has been researching and developing volumetric video technology at the Science & Technology Research Laboratories of NHK. He is a member of ITE.
close

Volumetric videos contain dynamic three-dimensional (3D) scenes that provide immersive experiences to users when used in augmented and virtual reality applications. Visual-hull, stereo-matching, and neural network-based methods are typically employed to reconstruct the 3D visuals...read more

Volumetric videos contain dynamic three-dimensional (3D) scenes that provide immersive experiences to users when used in augmented and virtual reality applications. Visual-hull, stereo-matching, and neural network-based methods are typically employed to reconstruct the 3D visuals of an object using multi-view images captured by multiple cameras surrounding the object. While most volumetric video capturing systems only provide information about the 3D shape and colors of an object, an integrated volumetric capturing system, called Meta Studio, proposed in this study provides additional information on the reflection coefficients and surface light fields. This unique feature enables an accurate reproduction of the actual lighting conditions of the captured object, allowing for photo-realistic re-lighting.
To facilitate the widespread adoption of volumetric videos, interoperability must be established such that volume data can be transmitted from TV stations to homes and rendered on a variety of devices. Therefore, research on file formats and rendering systems is essential. Existing research on volumetric video file formats has focused on conventional 3D file formats, which are not suitable for handling the additional parameters included in Meta Studio volumetric video data; therefore, a custom file format has been used to store such video data.
This paper proposes a hierarchical volumetric video format comprising base and enhancement layers. The base layer format encompasses fundamental attribute information such as vertex coordinates, colors, and reflection coefficients. Furthermore, the base layer information is represented as an extension of existing file formats, and the uncompressed base layer is represented in the Stanford triangle format (.ply) using vertex elements. Additionally, a compressed base layer format is proposed. As the proposed base layer format focuses on point cloud-based volumetric video, an extension of the MPEG volumetric video format, called visual volumetric video-based coding, is used. Furthermore, the study investigated an efficient rendering system for the base layer and conducted preliminary verification of the enhancement layer. The results show that the proposed system has high interoperability and renders efficient file formats. Overall, our proposed hierarchical volumetric video format, combined with an efficient rendering system, aims to advance the use of volumetric video technology and enable seamless integration across diverse platforms and devices.
close

Natalia Molinero Mingorance, a telecommunications engineer, received her BEng degree from the European University of Madrid, Spain, in 2013....read more

Natalia Molinero Mingorance, a telecommunications engineer, received her BEng degree from the European University of Madrid, Spain, in 2013. She continued her academic journey and obtained an MSc degree in Wireless Communications Systems from the University of Sheffield, U.K. From 2014 to 2017, Natalia served as a Research Associate in the Communications group of the Electrical and Electronic Engineering department at the University of Sheffield.

After her tenure at the University, Natalia joined Netmetrix Solutions, a telecommunications company based in Madrid, Spain, where she worked until November 2022. In April 2022, she was honored with the Alumni Singular first prize for her project proposal titled "New Methods for Energy Efficiency Optimization: Information, Generation, Storage, Transmission, and Reception" by the European University of Madrid and Banco Santander. Presently, Natalia is actively involved in the development of this project, known as "Harmony Valley," which aims to address energy efficiency concerns and streamline distribution processes.

Currently, Natalia is balancing her research and development commitments in Harmony Valley with her new role at Airbus. At Airbus, she has the opportunity to explore her other interests, such as wireless communications and cybersecurity. Her contributions to both the research project "Harmony Valley" and her work at Airbus reflect her dedication and commitment to advancing telecommunications and aerospace domains.
close

Over the past decade, the utilization of video consumption apps has experienced an unprecedented surge in popularity, and this trend is set to soar to new heights with the advent of the metaverse. As the metaverse continues to expand, it brings with it a substantial surge in the ...read more

Over the past decade, the utilization of video consumption apps has experienced an unprecedented surge in popularity, and this trend is set to soar to new heights with the advent of the metaverse. As the metaverse continues to expand, it brings with it a substantial surge in the magnitude of digital data and places an immense computational load on networks, data centers, and user devices. The constant processing of this vast amount of information inevitably leads to significant energy consumption, resulting in an alarming quantity of CO2 emissions on an annual basis.

Addressing this formidable challenge necessitates prioritizing the development of lightweight compression algorithms that can effectively reduce video file sizes and facilitate efficient transmission over the Internet. However, despite efforts in this domain, the current compression standards fall short of achieving the desired level of efficiency. Consequently, there is an urgent need to delve deeper into this issue and explore alternative approaches to video compression.

In this study, we embark on a thorough analysis of the resource-intensive tasks employed by two state-of-the-art compression algorithms, namely VP9 and HEVC. A novel implementation in Matlab has been developed specifically for this purpose, focusing on the Motion Estimation (ME) process of both algorithms, to enable a comprehensive and objective comparison. Moreover, this implementation incorporates innovative methodologies to evaluate sustainability factors, thereby considering the environmental impact of these algorithms.

Through this meticulous analysis, valuable insights have been gleaned, revealing crucial areas that require further refinement in future video compression algorithms. These findings are poised to revolutionize the field of video transmission in the metaverse, leading the way towards sustainable and optimized practices. By addressing the limitations of current compression standards, we can mitigate the environmental impact of video consumption in the metaverse and pave the path for a greener future.

Furthermore, the implications of this research extend beyond the metaverse itself. As video consumption continues to rise across various domains, from social media to online education, the need for sustainable video transmission becomes increasingly vital. The lessons learned from this study can be applied to enhance video compression algorithms across the board, benefiting not only the metaverse but also the broader digital landscape.

This study provides a comprehensive analysis of two prominent compression algorithms, highlighting their strengths, weaknesses, and environmental impact. The findings underscore the urgency of developing more efficient and sustainable video compression techniques, paving the way for optimized video transmission in the metaverse and beyond.
close

Dr. Jason Kao is an Associate Professor in Game Developments at the Game School, Inland Norway University of Applied Sciences based in Hamar...read more

Dr. Jason Kao is an Associate Professor in Game Developments at the Game School, Inland Norway University of Applied Sciences based in Hamar. His research outputs lies in visual perception, immersion evaluation and extended reality (XR) applications. His recent projects explore the further cinematic realism and narrative immersion through utilizing augmented reality.
close

Karen Kao is a lecturer in Marketing at the Glasgow Caledonian University London in the UK, whose research interest lies in digital consumer...read more

Karen Kao is a lecturer in Marketing at the Glasgow Caledonian University London in the UK, whose research interest lies in digital consumer behavior studies related to marketing strategy in promotion. She holds a decade of experience in industrial marketing and sales before her academic career. She publishes in the Journal of Retailing Consumer Services, Internet Research, and Journal of Computer Information Systems.
close

The pursuit of cinematic realism is unceasingly driven forward by technologies. The two phenomenal booms of stereoscopic three-dimensional (S3D) films in cinema history have verified three-dimensional as an promising spectatorship for immersion cinema. However, the depth-enhanced...read more

The pursuit of cinematic realism is unceasingly driven forward by technologies. The two phenomenal booms of stereoscopic three-dimensional (S3D) films in cinema history have verified three-dimensional as an promising spectatorship for immersion cinema. However, the depth-enhanced photorealism formed by binocular disparity in S3D limits the content to be perceived as so-called three-dimensional as objects cognized in physical reality. That is, S3D’s mechanism is built upon photographical images to achieve the depth illusion, which is doubt to bring perceptual realism that objects are perceived as existing objects to the perceiver. From this regard, this article proposes a concept of applying extended reality (XR) to bring the potential perceptual realism into cinema; we name it ‘Augmented Dimension’.
With the thriving of XR, using XR goggles for entertainment can be realized in the near future. Virtual production has been a fast-developing topic in the filmmaking industry investigating potentials of implementing game engines for a more efficient workflow. The tendency of integrating filmmaking into the technical platforms of gaming and XR is clear. As a visual perception researcher and a practitioner in XR, authors of this article elaborate the trend by specifying the potential of utilizing augmented reality (AR) as a medium for a new level of cinematic realism. The borderless feature in AR triggers the viewer’s psychological and emotional responses to immerse in both the physical reality and the virtual content. That is, AR’s feature of placing virtual content in physical reality creates the foundation of perceiving perceptual realism particularly in the condition of utilizing AR goggles. This article introduces the integration of contemporary screen medium and AR to form an innovative viewing approach to perceive the associated screen and AR contents simultaneously. The physical space outside of the viewing screen frame forms an expansion of the field of view for storytelling. We call such a field ‘Augmented Dimension’ (AD). Compared with S3D, the three-dimensional perception in AD can be considered as perceptual realism further than photorealism. Furthermore, with the advancement of Computer-Generated Imagery (CGI) technology, high resolution and realistic rendering is feasible in AR devices. These advantages suggest a great potential that the cinematic realism in AD can be brought to a new level that the three-dimensional experience is overwhelming and boundaryless.
close