Intensity control of projectors in parallel – a doorway to an augmented reality future

A challenge to adopting augmented reality (AR) in wider applications is working with dynamic objects, owing to a delay between their movement and the projection of light onto their new position. But, Tokyo Tech scientists may have a workaround. They have developed a method that uses multiple projectors while reducing delay time. Their method could open the door to a future driven by AR, helping us live increasingly technology-centered lives.

Technological advancements continue to redesign the way we interact with digital media, the world around us, and each other. Augmented reality (AR), which uses technology to alter the perception of objects in the real world, is unlocking unprecedented landscapes in entertainment, advertising, education, and across many other industries. The use of multiple projectors plays an important role in expanding the usage of AR, alongside a technique called projection mapping. However, an obstacle to the widespread adoption of AR is the application of this method to moving, or “dynamic,” targets without the loss of immersion in the AR space.

This technique, known as dynamic projection mapping, relies on a blend of cameras and projectors that visually detect target surfaces and project onto them, respectively. A critical aspect is the need for high speed in information transfer and low “latency,” or delay between detection and projection. Any latency leads to a misalignment of the projector, which effects our perception, and reduces the effectiveness of the AR space.

Other issues like changes in shadowing and target overlap are solved easily by using multiple projectors. However, the addition of new projectors correspondingly drives up the latency. This is a result of the need to calculate the intensity at every pixel simultaneously for every frame of a moving scene. Simply put, more projectors lead to longer and more complex calculations. The latency is a massive hurdle towards AR taking a true foothold in broader applications across society.

Thankfully, a team of scientists at Tokyo Institute of Technology (Tokyo Tech), led by Associate Professor Yoshihiro Watanabe, might just have the necessary solution. They have developed a novel method to calculate the intensity of each pixel on a target in parallel, reducing the need for a single large optimization calculation. Their method relies on the principle that if pixels are small enough, they can be evaluated independently. While based on an approximation, their results, published in IEEE Transactions on Visualization and Computer Graphics, suggest that they could achieve the same quality of images as conventional, more computationally expensive methods, while drastically increasing the mapping speed and thereby reducing the latency.

“Another advantage of our proposed method is, as there is no longer need for a single global calculation, it allows the use of multiple rendering computers connected through a network, each only controlling a single projector,” explains Dr. Watanabe. “Such a network system is easily customizable to incorporate more projectors, without major sacrifices to the latency.”

This new method can allow large spaces with many projectors for efficient dynamic projection mapping, taking us a step closer to broader AR applications, as Dr. Watanabe describes: “The presented high-speed multi-projection is expected to be a major part of important base technologies that will advance spatial AR to derive more practical uses in our daily life.”

We are excited to see how this technology evolves and how it illuminates the path towards an AR-centric future!

Related Links

In Good Light: A New High-Speed Projector with Visible and Infrared Capabilities 

The future of the virtual and the real | Taking your first step as a researcher 

Four teams receive first DLab Challenge research grant 

Watanabe Laboratory

Demonstration Video: https://youtu.be/ltwSmsYnlK8

About Tokyo Institute of Technology

Tokyo Tech stands at the forefront of research and higher education as the leading university for science and technology in Japan. Tokyo Tech researchers excel in fields ranging from materials science to biology, computer science, and physics. Founded in 1881, Tokyo Tech hosts over 10,000 undergraduate and graduate students per year, who develop into scientific leaders and some of the most sought-after engineers in industry. Embodying the Japanese philosophy of “monotsukuri,” meaning “technical ingenuity and innovation,” the Tokyo Tech community strives to contribute to society through high-impact research.

https://www.titech.ac.jp/english/

Dynamic Multi-projection Mapping Based on Parallel Intensity Control

Article Publication Date