Smartphone super-resolution tech plays 4k, 60fps videos in real-time
Wed 25 Nov 2015
A super-resolution technology which is capable of streaming up to 4k of real-time video data at 60fps is being developed at Kogakuin University in Japan.
The project, which started in March this year, involved collaboration from University researchers led by Seiichi Gohshi, professor at the Department of Information Design, and a team from Japanese IT firm Fujitsu. The new technology was designed for the Xevic image processing engine, installed in Fujitsu’s ARROWS NX F-02H smartphone.
Gohshi explained that unlike ‘reconstructed super-resolution’ and ‘learning super-resolution’, the new super-resolution technology employs an innovative method known as ‘nonlinear signal processing.’ He described that the function is applied to the edges detected by applying a high-pass filter to input images. Gohshi added that the data is then compressed and added to the original images.
Although this technique shares similarities with traditional enhancer tools, it is unique in that the nonlinear aspect can be used to supplement ‘high-frequency components.’ It therefore becomes possible to reproduce high-resolution images that exceed the ‘Nyquist frequency’ – the theoretical limit of super-resolution.
Additionally, unlike standard super-resolution technologies, the new practice does not require repeated calculations. It uses a simple algorithm, which makes it suitable for real-time processing.
Research on the nonlinear method began in 2012 in cooperation with imaging company Keisoku Giken, and is already available as a production model in its FE super resolution unit which expands high-vision video data to 4k video data.
This latest research, headed by Gohshi, further simplifies the algorithm. Fujitsu has also made modifications to task division and time control. This means that real-time footage has been realised using the smartphone software without the need for a tailored hardware unit.
The new breakthrough can now play 4k of video data with a frame rate of 30fps, without increasing the power consumption of the smartphone. The research team also confirmed that 60fps could be achieved.
The scientists hope that the technology could eventually be used beyond consumer smartphones, in medical equipment and monitoring cameras.
This year YouTube began testing 4k, 60fps video settings. However not all viewers can benefit, with the feature requiring a 4k monitor and an internet connection powerful enough to play the footage.