Algorithm that can recreate 3D objects from tiny 2D images

Aug. 25, 2017, 2:13 p.m. By: TechLeer Admin

Algorithm

We are very good at judging shapes of objects even if we look at it for a brief moment. But, this is where computer falls short. They cannot identify the shape of objects like blocks by just seeing it for a small time. But, now this problem can be solved using the new algorithm created by Berkeley AI researcher. It’s quite useful to see something in 2D and then guess the volume that it will take. This can have implementations in various sectors like AR, VR. It might sound simple but in a practical situation, this is very hard.

Converting something from 2D to 3D means we have to handle lot more data. Even a slight change in the shape of the 2D object would mean a heavy change in the 3D world. An accurate reproduction of an image, having 100 pixels per side, making for a total of a million pixels — voxels. But if want to go for 128 pixels, then you will need 2 million voxels. What’s in each voxel can be calculated by analyzing the image but the calculation will pile up fast in case of any real fidelity. It was one of the main bottlenecks for the possibility of extrapolating 3D forms from 2D images.

Cristian Hane figured a simpler and more computational clever way of doing this one. He found out that we do not calculate a whole volume of 100x100x100, but only try to describe the surface of an object. The empty space around it and inside it does not matter. He created a system that renders a 3D reconstruction of the 2D image in very low resolution. He observed that the outer third of the whole volume appears to be empty, which he threw away. Next, he did a higher-resolution render of the area that he kept.

He found out that the top and bottom are empty, but the middle is full of pixels, except for a big chunk in the center. So, he threw out the empty bits and repeated the process. At last, he left with a 3D volume of high spatial resolution that has taken comparatively little calculation to produce as he only calculated parts he knew have meaningful information.

On comparing his model with the traditional ones, the output images were of same (even better quality in some cases). This system might not be perfect as humans but sure it creates a good platform to start. This discovery clear indicates that we are moving closer toward our aim of creating a computer with human capabilities.

Cristian Hane research paper: Arxiv.org and Cristian Hane Blog: Click Here

Image Source: Anith