A review and comparison of multi-view 3D reconstruction methods
Abstract
ABSTRACT
3D reconstruction from multiple views is well studied, fundamental, yet a challenging problem in the field of computer vision. There is a large variety of approaches available in the literature. The methods use different representations for input scene/object and may provide different kinds of outputs. Some methods model entire scene as voxel-set where as some use level sets or polygon mesh representation. Output may be either volume or surface representing the reconstructed object/scene. Some methods work in image space where as some methods work in object space. These methods are developed to offer a good compromise between computation speed, computation complexity and accuracy along with feasibility in implementation. Selection of a particular method depends on the requirements of application and availability of required resources. Though, earlier reviews are available in the literature, fast advances in this field demand latest review. The paper presents a review and comparison of latest multi-view 3D reconstruction methods. This will help researchers to understand state of the art in this field.
References
Adipranata & Soo, 2007. 3D object reconstruction using voxel colouring with mean charts colour consistency checking. Proceedings of International Conference in Advances in Computer Science and Technology: 206-211.
Alexiadis, D. S., Zarpalas, D., Daras P., 2013. Real-time, full 3-D reconstruction of moving foreground objects from multiple consumer depth cameras. IEEE Transactions on Multimedia, 15(2): 339-358.
Bottino, A. & Laurentini A., 2003. Introducing a new problem: Shape-from-silhouette when the relative positions of the viewpoints is unknown. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(11): 1484-1493.
Boykov, Y. & Kolmogorov V., 2004. An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(9)1124-1137.
Bradley, D., Boubekeur T., Heidrich, W., 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), Jun. 08, 1-8.
Brisc F., 2004. Multi-resolution volumetric reconstruction using labeled regions. 6th IEEE Southwest Symposium on Image Analysis and Interpretation, Mar. 04,114-118.
Campbell, N.D.F., Vogiatzis, G., Hernandez, C., Cipolla, R., 2011. Automatic object segmentation from calibrated images. Proceedings of European Conference on Visual Media Production (CVMP), Nov.11, 126-137.
Campbell, N.D.F., Vogiatzis, G., Hernandez, C., Cipolla R., 2008. Using multiple hypotheses to improve depth-maps for multi-view stereo. Proceedings of 10th European Conference on Computer Vision: Part I (ECCV’08), 766-779.
Chang, J. Y., Lee, K. M., Lee, S. U., 2007. Multiview normal field integration using level set methods. Proceedings IEEE International Conference on Computer Vision and Pattern Recognition, Jun. 07, 1-8.
Cremers, D. & Kolev K., 2011. Multiview stereo and silhouette consistency via convex functionals over convex domains. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(6)1161-1174.
Dainese, G., Marcon, M., Sarti, A., Tubaro, S., 2005. 3D object modeling with a voxelset carving approach. Proceedings of IEEE International Conference on Image Processing (ICIP 05), Sep. 05, 1: 1053-1056.
Dinh, H. Q., Turk, G., Slabaugh, G., 2002. Reconstructing surfaces by volumetric regularization using radial basis functions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(10)1358-1371.
Divyalakshmi K & Vaithiyanathan V., 2016. Image registration techniques based on the Scale Invariant Feature Transform. IETE Technical Review: 1-8.
Dyer C., 2001. Volumetric scene reconstruction from multiple views. Foundations of Image Understanding: 469-489.
Feldmann, T., Brand, K., Worner, A., 2010. Enhancing voxel carving by capture volume calculations. 17th IEEE International Conference on Image Processing (ICIP): 4053-4056.
Franco, J. & Boyer E., 2009. Efficient polyhedral modeling from silhouettes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(3): 414-427.
Furukawa Y., Ponce, J., 2010. Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8):1362-1376.
Goldluecke, B. & Magnor, M., 2004. Space-time isosurface evolution for temporally coherent 3D reconstruction. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2004), Jul. 04, 1, I350-I355.
Hernandez, C., Vogiatzis, G., Cipolla, R., 2008. Multiview photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3):548-554.
Hu, X. & Mordohai, P., 2012. Least commitment, viewpoint-based, multi-view stereo. Proceedings of Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT’ 12), Oct.12, 531-538.
Ilic, S. & Fua, P., 2006. Implicit meshes for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(2):328-333.
Ju, X., Siebert, J.P., Khambay, B. S. , Ayoub, A. F., 2009. Self-correction of 3D reconstruction from multi-view stereo images. IEEE 12th International Conference on Computer Vision Workshops (ICCV workshops): 1606-1613.
Khan, S. M., Yan, P., Shah, M., 2007. A homographic framework for the fusion of multi-view silhouettes. Proceedings of IEEE International Conference on Computer Vision, Oct.07, 1-8.
Kim, Y. M., Theobalt, C., Diebel, J., Kosecka, J., Miscusik, B., Thrun, S., 2009. Multi-view image and ToF sensor fusion for dense 3D reconstruction. IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops): 1542-1549.
Kimura, M., Saito, H., Kanade, T., 1999. 3D voxel construction based on epipolar geometry. Proceedings of IEEE International Conference on Image Processing (ICIP-99) , Oct.1999, 3,135-139.
Kolev, K. & Cremers, D., 2008. Integration of multiview stereo and silhouettes via convex functionals on convex domains. Proceedings of European Conference on Computer Vision (ECCV), 752-765.
Labatut, P., Pons, J. P., Keriven, R., 2007. Efficient multi-view reconstruction of large-scale scenes using interest points, delaunay triangulation and graph cuts. Proceedings of IEEE International Conference on Computer Vision, Oct. 07, 1-8 .
Labatut, P., Pons, J. P., Keriven, R., 2009. Hierarchical shape-based surface re-construction for dense multi-view stereo. Proceedings of IEEE International Conference on Computer Vision Workshops (ICCV) , Sep.09,1598-1605 .
Lambert P., Hebert P., 2009. Robust multi-view stereo without matching. Proceedings of IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops): 1614-1621 .
Lhuillier, M. & Quan, L., 2005. A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3): 418-433.
Li, J., Li, E., Chen, Y., Xu, L., Zhang, Y., 2010. Bundled depth-map merging for multi-view stereo. IEEE Computer Society Conference on Computer Vision and Pattern Recognition: 2769-2776.
Liu, S., Kang, K., Tarel, J. P., Cooper, D. B., 2008. Free-form object reconstruction from silhouettes, occluding edges and texture edges: A unified and robust operator based on duality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(1)13-146.
Liu, Y., Cao, X., Dai, Q., Xu, W., 2009. Continuous Depth Estimation for Multi-view Stereo. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Jun.09,2121-2128.
Matusik, W., Buehler, C., Raskar, R., Gortler, S., McMillan, L., 2000. Image based visual hulls. Proceedings of ACM SIGGRAPH ’00, ACM Computer Graphics: 369-374.
Montenegro, A. A., Velho, L., Carvalho, P., Sossai, J. J., 2006. Polygonization of volumetric reconstructions from silhouettes. 19th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI'06), Oct. 06, 11-18.
Moretto, N., Cugola, N., Frezza R., 2003. Volumetric reconstruction by random sampling. Proceedings of 42nd IEEE Conference on Decision and Control, Maui, Hawaii USA, Dec.03,5, 4896 - 4901 .
Nehab, D., Weyrich, T., Rusinkiewicz, S., 2008. Dense 3D reconstruction from specularity consistency. Proceedings of IEEE Conference on computer vision, Jun.08, 1 - 8.
Nghiem, V. N., Cai, J., Zheng, J., 2010. Rate-distortion optimized progressive 3D reconstruction from multi-view Images. 18th Pacific Conference on Computer Graphics and Alications (PG), Sept.10, 70-77.
Nurzynska, K., 2009. Constrained dynamical surface reconstruction from unorganized point cloud. Proceedings of IEEE EUROCON’09, May09, 432-437.
Pons, J. P., Keriven, R., Faugeras, O., 2007. Multiview stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision, 72(2): 179-193.
Salvador, J. & Casas, J. R., 2010. Photo consistent surfaces from a sparse set of viewpoints. Proceedings of IEEE 17th International Conference on Image Processing, Sept.10, 4045-4048.
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R., 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’06), Jun.06, 1,519-528.
Shin, D., Tjahjadi, T., 2008. Local hull-based surface construction of volumetric data from silhouettes. IEEE Transactions on Image Processing, 17(8): 1251-1260.
Slabaugh, G., Culbertson, B., Malzbender, T., Schafer, R., 2001. A Survey of methods for volumetric scene reconstruction from photographs. Proceedings of the Joint IEEE TCVG and Eurographics Workshop, Volume Graphics 2001, Springer Computer Science, 81-100.
Song, P., Wu, X., Wang, M. Y., Wu, J., 2010. Expansion-based depth map estimation for multi-view stereo. Proceedings International Conference on Intelligent Robots and System, Oct.10, 3213-3218.
Taylor, C. J., 2003. Surface reconstruction from feature based stereo. Proceedings of IEEE International Conference on Computer Vision, Oct.03, 1, 184-190.
Teney, D., Piater, J., 2012. Sampling-based multiview reconstruction without correspondences for 3D edges. Proceedings of IEEE International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DimPVT), Oct.12, 160-167.
Tola, E., Lepetit, V., Fua, P., 2010. DAISY: An efficient dense descriptor allied to wide-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (5): 815-830.
Tran S., Davis L., 2006. 3D surface reconstruction using graph cuts with surface constraints. Proceedings of European Conference on Computer Vision: Part II (ECCV), 219-231.
Tung, T., Nobuhara, S., Matsuyama T., 2009. Complete multi-view reconstruction of dynamic scenes from probabilistic fusion of narrow and wide baseline stereo. IEEE 12th International Conference on Computer Vision, Sept.09, 1709-1716.
Vogiatzis, G., Hernandez, C., Torr, P., Cipolla, R., 2007. Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12): 2241-2246.
Vu, H., Keriven, R., Labatut, P., Pons, J. P., 2009. Towards high-resolution large scale multi-view stereo. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), Jun.09, 1430-1437.
Wang, J., Zhang, C., Zhu, W., Zhang, Z., Xiong, Z., Chou, P. A., 2012. 3D scene reconstruction by multiple structured-light based commodity depth cameras. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),Mar.12, 5429-5432.
Wu, J., Deng, L., Paul, A., 2015. 3D Terrain real time rendering method based on CUDA-OpenGL interoperability. IETE Technical Review, 32(6)471-478.
Zaharescu, A., Boyer, E., Horaud, R., 2011. Topology-adaptive mesh deformation for surface evolution, morphing and multi-view reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4):823-837.
Zao, C& Xiao W., 2005. A parallelizing and improving on voxel colouring technology for 3D reconstruction based on images. Proceedings of IEEE International Conference on Mechatronics and Automation, Aug.05, 4, 2185-2189.
Zeng, G., Paris, S., Quan, L., Lhuillier, M., 2004. Surface reconstruction by propagating 3D stereo data in multiple 2D images. Proceedings of 8th European Conference on Computer Vision, Springer Verlag: 163-174.
Zhou, Q. Y., Miller, S., Koltun, V., 2013. Elastic fragments for dense scene reconstruction. Proceedings of IEEE International Conference on Computer Vision, Dec. 13, 473-480.
