Research Papers

Poisson Mesh Reconstruction for Accurate Object Tracking With Low-Fidelity Point Clouds

[+] Author and Article Information
Timothy Garrett

Virtual Reality Applications Center,
Iowa State University,
Ames, IA 50011
e-mail: garrettt@iastate.edu

Saverio Debernardis

Department of Mechanics,
Mathematics and Management,
Politecnico di Bari,
Bari 70126, Italy
e-mail: saverio.debernardis@poliba.it

James Oliver

Fellow ASME
Virtual Reality Applications Center,
Iowa State University,
Ames, IA 50011
e-mail: oliver@iastate.edu

Rafael Radkowski

Assistant Professor
Virtual Reality Applications Center,
Iowa State University,
Ames, IA 50011
e-mail: rafael@iastate.edu

Contributed by the Design Engineering Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received October 6, 2015; final manuscript received July 25, 2016; published online November 7, 2016. Editor: Bahram Ravani.

J. Comput. Inf. Sci. Eng 17(1), 011003 (Nov 07, 2016) (9 pages) Paper No: JCISE-15-1319; doi: 10.1115/1.4034324 History: Received October 06, 2015; Revised July 25, 2016

Tracking refers to a set of techniques that allows one to calculate the position and orientation of an object with respect to a global reference coordinate system in real time. A common method for tracking with point clouds is the iterative closest point (ICP) algorithm, which relies on the continuous matching of sequential sampled point clouds with a reference point cloud. Modern commodity range cameras provide point cloud data that can be used for that purpose. However, this point cloud data is generally considered as low-fidelity and insufficient for accurate object tracking. Mesh reconstruction algorithms can improve the fidelity of the point cloud by reconstructing the overall shape of the object. This paper explores the potential for point cloud fidelity improvement via the Poisson mesh reconstruction (PMR) algorithm and compares the accuracy with a common ICP-based tracking technique and a local mesh reconstruction operator. The results of an offline simulation are promising.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.


Besl, P. , and McKay, N. , 1992, “ A Method for Registration of 3-D Shapes,” IEEE Trans. Pattern Anal. Mach. Intell., 14(2), pp. 239–256. [CrossRef]
Zhang, Z. , 1992, On Local Matching of Free-Form Curves, Springer, London, pp. 347–356.
Segal, A. , Haehnel, D. , and Thrun, S. , 2010, “ Generalized-ICP,” Robotics: Science and Systems, Vol. 2, J. Trinkle, ed., The MIT Press, Cambridge, MA, pp. 161–168.
Fitzgibbon, A. W. , 2003, “ Robust Registration of 2D and 3D Point Sets,” Image Vision Comput., 21(13), pp. 1145–1153. [CrossRef]
Prasad, T. , Hartmann, K. , Weihs, W. , Ghobadi, S. E. , and Sluiter, A. , 2006, “ First Steps in Enhancing 3D Vision Technique Using 2D/3D Sensors,” Computer Vision Winter Workshop, pp. 82–86.
Yang, Q. , Yang, R. , Davis, J. , and Nister, D. , 2007, “ Spatial-Depth Super Resolution for Range Images,” IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1–8.
Zhu, J. , Wang, L. , Yang, R. , Davis, J. E. , and Pan, Z. , 2011, “ Reliability Fusion of Time-of-Flight Depth and Stereo Geometry for High Quality Depth Maps,” IEEE Trans. Pattern Anal. Mach. Intell., 33(7), pp. 1400–1414. [CrossRef] [PubMed]
Kowalczuk, J. , Psota, E. , and Perez, L. , 2013, “ Real-Time Stereo Matching on CUDA Using an Iterative Refinement Method for Adaptive Support-Weight Correspondences,” IEEE Trans. Circuits Syst. Video Technol., 23(1), pp. 94–104. [CrossRef]
Jung, S.-W. , and Choi, O. , 2014, “ Learning-Based Filter Selection Scheme for Depth Image Super Resolution,” IEEE Trans. Circuits Syst. Video Technol., 24(10), pp. 1641–1650. [CrossRef]
Nguyen, C. , Izadi, S. , and Lovell, D. , 2012, “ Modeling Kinect Sensor Noise for Improved 3D Reconstruction and Tracking,” Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 524–530.
Zhang, G. , Lei, M. , and Liu, X. , 2009, “ Novel Template Matching Method With Sub-Pixel Accuracy Based on Correlation and Fourier-Mellin Transform,” Opt. Eng., 48(5), p. 057001. [CrossRef]
Awange, J. , Paláncz, B. , and Lewis, R. , 2014, “ Maximizing Likelihood Function for Parameter Estimation in Point Clouds Via Groebner Basis,” Mathematical Software—ICMS, H. Hong and C. Yap, eds., Springer, Berlin, pp. 359–366.
Stoyanov, T. , and Lilienthal, A. , 2009, “ Maximum Likelihood Point Cloud Acquisition From a Mobile Platform,” International Conference on Advanced Robotics, ICAR, pp. 1–6.
Grisetti, G. , Grzonka, S. , Stachniss, C. , Pfaff, P. , and Burgard, W. , 2007, “ Efficient Estimation of Accurate Maximum Likelihood Maps in 3D,” IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, pp. 3472–3478.
Huang, A. S. , Bachrach, A. , Henry, P. , Krainin, M. , Maturana, D. , Fox, D. , and Roy, N. , 2011, “ Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera,” International Symposium on Robotics Research (ISRR), Aug. 28–Sept. 1, Flagstaff, AZ. http://www.isrr-2011.org/ISRR-2011//Program_files/Papers/Huang-ISRR-2011.pdf
Pereira, N. , and Sitek, A. , 2011, “ An SVD Based Analysis of the Noise Properties of a Point Cloud Mesh Reconstruction Method,” IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), pp. 4135–4142.
Moghari, M. H. , and Abolmaesumi, P. , 2008, “ Maximum Likelihood Estimation of the Distribution of Target Registration Error,” Proc. SPIE, 6918, p. 69180I.
Gasparovic, M. , and Malaric, I. , 2012, “ Increase of Readability and Accuracy of 3D Models Using Fusion of Close Range Photogrammetry and Laser Scanning,” Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., 39(B5), pp. 93–98. [CrossRef]
Tascon Vidarte, J. , and Loaiza Correa, H. , 2014, “ Consistent 3D Models From Unorganized RGB-D Images,” IEEE 27th Canadian Conference on Electrical and Computer Engineering (CCECE), pp. 1–6.
Hara, Y. , Bando, S. , Tsuboucffl, T. , Oshima, A. , Kitahara, I. , and Kameda, Y. , 2013, “ 6DOF Iterative Closest Point Matching Considering a Priori With Maximum a Posteriori Estimation,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4172–4179.
Kim, Y. M. , Mitra, N. J. , Yan, D.-M. , and Guibas, L. , 2012, “ Acquiring 3D Indoor Environments With Variability and Repetition,” ACM Trans. Graphics, 31(6), p. 138. [CrossRef]
Wells, L. J. , Shafae, M. S. , and Camelio, J. A. , 2013, “ Automated Part Inspection Using 3D Point Clouds,” ASME Paper No. MSEC2013-1212.
Tucker, T. M. , and Kurfess, T. R. , 2006, “ Point Cloud to CAD Model Registration Methods in Manufacturing Inspection,” ASME J. Comput. Inf. Sci. Eng., 6(4), pp. 418–421. [CrossRef]
Grimm, C. M. , Crisco, J. J. , and Laidlaw, D. H. , 2001, “ Fitting Manifold Surfaces to Three-Dimensional Point Clouds,” ASME J. Biomech. Eng., 124(1), pp. 136–140. [CrossRef]
Shi, Q. , Xi, N. , and Zhang, C. , 2010, “ Develop a Robot-Aided Area Sensing System for 3D Shape Inspection,” ASME J. Manuf. Sci. Eng., 132(1), p. 014502. [CrossRef]
Miller, A. , White, B. , Charbonneau, E. , Kanzler, Z. , and LaViola, J. J. , 2012, “ Interactive 3D Model Acquisition and Tracking of Building Block Structures,” IEEE Trans. Visualization Comput. Graphics, 18(4), pp. 651–659. [CrossRef]
Gupta, A. , Fox, D. , Curless, B. , and Cohen, M. , 2012, “ DuploTrack: A Real-Time System for Authoring and Guiding DUPLO Block Assembly,” 25th Annual ACM Symposium on User Interface Software and Technology, pp. 389–402.
Khuong, B. M. , Kiyokawa, K. , Mashita, T. , Miller, A. , Takemura, H. , and La Viola, J. J. , 2014, “ The Effectiveness of an AR-Based Context-Aware Assembly Support System in Object Assembly,” IEEE Virtual Reality (VR), pp. 57–62.
Salas-Moreno, R. F. , Newcombe, R. A. , Strasdat, H. , Kelly, P. H. J. , and Davison, A. J. , 2013, “ Slam++: Simultaneous Localisation and Mapping at the Level of Objects,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1352–1359.
Newcombe, R. A. , Izadi, S. , Hilliges, O. , Molyneaux, D. , Kim, D. , Davison, A. J. , Kohli, P. , Shotton, J. , Hodges, S. , and Fitzgibbon, A. , 2011, “ Kinectfusion: Real-Time Dense Surface Mapping and Tracking,” 10th IEEE International Symposium on Mixed and Augmented Reality, ISMAR’11, IEEE Computer Society, pp. 127–136.
Rusinkiewicz, S. , and Levoy, M. , 2001, “ Efficient Variants of the ICP Algorithm,” IEEE International Workshop on 3D Digital Imaging and Modeling (3DIM), pp. 145–152.
Garrett, T. , Debernardis, S. , Radkowski, R. , Chang, C. K. , Fiorentino, M. , Uva, A. E. , and Oliver, J. , 2014, “ Rigid Object Tracking Algorithms for Low-Cost AR Devices,” ASME Paper No. DETC2014-35304.
Radkowski, R. , 2015, “ Object Tracking With a Range Camera for Augmented Reality Assembly Assistance,” ASME J. Comput. Inf. Sci. Eng., 16(1), pp. 1–8.
Greenspan, M. , and Yurick, M. , 2003, “ Approximate K-D Tree Search for Efficient ICP,” Fourth International Conference on 3-D Digital Imaging and Modeling, 3DIM, pp. 442–448.
Arun, K. , Huang, T. S. , and Blostein, S. D. , 1987, “ Least-Squares Fitting of Two 3-D Point Sets,” IEEE Trans. Pattern Anal. Mach. Intell., PAMI-9(5), pp. 698–700. [CrossRef]
Kazhdan, M. , Bolitho, M. , and Hoppe, H. , 2006, “ Poisson Surface Reconstruction,” Fourth Eurographics Symposium on Geometry Processing, Eurographics Association, pp. 61–70.
Holz, D. , and Behnke, S. , 2012, “ Fast Range Image Segmentation and Smoothing Using Approximate Surface Reconstruction and Region Growing,” 12th International Conference on Intelligent Autonomous Systems (IAS), Jeju Island, Korea, June 26–29, pp. 61–73.
Radkowski, R. , 2015, “ A Point Cloud-Based Method for Object Alignment Verification for Augmented Reality Applications,” ASME Paper No. DETC2015-47842.
Chen, Y. , and Medioni, G. , 1991, “ Object Modeling by Registration of Multiple Range Images,” IEEE International Conference on Robotics and Automation, Vol. 3, pp. 2724–2729.
Ridene, T. , and Goulette, F. , 2009, “ Registration of Fixed-and-Mobile-Based Terrestrial Laser Data Sets With DSM,” IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA), pp. 375–380.


Grahic Jump Location
Fig. 1

Iterations to minimize the distance between a reference object P and an environment point cloud X

Grahic Jump Location
Fig. 2

The matching process

Grahic Jump Location
Fig. 3

Fidelity of the reproduced mesh in comparison to the regular point cloud

Grahic Jump Location
Fig. 4

The main desktop of the simulation tool which has been used for testing

Grahic Jump Location
Fig. 5

Test objects for the analysis: (a) a gear switch, (b) an axial piston motor, (c) valves, (d) the Stanford bunny model, (e) dragon model, and (f) Happy Buddha

Grahic Jump Location
Fig. 6

Matching results for the objects in Figs. 5(a)5(c)

Grahic Jump Location
Fig. 7

Matching results for the objects in Figs. 5(d)5(f)

Grahic Jump Location
Fig. 8

(a) The reference model for the gear switch and (b) the reference model for the piston motor

Grahic Jump Location
Fig. 9

Mean squared error for ICP with and without mesh generation

Grahic Jump Location
Fig. 10

Number of Iterations for ICP with and without mesh generation



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In