Research Papers

Theory and Algorithms for Weighted Total Least-Squares Fitting of Lines, Planes, and Parallel Planes to Support Tolerancing Standards

[+] Author and Article Information
Craig M. Shakarji

Physical Measurement Laboratory,
National Institute of Standards and Technology,
Gaithersburg, MD 20899
e-mail: craig.shakarji@nist.gov

Vijay Srinivasan

Fellow ASME
Engineering Laboratory,
National Institute of Standards and Technology,
Gaithersburg, MD 20899
e-mail: vijay.srinivasan@nist.gov

The case of fitting parallel lines in two-dimensions is also solved in this paper as a simple restriction of the parallel planes fit to 2D. The case of fitting parallel lines in three-dimensions is solved by going from the single line solution to multiple parallel lines with exactly analogous methods as going from single plane solution to multiple parallel planes as shown in this paper. The reason this case is not explicitly dealt with in this paper is due to the lack of immediate application to tolerancing standards.

Including the word “weighted” before total least-squares might be thought of as redundant, but total least-squares is often employed to weigh the coordinates of each point differently from each other. Here, the points themselves are weighted differently from each other, so we explicitly say weighted, even though the most general application of total least-squares allows for all such weights.

Certain commercial software packages are identified in this paper in order to specify the experimental procedures and code adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that the software tools identified are necessarily the best available for the purpose.

When considering convergence to the continuous case, we treat the surfaces as mathematical, ignoring the fact that, at very small scales, the molecular makeup of the material differs from our understanding of a continuous, mathematical surface.

Contributed by the Computers and Information Division of ASME for publication in the Journal of Computing and Information Science in Engineering. Manuscript received September 28, 2012; final manuscript received May 10, 2013; published online August 16, 2013. Assoc. Editor: Bahram Ravani.

J. Comput. Inf. Sci. Eng 13(3), 031008 (Aug 16, 2013) (11 pages) Paper No: JCISE-12-1174; doi: 10.1115/1.4024854 History: Received September 28, 2012; Revised May 10, 2013

We present the theory and algorithms for fitting a line, a plane, two parallel planes (corresponding to a slot or a slab), or many parallel planes in a total (orthogonal) least-squares sense to coordinate data that is weighted. Each of these problems is reduced to a simple 3 × 3 matrix eigenvalue/eigenvector problem or an equivalent singular value decomposition problem, which can be solved using reliable and readily available commercial software. These methods were numerically verified by comparing them with brute-force minimization searches. We demonstrate the need for such weighted total least-squares fitting in coordinate metrology to support new and emerging tolerancing standards, for instance, ISO 14405-1:2010. The widespread practice of unweighted fitting works well enough when point sampling is controlled and can be made uniform (e.g., using a discrete point contact coordinate measuring machine). However, we show by example that nonuniformly sampled points (arising from many new measurement technologies) coupled with unweighted least-squares fitting can lead to erroneous results. When needed, the algorithms presented also solve the unweighted cases simply by assigning the value one to each weight. We additionally prove convergence from the discrete to continuous cases of least-squares fitting as the point sampling becomes dense.

Copyright © 2013 by ASME
Your Session has timed out. Please sign back in to continue.


Shakarji, C. M., and Srinivasan, V., 2012, “Fitting Weighted Total Least-Squares Planes and Parallel Planes to Support Tolerancing Standards,” Proceedings, ASME 2012 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Chicago, Paper No. DETC2012-70398.
International Organization for Standardization (ISO), 2010, “Geometrical Product Specifications (GPS)—Dimensional Tolerancing—Part 1: Linear Sizes,” ISO 14405-1:2010.
International Organization for Standardization (ISO), “Geometrical Tolerancing—Tolerances of Form, Orientation, Location and Run-Out,” ISO 1101:(Expected Future Version).
International Organization for Standardization (ISO), “Geometrical Product Specifications (GPS)—Dimensional Tolerancing—Part 3: Angular Sizes,” ISO 14405-3:(Expected Future Version).
Nielson, H. S., 2012, “Recent Developments in ISO-GPS Standards and Strategic Plans for Future Work,” Proceedings of the 12th CIRP International Conference on Computer Aided tolerancing, University of Huddersfield, UK.
Parratt, S., and Voelcker, H., 1997, Do You Know How to Tolerance a Hinge?, Brown & Sharpe Manufacturing Company, pp. 42–43.
Shakarji, C. M., 1998, “Least-Squares Fitting Algorithms of the NIST Algorithm Testing System,” J. Res. Natl. Inst. Stand. Technol., 103, pp. 633–641. [CrossRef]
Forbes, A., 1989, “Least-Squares Best-Fit Geometric Elements,” NPL Report.
Nievergelt, Y., 1994, “Total Least Squares: State-of-the-Art Regression in Numerical Analysis,” SIAM Rev., 36, pp. 258–264. [CrossRef]
Nievergelt, Y., 2001, “Hyperspheres and Hyperplanes Fitted Seamlessly by Algebraic Constrained Total Least-Squares,” Linear Algebra Appl., 331, pp. 43–51. [CrossRef]
de Groen, P., 1996, “An Introduction to Total Least Squares,” Nieuw Arch. Wiskd, Vierde serie, deel, 14, pp. 237–253.
Golub, G. H., and Van Loan, C. F., 1980, “An Analysis of the Total Least Squares Problem,” SIAM (Soc. Ind. Appl. Math.) J. Numer. Anal., 17, pp. 883–893. [CrossRef]
VanHuffel, S., and Vandervalle, J., 1991, The Total Least Squares Problem: Computational Aspects and Analysis, SIAM, Philadelphia, PA.
Krystek, M., and Anton, M., 2007, “A Weighted Total Least-Squares Algorithm for Fitting a Straight Line,” Meas. Sci. Technol., 18, pp. 3438–3442. [CrossRef]
Srinivasan, V., Shakarji, C. M., and Morse, E. P., 2012, “On the Enduring Appeal of Least-Squares Fitting in Computational Coordinate Metrology,” ASME J. Comput. Inf. Sci. Eng., 12(1), p. 011008. [CrossRef]
Shakarji, C. M., 2011, “Coordinate Measuring System Algorithms and Filters,” Coordinate Measuring Machines and Systems, R. J.Hocken, and P. H.Pereira, eds., CRC Press, Boca Raton, FL, pp. 153–182.
Krommer, A. R., and Ueberhuber, C. W., 1998, Computational integration, Society for Industrial and Applied Mathematics, Philadelphia.
Stewart, G. W., 1973, Introduction to Matrix Computations, Academic, New York.
Meyer, C. D., and Stewart, G. W., 1988, “Derivatives and Perturbations of Eigenvectors,” SIAM (Soc. Ind. Appl. Math.) J. Numer. Anal., 25(3), pp. 679–691. [CrossRef]
Watkins, D. S., 1991, Fundamentals of Matrix Computations, John Wiley and Sons, New York.


Grahic Jump Location
Fig. 1

Point sampling density adversely affecting the least-squares fit

Grahic Jump Location
Fig. 2

Fitting a plane to a surface patch

Grahic Jump Location
Fig. 3

Fitting two parallel planes

Grahic Jump Location
Fig. 4

Fitting a line to a curve

Grahic Jump Location
Fig. 5

Histogram of angular deviations in radians (log base 10 scaling)

Grahic Jump Location
Fig. 6

Histogram of relative distance deviations (mm, log base 10 scaling)



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In