High Degree of Freedom Hand Pose Tracking Using Limited Strain Sensing and Optical Training

[+] Author and Article Information
Wentai Zhang

Carnegie Mellon University, Pittsburgh, PA, USA

Jonelle Z. Yu

Carnegie Mellon University, Pittsburgh, PA, USA

Fangcheng Zhu

Carnegie Mellon University, Pittsburgh, PA, USA

Zhu Yifang

Carnegie Mellon University, Pittsburgh, PA, USA

Zhangsihao Yang

Carnegie Mellon University, Pittsburgh, PA, USA

Nurcan Gecer Ulu

Carnegie Mellon University, Pittsburgh, PA, USA

Batuhan Arisoy

Siemens Corporate Technology, Princeton, NJ, USA

Levent Burak Kara

Carnegie Mellon University, Pittsburgh, PA, USA

1Corresponding author.

ASME doi:10.1115/1.4043757 History: Received September 15, 2018; Revised February 27, 2019


The ability to track human operators' hand usage when working in production plants and factories is critically important for developing realistic digital factory simulators as well as manufacturing process control. We propose an instrumented glove with only a few strain gauge sensors and a micro-controller that continuously tracks and records the hand configuration during actual use. At the heart of our approach is a trainable system that can predict the fourteen joint angles in the hand using only a small set of strain sensors. First, ten strain gauges are placed at the various joints in the hand to optimize the sensor layout using the English letters in the American Sign Language as a benchmark for assessment. Next, the best sensor configurations for three through ten strain gauges are computed using a support vector machine classifier. Following the layout optimization, our approach learns a mapping between the sensor readouts to the actual joint angles optically captured using a Leap Motion system. Five regression methods including linear, quadratic and neural regression are then used to train the mapping between the strain gauge data and the corresponding joint angles. The final proposed model involves four strain gauges mapped to the fourteen joint angles using a two-layer feed-forward neural network.

Siemens AG
Your Session has timed out. Please sign back in to continue.





Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In