Research Papers

Motion Imitation Based on Sparsely Sampled Correspondence

[+] Author and Article Information
Shuo Jin

Department of Mechanical and
Automation Engineering,
The Chinese University of Hong Kong,
Hong Kong 999077, China
e-mail: jerry.shuojin@gmail.com

Chengkai Dai

Department of Mechanical and
Automation Engineering,
The Chinese University of Hong Kong,
Hong Kong 999077, China
e-mail: ckdai@mae.cuhk.edu.hk

Yang Liu

Microsoft Research Asia,
Beijing 100080, China
e-mail: yangliu@microsoft.com

Charlie C. L. Wang

Department of Design Engineering and
TU Delft Robotics Institute,
Delft University of Technology,
Delft 2628, The Netherlands
e-mail: c.c.wang@tudelft.nl

1Corresponding author.

Contributed by the Computers and Information Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received August 16, 2016; final manuscript received May 2, 2017; published online June 15, 2017. Editor: Bahram Ravani.

J. Comput. Inf. Sci. Eng 17(4), 041009 (Jun 15, 2017) (7 pages) Paper No: JCISE-16-2045; doi: 10.1115/1.4036923 History: Received August 16, 2016; Revised May 02, 2017

Existing techniques for motion imitation often suffer a certain level of latency due to their computational overhead or a large set of correspondence samples to search. To achieve real-time imitation with small latency, we present a framework in this paper to reconstruct motion on humanoids based on sparsely sampled correspondence. The imitation problem is formulated as finding the projection of a point from the configuration space of a human's poses into the configuration space of a humanoid. An optimal projection is defined as the one that minimizes a back-projected deviation among a group of candidates, which can be determined in a very efficient way. Benefited from this formulation, effective projections can be obtained by using sparsely sampled correspondence, whose generation scheme is also introduced in this paper. Our method is evaluated by applying the human's motion captured by an RGB-depth (RGB-D) sensor to a humanoid in real time. Continuous motion can be realized and used in the example application of teleoperation.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.



Grahic Jump Location
Fig. 1

An illustration of our framework for motion imitation using configuration projection

Grahic Jump Location
Fig. 2

An example of imitation realized by our framework working with the Nao humanoid

Grahic Jump Location
Fig. 3

An illustration of finding an optimal point that minimizes a back-projected deviation (with L=M=4)

Grahic Jump Location
Fig. 4

Feature vectors of human and humanoid: (a) the human skeleton from a Kinect sensor, (b) the corresponding pose descriptor of a human body consists of 19 unit vectors, and (c) the pose descriptor for an Nao humanoid formed by all DOFs on its joints

Grahic Jump Location
Fig. 5

Basic poses serve as benchmarks for similarity evaluation

Grahic Jump Location
Fig. 6

Eight basic poses are reconstructed by our method (left of each pair) and compared with the ground truth (right of each pair). The similarity metrics, Mmax and Mavg, of each pair are also reported. The evaluation is taken on a projection defined by using 1644 landmark pairs.

Grahic Jump Location
Fig. 7

Statistics in eight motions for the change of two metrics in degree: Mmax (the upper curves in each sub-figures) and Mavg (the lower curves in sub-figures). The evaluation is also taken on a projection with 1644 landmark pairs.

Grahic Jump Location
Fig. 8

To reconstruct motion using landmark sets with different number of corresponding samples, statistics of Mmax and Mavg in degree indicate that more landmark pairs lead to better results

Grahic Jump Location
Fig. 9

Application of teleoperation using Nao humanoid: picking up a ring and putting it into a box (left), and lifting up a poster by two hands (right)

Grahic Jump Location
Fig. 10

Tests on the lab-made Poppy humanoid: full body motion (left) and simultaneous imitation (right) in the heterogeneous environment with an Nao and a Poppy




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In