0
Research Papers

Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications

[+] Author and Article Information
Charlie C. L. Wang

Associate Professor
Fellow ASME
e-mail: cwang@mae.cuhk.edu.hk

Department of Mechanical and
Automation Engineering,
The Chinese University of Hong Kong,
Hong Kong 1, China

The bone-length can be obtained during the initialization step.

Contributed by the Computers and Information Division of ASME for publication in the JOURNAL OF COMPUTERS AND INFORMATION DIVISION IN ENGINEERING. Manuscript received April 20, 2013; final manuscript received July 4, 2013; published online October 16, 2013. Editor: Bahram Ravani.

J. Comput. Inf. Sci. Eng 13(4), 041007 (Oct 16, 2013) (10 pages) Paper No: JCISE-13-1078; doi: 10.1115/1.4025404 History: Received April 20, 2013; Revised July 04, 2013

Recent development of per-frame motion extraction method can generate the skeleton of human motion in real-time with the help of RGB-D cameras such as Kinect. This leads to an economic device to provide human motion as input for real-time applications. As generated by a single-view image plus depth information, the extracted skeleton usually has problems of unwanted vibration, bone-length variation, self-occlusion, etc. This paper presents an approach to overcome these problems by synthesizing the skeletons generated by duplex Kinects, which capture the human motion in different views. The major technical difficulty of this synthesis comes from the inconsistency of two skeletons. Our algorithm is formulated under the constrained optimization framework by using the bone-lengths as hard constraints and the tradeoff between inconsistent joint positions as soft constraints. Schemes are developed to detect and re-position the problematic joints generated by per-frame method from duplex Kinects. As a result, we develop an easy, cheap and fast approach that can improve the skeleton of human motion at an average speed of 5 ms per frame.

FIGURES IN THIS ARTICLE
<>
Copyright © 2013 by ASME
Your Session has timed out. Please sign back in to continue.

References

Figures

Grahic Jump Location
Fig. 1

Problems of skeleton tracking by single Kinect—The viewing direction of Kinect sensor is specified by arrows on the photo. (Top-left) Self-occlusion: the left arm is hidden by the main body so that the positions of elbow and wrist are estimated to incorrect places. (Top-right) Bone-length variation: when viewing from a different direction, the length of forearm in an open-arm pose (right) changes significantly from its length that is captured when stands facing the Kinect camera (left). (Bottom) Artificial vibration: When viewing from a specific angle, the position of elbow joint has unwanted vibration even if a static pose is kept.

Grahic Jump Location
Fig. 2

Inconsistent skeletons extracted by two Kinects. (Top row) The 3D information captured by KA (first column) and KB (second column) are only partially overlapped even after carefully applying a registration procedure; as a result, the extracted skeletons, SA and SB, can be very close to each other but seldom be coincident. (Bottom row) In the view of KA, the elbow joint of SA is misclassified to the region of waist joint; although the position of this elbow joint on SB is correct, simply computing the average of SA and SB (i.e., 12(SA+SB)) will not give the result as good as S* generated by our approach.

Grahic Jump Location
Fig. 3

An illustration for explaining the observation that the distance between mistracked joints in one viewing plane will generally be much shorter than the distance in another viewing plane

Grahic Jump Location
Fig. 4

Our algorithm can correct the positions of problematic joints by resolving inconsistency under our constrained optimization framework while preserving the bone-lengths. The joints in the circles are problematic.

Grahic Jump Location
Fig. 5

The statistics of bone-length variation at different parts of skeletons in a Badminton playing motion, where dashed, dot-dashed and solid curve are representing the bone-lengths of SA, SB, and S*, respectively. The target bone-lengths, which are obtained from the initialization step, are displayed as a horizontal dot line in black.

Grahic Jump Location
Fig. 6

The motion of badminton playing: the enhanced skeletons, S*, generated by our algorithm are listed in the third and the fifth rows, the skeletons generated by Microsoft Kinect SDK, SA and SB, are listed in the second row and the fourth row, respectively. In our tests, we also use a video camera to capture the motion (shown in top row) so that the real motion can be illustrated more clearly. The orientations of two Kinect cameras, KA and KB, are also illustrated in the first row—see the arrow in the first column.

Grahic Jump Location
Fig. 7

The motion of basket-ball playing: enhanced skeletons in the motion are displayed in the third and fifth rows along the same viewing direction of Kinect cameras (i.e., KA and KB), which are shown in the second and the fourth rows. The problematic skeletons in the motion extracted by KA and KB independently are circled by dashed lines.

Grahic Jump Location
Fig. 8

An illustration for limitation: the wrist joints and the elbow joints are hardly separated from the main body in both views of Kinect cameras; therefore, the resultant joint position may not be fixed correctly

Grahic Jump Location
Fig. 9

The squatting pose does not included in the database of Kinect SDK, and the positions estimated by Kinect SDK is not good. As a result, our approach cannot generate a reasonable skeleton with those input.

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In