Research Papers

Real-Time Trajectory Generation for Haptic Feedback Manipulators in Virtual Cockpit Systems

[+] Author and Article Information
Shiyu Zhang

State Key Laboratory of Virtual Reality
Technology and Systems,
Beihang University,
XueYuan Road No.37, HaiDian District,
Beijing 100191, China
e-mail: zhangshiyu@buaa.edu.cn

Shuling Dai

State Key Laboratory of
Virtual Reality Technology and Systems,
Beihang University,
XueYuan Road No.37, HaiDian District,
Beijing 100191, China
e-mail: sldai@buaa.edu.cn

1Corresponding author.

Contributed by the Computers and Information Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received December 7, 2017; final manuscript received August 2, 2018; published online September 7, 2018. Assoc. Editor: Caterina Rizzi.

J. Comput. Inf. Sci. Eng 18(4), 041015 (Sep 07, 2018) (11 pages) Paper No: JCISE-17-1296; doi: 10.1115/1.4041166 History: Received December 07, 2017; Revised August 02, 2018

To obtain real-time interactions in the virtual cockpit system (VCS), a real-time trajectory generation method based on dynamical nonlinear optimization and regression prediction for the haptic feedback manipulator (HFM) is presented in this paper. First, a haptic feedback system based on servoserial manipulator is constructed. Then, the trajectory planning problem for the HFM is formulated as a nonlinear optimization problem to balance the motion time and power consumption and ensure the safety of physical human–robot interactions (pHRI). Multiple optimization problems are solved to generate the optimal database off-line. Finally, the classified multivariate (CM) regression method is presented to learn the database and generate optimal trajectories with arbitrary initial and objective positions on-line. Results show that trajectories with rapidity, safety, and lower power consumption can be generated in real-time by this method, which lay a basis of haptic interactions in the VCS.

Copyright © 2018 by ASME
Your Session has timed out. Please sign back in to continue.


Dai, S. , Lei, X. , and Mei, J. , 2002, “Virtual Cockpit System,” J. Syst. Simul., 14(4), pp. 488–492.
Wang, D. , Jiao, J. , Zhang, Y. , and Zhao, X. , 2016, “Computer Haptics: Haptic Modeling and Rendering in Virtual Reality Environments,” J. Comput.-Aided Des. Comput. Graphics, 28(6), pp. 881–895.
Tang, Y. , Gu, H. , and Li, P. , 2012, “Augmented Semi-Virtual Reality Cockpit Technology,” J. Traffic Transp. Eng., 12(2), pp. 62–69. http://transport.chd.edu.cn/oa/DArticle.aspx?type=view&id=201202009
CGSD, 1998, “Force/Tactile Feedback System for Virtual Reality Environments,” Mountain View, CA, Report No. ADA342328. http://www.dtic.mil/docs/citations/ADA342328
Stonum, R. , 1998, “A Survey of Immersive Technology for Maintenance Evaluations,” Battelle Memorial Institute, Dayton, OH, Report No. AFRL-HE-WP-TR-1998-0038.
Kim, K. W. , Kim, H. S. , Choi, Y. K. , and Park, J. H. , 1997, “Optimization of Cubic Polynomial Joint Trajectories and Sliding Mode Controllers for Robots Using Evolution Strategy,” 23rd International Conference on Industrial Electronics, Control, and Instrumentation (IECON), New Orleans, LA, Nov. 14, pp. 1444–1447.
Bazaz, S. A. , and Tondu, B. , 1997, “Online Computing of a Robotic Manipulator Joint Trajectory With Velocity and Acceleration Constraints,” IEEE International Symposium on Assembly and Task Planning (ISATP'97), Marina del Rey, CA, Aug. 7–9, pp. 1–6.
Lampariello, R. , and Hirzinger, G. , 2013, “Generating Feasible Trajectories for Autonomous On-Orbit Grasping of Spinning Debris in a Useful Time,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan, Nov. 3–7, pp. 5652–5659.
Liu, X. , Meng, Z. , Ni, J. , and Zhu, Z. , 2015, “Trajectory Planning Algorithm for Hydraulic Servo Manipulator of Three Freedom,” J. Zhejiang Univ., 49(9), pp. 1776–1782.
Miossec, S. , Yokoi, K. , and Kheddar, A. , 2006, “Development of a Software for Motion Optimization of Robots-Application to the Kick Motion of the Hrp-2 Robot,” IEEE International Conference on Robotics and Biomimetics (ROBIO), Kunming, China, Dec. 17–20, pp. 299–304.
Bäuml, B. , Wimböck, T. , and Hirzinger, G. , 2010, “Kinematically Optimal Catching a Flying Ball With a Hand-Arm-System,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan, Oct. 18–22, pp. 2592–2599.
Schulman, J. , Duan, Y. , Ho, J. , Lee, A. , Awwal, I. , Bradlow, H. , Pan, J. , Patil, S. , Goldberg, K. , and Abbeel, P. , 2014, “Motion Planning With Sequential Convex Optimization and Convex Collision Checking,” Int. J. Rob. Res., 33(9), pp. 1251–1270.
von Stryk, O. , and Schlemmer, M. , 1994, “Optimal Control of the Industrial Robot Manutec r3,” Computational Optimal Control. ISNM International Series of Numerical Mathematics, Vol. 115, R. Bulirsch and D. Kraft D, eds., Birkhäuser Basel, Switzerland.
Chettibi, T. , Lehtihet, H. , Haddad, M. , and Hanchi, S. , 2004, “Minimum Cost Trajectory Planning for Industrial Robots,” Eur. J. Mech.-A, 23(4), pp. 703–715. [CrossRef]
Pan, J. , Chen, Z. , and Abbeel, P. , 2014, “Predicting Initialization Effectiveness for Trajectory Optimization,” IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May. 31–June 7, pp. 5183–5190.
Jetchev, N. , and Toussaint, M. , 2013, “Fast Motion Planning From Experience: Trajectory Prediction for Speeding Up Movement Generation,” Auton. Robots, 34(1–2), pp. 111–127. [CrossRef]
Hauser, K. , 2017, “Learning the Problem-Optimum Map: Analysis and Application to Global Optimization in Robotics,” IEEE Trans. Rob., 33(1), pp. 141–152. [CrossRef]
Lampariello, R. , Nguyen-Tuong, D. , Castellini, C. , Hirzinger, G. , and Peters, J. , 2011, “Trajectory Planning for Optimal Robot Catching in Real-Time,” IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9–13, pp. 3719–3726.
Werner, A. , Trautmann, D. , Lee, D. , and Lampariello, R. , 2015, “Generalization of Optimal Motion Trajectories for Bipedal Walking,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, Sept. 28–Oct. 2, pp. 1571–1577.
Denavit, J. , and Hartenberg, R. S. , 1955, “A Kinematic Notation for Lower-Pair Mechanisms,” ASME J. Appl. Mech., 22, pp. 215–221.
De Santis, A. , Siciliano, B. , De Luca, A. , and Bicchi, A. , 2008, “An Atlas of Physical Human–Robot Interaction,” Mechanism Mach. Theory, 43(3), pp. 253–270. [CrossRef]
Corrales, J. , Candelas, F. , and Torres, F. , 2011, “Safe Human–Robot Interaction Based on Dynamic Sphere-Swept Line Bounding Volumes,” Rob. Comput.-Integr. Manuf., 27(1), pp. 177–185. [CrossRef]
Pereira, A. , and Althoff, M. , 2015, “Safety Control of Robots Under Computed Torque Control Using Reachable Sets,” IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, May 26–30, pp. 331–338.
Rastegar, J. , and Perel, D. , 1990, “Generation of Manipulator Workspace Boundary Geometry Using the Monte Carlo Method and Interactive Computer Graphics,” ASME J. Mech. Des., 112(3), pp. 452–454. [CrossRef]
ISO, 2016, “Robots and Robotic Devices-Collaborative Robots,” International Organization for Standardization, Geneva, Switzerland, Standard No. ISO/TS 15066. https://www.iso.org/standard/62996.html
Hargraves, C. R. , and Paris, S. W. , 1987, “Direct Trajectory Optimization Using Nonlinear Programming and Collocation,” J. Guid., Control, Dyn., 10(4), pp. 338–342. [CrossRef]


Grahic Jump Location
Fig. 4

The structure of the HFM (1—waist, 2—shoulder, 3—elbow, 4—pitching wrist joint, 5—yawing wrist joint, and 6—rotating wrist joint)

Grahic Jump Location
Fig. 5

Denavit–Hartenberg coordinate system of the HFM

Grahic Jump Location
Fig. 3

Interaction of the control panel and user's hand: (a) haptic feedback in real world and (b) visual feedback in virtual environment

Grahic Jump Location
Fig. 2

Work flow of haptic feedback system

Grahic Jump Location
Fig. 1

Components of haptic feedback system based on servo serial manipulator: (a) schematic diagram and (b) prototype

Grahic Jump Location
Fig. 6

End-effector trajectory of HFM (dashed: trajectories generated in every period and solid: the actual trajectory of the whole process)

Grahic Jump Location
Fig. 8

Motion curve (solid: nonlinear optimization, dashed: CM-6-NN, and dotted–dashed: CM-LWR; red: joint 1, green: joint 2, and blue: joint 3): (a) velocity, (b) accelerate, (c) torque, and (d) linear velocity

Grahic Jump Location
Fig. 9

The reachable workspace and the control panel area of the HFM: (a) spatial point cloud, (b) projection on plane xy, (c) projection on plane xz, and (d) projection on plane yz

Grahic Jump Location
Fig. 10

Database of the initial and objective points

Grahic Jump Location
Fig. 11

eF with different database sizes (k-NN)

Grahic Jump Location
Fig. 12

Variation trend of the weighting coefficient: (a) k-NN method and LWR method with larger λ and (b) LWR method with smaller λ

Grahic Jump Location
Fig. 13

eF with different database sizes (LWR)

Grahic Jump Location
Fig. 7

Real-time trajectory generation base on regression and prediction

Grahic Jump Location
Fig. 14

TR of k-NN with different ks



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In