Research Papers

A Cloud Service Framework for Virtual Try-On of Footwear in Augmented Reality

[+] Author and Article Information
Chih-Hsing Chu

Department of Industrial Engineering and
Engineering Management,
National Tsing Hua University,
Hsinchu 30013, Taiwan
e-mail: chchu@ie.nthu.edu.tw

Chih-Hung Cheng, Han-Sheng Wu

Department of Industrial Engineering and
Engineering Management,
National Tsing Hua University,
Hsinchu 30013, Taiwan

Chia-Chen Kuo

National Center for
High-Performance Computing,
Hsinchu 30076, Taiwan

Contributed by the Computers and Information Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received March 20, 2018; final manuscript received November 26, 2018; published online February 4, 2019. Assoc. Editor: Monica Bordegoni.

J. Comput. Inf. Sci. Eng 19(2), 021002 (Feb 04, 2019) (7 pages) Paper No: JCISE-18-1067; doi: 10.1115/1.4042102 History: Received March 20, 2018; Revised November 26, 2018

This paper presents an experimental cloud service framework for design evaluation of personalized footwear in augmented reality (AR) via networks. The service allows users to ubiquitously perceive themselves trying on three-dimensional (3D) shoe models in a video stream. They upload a clip of feet motion recorded by a commercial depth camera to the cloud. A new clip is generated to display the try-on process and made available to specified receivers via video streaming on a mobile device. The framework design emphasizes making most use of open-source software and off-the-shelf technologies commercially available. A prototyping cloud system implementing the framework demonstrates the practical value of virtual footwear try-on as AR as a service (ARaaS). This experimental study realizes the idea of human-centric design evaluation in modern e-commerce. The cloud framework may provide a feasible example to improve the usability for real-time applications of AR.

Copyright © 2019 by ASME
Your Session has timed out. Please sign back in to continue.


Tseng, M. M. , and Piller, F. T. , 2003, The Customer Centric Enterprise, Advances in Mass Customization and Personalization, Springer-Verlag, Berlin.
Lo, C. H. , and Chu, C. H. , 2014, “ An Investigation of the Social-Affective Effects Invoked by Appearance-Related Products,” Hum Factors Ergonom. Manuf. Serv. Ind., 24(1), pp. 71–85. [CrossRef]
Huang, S. H. , Yang, Y. I. , and Chu, C. H. , 2012, “ Human-Centric Design Personalization of 3D Glasses Frame in Markerless Augmented Reality,” Adv. Eng. Inf., 16, pp. 35–45. https://www.sciencedirect.com/science/article/pii/S1474034611000565
Wang, C. C. L. , Hui, K. C. , and Tong, K. M. , 2007, “ Volume Parameterization for Design Automation of Customized Free-Form Products,” IEEE Trans. Autom. Sci. Eng., 4(1), pp. 11–21. [CrossRef]
Chu, C. H. , Wang, I. J. , Wang, J. B. , and Luh, Y. P. , 2017, “ 3D Parametric Human Face Modeling for Personalized Product Design: Eyeglasses Frame Design Case,” Adv. Eng. Inf., 32, pp. 202–223. [CrossRef]
Huang, S. H. , Yang, C. K. , Tseng, C. Y. , and Chu, C. H. , 2015, “ Design Customization of Respiratory Mask Based on 3D Face Anthropometric Data,” Int. J. Precis. Eng. Manuf., 16(3), pp. 487–494. https://link.springer.com/article/10.1007/s12541-015-0066-5
Cheng, K. , Nakazawa, M. , and Masuko, S. , 2017, “ MR-Shoppingu: Physical Interaction With Augmented Retail Products Using Continuous Context Awareness,” International Conference on Entertainment Computing, Tsukuba City, Japan, Sept. 18–21, pp. 452–455.
Speicher, M. , Cucerca, S. , and Krüger, A. , 2017, “ VRShop: A Mobile Interactive Virtual Reality Shopping Environment Combining the Benefits of On-and Offline Shopping,” Proc. ACM Interact., Mobile, Wearable Ubiquitous Technol., 1(3).
Yang, Y. I. , Yang, C. K. , and Chu, C. H. , 2014, “ A Virtual Try-On System in Augmented Reality Using RGB-D Cameras for Footwear Personalization,” J. Manuf. Syst., 33(4), pp. 690–698. [CrossRef]
Hauswiesner, S. , Straka, M. , and Reitmayr, G. , 2013, “ Virtual Try-On Through Image-Based Rendering,” IEEE Trans. Visualization Comput. Graph., 19(9), pp. 1552–1565. [CrossRef]
Antonio, J. M. , Jose Luis, S. R. , and Faustino, S. P. , 2013, “ Augmented and Virtual Reality Techniques for Footwear,” Comput. Ind., 64(9), pp. 1371–1382. https://www.sciencedirect.com/science/article/pii/S016636151300122X
Yuan, X. , Tang, D. , Liu, Y. , Ling, Q. , and Fang, L. , 2017, “ Magic Glasses: From 2D to 3D,” IEEE Trans. Circuits Syst. Video Technol., 27(4), pp. 843–854. [CrossRef]
Sesana, M. , and Canepa, A. , 2017, “ Virtual Showroom for New Textile Design and Sales Decrease Textile Collection Costs by Virtual Reality Samples,” International Conference on Engineering, Technology and Innovation (ICE/ITMC), Madeira Island, Portugal, June 27–29, pp. 1534–1537.
Vitali, A. , and Rizzi, C. , 2017, “ A Virtual Environment to Emulate Tailor's Work,” Comput.-Aided Des. Appl., 14(5), pp. 671–679. [CrossRef]
Eisert, P. , Fechteler, P. , and Rurainsky, J. , 2008, “ 3-D Tracking of Shoes for Virtual Mirror Applications,” IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, June 23–28.
Greci, L. , Sacco, M. , Cau, N. , and Buonanno, F. , 2012, “ FootGlove: A Haptic Device Supporting the Customer in the Choice of the Best Fitting Shoes,” Haptics: Perception, Devices, Mobility, and Communication, Springer, Berlin, pp. 148–159.
Shen, J. , Su, P. C. , Cheung, S. C. S. , and Zhao, J. , 2013, “ Virtual Mirror Rendering With Stationary RGB-D Cameras and Stored 3D Background,” IEEE Trans. Image Process., 22(9), pp. 3433–3448. [CrossRef] [PubMed]
Rızvanoğlu, K. , and Çetin, G. , 2013, Research and Design Innovations for Mobile User Experience, IGI Global, Hershey, PA.
Gold, S. , Rangarajan, A. , Lu, C. P. , Suguna, P. , and Mjolsness, E. , 1998, “ New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence,” Pattern Recognit., 38(8), pp. 1019–1031. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=
Jolliffe, I. T. , 2002, Principle Component Analysis, 2nd ed., Springer-Verlag, New York.
Besl, P. , and McKay, N. , 1992, “ A Method for Registration of 3-D Shapes,” IEEE Trans. Pattern Anal. Mach. Intell., 14(2), pp. 239–256. [CrossRef]
Yuan, M. , Khan, I. R. , Farbiz, F. , Yao, S. , Niswar, A. , and Foo, M. H. , 2013, “ A Mixed Reality Virtual Clothes Try-On System,” IEEE Trans. Multimedia, 15(8), pp. 1958–1968. [CrossRef]
Horaud, R. , Hansard, M. , Evangelidis, G. , and Ménier, C. , 2016, “ An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologies,” Mach. Vision Appl., 27(7), pp. 1005–1020. [CrossRef]


Grahic Jump Location
Fig. 1

A simplified framework presenting the cloud-based try-on service

Grahic Jump Location
Fig. 2

Predefine the relationship between the template foot and shoe models

Grahic Jump Location
Fig. 3

Problem description of virtual shoe try-on

Grahic Jump Location
Fig. 4

Skeleton of a human user extracted from Kinect

Grahic Jump Location
Fig. 5

Estimating the ankle axis of a high precision

Grahic Jump Location
Fig. 6

Approximating the ankle with linking truncated circular cones

Grahic Jump Location
Fig. 7

The proposed cloud service framework

Grahic Jump Location
Fig. 8

A use scenario of the cloud service: (a) select a shoe model to try on, (b) record a clip of the foot motion using Kinect v2, (c) upload the clip to the cloud, (d) foot recognition in a depth frame, (e) completion notification to the receiver, and (f) watch the virtual try-on process in a smart device

Grahic Jump Location
Fig. 9

Snapshots of the virtual try-on process

Grahic Jump Location
Fig. 10

Poor occlusion culling results (left: unexpected spacing, right: incorrect blocking)



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In