0
Research Papers: SPECIAL SECTION PAPERS

Proto-TAI++: Exploring Perceptually Consistent Creation of Planar Shape Assemblies Through Multimodal Tangible Interactions

[+] Author and Article Information
Cecil Piya

C-Design Lab,
School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907
e-mail: cpiya@purdue.edu

Vinayak

C-Design Lab,
School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907
e-mail: fvinayak@purdue.edu

Karthik Ramani

Donald W. Feddersen Professor
School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907
e-mail: ramani@purdue.edu

1Corresponding author.

Contributed by the Computers and Information Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received March 14, 2016; final manuscript received July 12, 2016; published online August 11, 2016. Editor: Bahram Ravani.

J. Comput. Inf. Sci. Eng 16(3), 030906 (Aug 11, 2016) (10 pages) Paper No: JCISE-16-1888; doi: 10.1115/1.4034266 History: Received March 14, 2016; Revised July 12, 2016

We explore tangible 3D interactions that allow for geometric and perceptual correspondence between a midair modality and the 3D elements it controls. To demonstrate our approach, we use a concrete application scenario through Proto-TAI++, a multimodal system using pen-based drawing of planar shapes and their subsequent midair assembly via a hand-held planar proxy. The planarity of the proxy is a key element that physically embodies virtual planar shapes during 3D manipulation. We also investigate advanced operations such as patterning and blending using asymmetric bimanual interactions that augment midair motion with multitouch gestures. We describe a three-stage user evaluation with our system wherein our goal is to (a) study the effectiveness of the planar proxy as a tangible 3D modality, (b) contrast key features of our approach with a GUI-based planar shape assembly system, and (c) evaluate user experience and performance in creative tasks using Proto-TAI++.

Copyright © 2016 by ASME
Topics: Manufacturing , Shapes
Your Session has timed out. Please sign back in to continue.

References

Figures

Grahic Jump Location
Fig. 1

General workflow: (a) planar shapes are drawn and modified on a digital surface, (b) concurrent midair inputs from a planar proxy and multitouch inputs on the tablet (with nondominant hand) are used to assemble shapes in 3D, and (c) the shapes can be laser cut and physically assembled1

Grahic Jump Location
Fig. 2

Generalized setup of proposed system

Grahic Jump Location
Fig. 3

Planar proxy: (a) preliminary version and (b) refined version with increased thickness and a flat handle. Arrows define local reference.

Grahic Jump Location
Fig. 4

Tracking proxy’s motion: (a) interaction space viewed from depth sensor, (b) depth map with 3D position and bounding box, (c) 3D data inside bounding box, and (d) segmented faces with normals

Grahic Jump Location
Fig. 5

Segmenting planar regions on 3D data: (left) three faces visible to sensor; (middle) two faces visible; and (right) only one face visible

Grahic Jump Location
Fig. 6

Temporal coherence of orientation: (a) face normals at frame i and (b) assigning measured directions in frame i + 1 to appropriate faces, based on angular proximity with normals in frame i

Grahic Jump Location
Fig. 7

Using planar cursor to clutch and manipulate: (a) a planar shape and (b) a nonplanar shape

Grahic Jump Location
Fig. 8

Task 1: (left) docking task and (right) task completion times for seven asymmetric shapes at three levels of docking tolerances

Grahic Jump Location
Fig. 9

Task 2: (left) guiding a circular hoop across a 3D wire and (right) duration of maintaining the wire within the hoop

Grahic Jump Location
Fig. 10

Two-dimensional drawing interface displayed on the tablet

Grahic Jump Location
Fig. 11

Drawing operations: (top) drawing shapes; (middle) editing shapes; and (bottom) creating a symmetric shapes

Grahic Jump Location
Fig. 12

Converting 2D drawing to planar mesh for use in assembly

Grahic Jump Location
Fig. 13

Three-dimensional scene for assembling planar shapes

Grahic Jump Location
Fig. 14

Three-dimensional assembly interactions using asymmetric bimanual coordination of the proxy’s motion with multitouch gestures on the tablet. Each cell shows the hover state of the planar cursor, the multitouch input, the proxy’s motion, and the outcome in the 3D scene: (a) clutch shape for 6DOF control, (b) rotate shape about normal, (c) rotate full assembly, (d) clutch shape for translation only, (e) uniform scale shape, (f) translate full assembly, (g) linear pattern, (h) contour pattern, and (i) sectional blending.

Grahic Jump Location
Fig. 15

(a) Initial assembly, (b) adjusted neighbors after base shape modification, and (c) graph structure representing the assembly

Grahic Jump Location
Fig. 16

(a) Automatic insertion of a slit joints, (b) laser cut shapes, and (c) physical assembly built using Proto-TAI++

Grahic Jump Location
Fig. 17

Ground truth assemblies that are replicated by participants during user study 1

Grahic Jump Location
Fig. 18

Statistical distribution of task completion times using the two interfaces

Grahic Jump Location
Fig. 19

Error metrics with respect to ground truth shapes: (left) positional and angular errors, (middle) twist error, and (right) shape similarity

Grahic Jump Location
Fig. 20

Statistical distribution for errors on user-defined shapes measured with respect to corresponding ground truth shapes. Each pair of box plots shows results for a given shape using the two interfaces. Pairwise t-test results (p-values) for comparing errors are shown above each pair. The shapes are labeled according to the assembly they lie on and their id within the assembly (e.g., A4S3).

Grahic Jump Location
Fig. 21

Five-point Likert scale results on usability of two interfaces. The rows correspond to interfaces and each column indicates a given usability metric. Each number represents how many participants selected a given option in the scale.

Grahic Jump Location
Fig. 22

Planar shape assembly models constructed by participants using Proto-TAI++. The models are grouped by broad level categories observed in the study.

Grahic Jump Location
Fig. 23

Five-point Likert scale-based feedback: (a) factors influencing participants’ self-evaluated creative expression when using Proto-TAI++ and (b) ease of use of specific system features and interactions

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In