0
Research Papers

# FEAsy: A Sketch-Based Tool for Finite Element AnalysisOPEN ACCESS

[+] Author and Article Information
Sundar Murugappan

C-Design Lab,
School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907
e-mail: sundar.murugappan@gmail.com

Cecil Piya

C-Design Lab,
School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907
e-mail: cpiya@purdue.edu

Maria C. Yang

Associate Professor
Department of Mechanical Engineering;
Engineering Systems Division,
Massachusetts Institute of Technology,
Cambridge, MA 02139
e-mail: mcyang@mit.edu

Karthik Ramani

Donald W. Feddersen Professor
School of Mechanical Engineering;
School of Electrical Engineering (by courtesy),
Purdue University,
West Lafayette, IN 47907
e-mail: ramani@purdue.edu

Contributed by the Design Engineering Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received July 18, 2016; final manuscript received July 31, 2016; published online February 16, 2017. Editor: Bahram Ravani.

J. Comput. Inf. Sci. Eng 17(3), 031009 (Feb 16, 2017) (11 pages) Paper No: JCISE-16-2015; doi: 10.1115/1.4034387 History: Received July 18, 2016; Revised July 31, 2016

## Abstract

Freehand sketching is an integral part of early design process. Recent years have seen an increased interest in supporting sketching in computer-based design systems. In this paper, we present finite element analysis made easy (FEAsy), a naturalistic environment for static finite element analysis. This tool allows users to transform, simulate, and analyze their finite element models quickly and easily through freehand sketching. A major challenge here is to beautify freehand sketches, and to this extent, we present a domain-independent, multistroke, multiprimitive method which automatically detects and uses the spatial relationships implied in the sketches for beautification. Further, we have also developed a domain-specific rules-based algorithm for recognizing commonly used symbols in finite element analysis (FEA) and a method for identifying different contexts in finite element modeling through combined interpretation of text and geometry. The results of the user study suggest that our proposed algorithms are efficient and robust. Pilot users found the interface to be effective and easy to use.

<>

## Introduction

Freehand sketching is an activity that can take place throughout the engineering design process and is a natural, efficient, and convenient way to capture, represent, and communicate design ideas [1,2]. Sketches are particularly useful in early stages of design where their fluidity and ease of construction enable creativity and the rapid exploration of ideas [3,4]. Over the past few years, there has been increased interest in supporting freehand sketching in user interfaces and tools for various applications in diverse domains such as computer-aided design (CAD), simulation, and computer animation. As freehand sketch-based interfaces mimic the pen–paper paradigm of interaction, they may provide a host of advantages over the traditional windows, icons, menus, and pointers (WIMP) style graphical user interfaces (GUIs). Designers can seamlessly and directly interact with the computer with only limited training, whereas in menu-based interfaces, the users are forced to learn the system rather than the system having to learn users' intentions.

In this paper, we describe finite element analysis made easy (FEAsy), a sketch-based interface for static finite element analysis. FEAsy allows users to transform, simulate, and analyze their finite element models quickly and easily through freehand sketching (Fig. 1). A major challenge here is the need for techniques to transform ambiguous freehand strokes of a sketch into usable parametric geometric entities making up a formal diagram. Such a transformation is referred to as “beautification” [57]. Most current beautification methods do not consider important information implied in sketches such as the spatial relationships between different primitives in a stroke and between strokes [8]. These spatial relationships are usually represented as geometric constraints (such as parallelism and tangency). Cognitive studies show that users preferentially attend toward certain geometric features while drawing and recognizing shapes [9]. To this extent, we present a multistroke, multiprimitive beautification method that identifies these spatial relationships and uses them to drive beautification. We also posit that it is more intuitive to specify loading and boundary conditions through symbols as shown in Fig. 1, than through traditional menu-based input. Hence, we have developed a domain-specific algorithm for recognizing commonly used symbols in FEA. In addition, we have also developed an algorithm for combined interpretation of geometry and symbols in the sketch to identify different contexts like loading and boundary conditions observed in finite element analysis.

We foresee this tool to be used in engineering education and early design. It will allow analyses to be conducted even earlier in the design process because it reduces the reliance on preparing formal computer models beforehand. Formal, computational models such as CAD models may require a great deal of preparation and a clear understanding of design details. However, at the preliminary stages of design, hand generated sketches are often more appropriate to explore a wide range of potential ideas. Our tool would permit formal analysis of ideas that exist as only a hand-drawn sketch. It can be used as a learning tool for undergraduate students especially in mechanical and civil engineering. The students can use this tool to quickly verify answers to hand-worked problems and also in preliminary stages of design projects to evaluate their ideas.

###### Contributions.

This paper extends our prior work [10] and makes a number of contributions to research in sketching, interfaces, and analysis in engineering design:

1. (1)a multistroke, multiprimitive beautification method that incorporates automatic constraint detection and solving, to transform the ambiguous freehand input into more structured formal drawings
2. (2)a symbol and text recognition algorithm for the finite element domain
3. (3)an algorithm for combined text and geometry interpretation, based on different contexts
4. (4)a novel interface that integrates freehand sketching, geometry constraint solving, and symbol recognition in a unified framework for structural and thermal finite element analysis

## Related Work

This section provides an overview of the past work in beautification and sketch-based interfaces for varied applications.

###### Beautification—Segmentation and Recognition.

Two of the main challenges that have hindered the development of a robust beautification system are: segmentation—identification of critical points on the strokes, and recognition—classifying the segment between adjacent critical points as low-level geometric primitives (such as lines, circles, and arcs). Much earlier work [5,1113] assumed that each pen stroke represented a single primitive such as a line segment or a curve in sketches. In spite of their simplicity, the strategy based on single primitive or stroke usually results in a less natural interaction because of the constraints imposed on the users' drawing freedom. Other works [14,15] have also utilized predefined templates of higher-order splines to neaten sketch inputs and smoothly combine the segments. By taking advantage of the interactive nature of sketching, several works [1618] have used the pen-speed and curvature properties of the stroke to determine the critical points. They found that it was natural to slow the pen when making intentional discontinuities in the shape. When a user is sketching at a constant speed, many segmentation points will be missed due to this biased assumption. Kim and Kim [19] proposed new metrics based on curvature—local convexity and local monotonicity for segmentation. Hammond and coworkers [20] introduced an effective method to find corners in polylines. Their method is founded on a simple concept based on the length between two points. They showed higher accuracy over Sezgin et al. [16] and Kim and Kim [19]. Other approaches to segmentation utilized artificial intelligence, such as the template-based approach [21], conic section fitting [3], and domain-specific knowledge [22]. Despite their relative success in sketch segmentation, these are dependent on various restrictive conditions. For example, a large number of sketch examples are required for the purpose of training the computer in the methods proposed in Ref. [3]; otherwise, the segmentation performance will be affected. For recognizing the segments, Shpitalni and Lipson [3] and Zhang et al. [23] used a least-squares based method. Sezgin et al. [16] and Wolin et al. [20] compared the Euclidean distance between adjacent critical points and accumulated arc length of the segment. The ratio of arc length to Euclidean distance is close to 1 for a linear region and significantly higher for curved region. Xiong and LaViola [24] improved the algorithm described in Ref. [20] to include curves in addition to just lines. However, the algorithm does not recognize corners at a place where a line smoothly (tangentially) transitions into an arc and also those between two arcs.

More recently, with the advent of commodity level depth sensors (e.g., MS Kinect), there has been some research devoted toward segmentation and recognition of freeform 3D strokes drawn in midair using finger-based gestures. For example, Taele and Hammond [25] investigate techniques for developing intelligent interfaces and optimal interaction techniques for surfaceless sketching. Similarly, Babu et al. [26] provide a system from recognizing freehand 3D input strokes and matching them to specific predefined 3D symbols. However, 3D sketching methods are limited as they cannot provide tactile feedback required for controlled sketch creation and also require advanced display media for colocation of the interaction and modeling spaces. As a result, their use is restricted to simple symbolic inputs and is therefore unsuitable for creating shapes for structural analysis. Wang [27] utilizes stereoscopic display with bimanual interactions through digital motion trackers to enhance sketching in 3D. But their system relies on dedicated hardware that are not commonly available.

###### Sketch-Based Interfaces.

The emergence of pen-input devices such as tablet PCs, large electronic Whiteboards, and personal digital assistants has led to demand for sketch-based interfaces in diverse applications [6]. Here, we list a few examples of such existing experimental systems. In CAD-based applications such as QuickSketch [28] and SKETCH [29], the user has to draw objects in pieces, i.e., only one primitive at a time, thereby reducing the sense of natural sketching. Arisoy et al. [30] utilize a predictive modeling approach to automatically complete preliminary rough sketches created by users. Our interface also facilitates automatic completion of sketches, but in addition provides a suggestive interface to allow users to explore different possibilities resulting from their input.

Sketch-based interfaces have also been used in early design [31] and in user-interface design [32]. ShadowDraw [33] provides dynamically adaptive suggestions in the background to guide users create esthetically pleasing sketches. Similarly, in Juxtapose [34], sketch inputs are used to drive search of 2D images with the intent of making serendipitous discoveries during clipart composition. In contrast, our system provides dynamically updating suggestions to guide parametric sketching for engineering applications. Gesture-based systems have been explored in 2D pen-based applications [35,36] where input strokes are converted or replaced with predefined primitives. Other works have also explored creation of 3D wireframe models based on multiview planar sketch inputs [37] or scaffold-based perspective drawings [38]. In contrast, our work is mainly related to creation of 2D geometry for structural analysis. ParSketch [39] is a sketch-based interface for editing 2D parametric geometry. MathPad [40] is a tool for solving mathematical problems. Kara et al. [41] developed a sketch-based system for vibratory mechanical systems. Kirchoff's pen [42] is a pen-based tutoring system that teaches students to apply Kirchhoff's voltage law and current law. Hutchinson et al. [43] developed a unified framework for structural analysis. They used an existing freehand sketch recognition interface which is not robust in handling freehand strokes representing multiple primitives combined together. In addition open circular arcs or curves are not handled, constraining the variety of input that can be specified and also designer's drawing freedom. Moreover, the system does not address the problems related to the ambiguous nature of freehand input.

For symbol recognition, Fonseca and Jorge [44] developed an online scribble recognizer called CALI. The recognition algorithm uses fuzzy logic and geometric features, combined with an extensible set of heuristics, to classify scribbles. Since their classification relies on aggregated features of the pen strokes, it might be difficult to differentiate between similar shapes. Kara and Stahovich [45] described a hand-drawn symbol recognizer based on a multilayer image recognition scheme. Similarly, Johnson et al. [46] enable users to apply standard drafting symbols to define constraints such as equality and perpendicularity, and edit 2D shapes by latching or erasing sketch segments. However, these methods require training, and in the case of Ref. [45] are also sensitive to nonuniform scaling. Veselova and Davis [9] used results from perceptual studies to build a system capable of learning descriptions of hand-drawn symbols which are invariant to rotation and scaling.

## Overview of the Approach

Freehand sketches are usually composed of a series of strokes. A stroke is a set of temporally ordered sampling points captured in a single sequence of pen-down, pen-move, and pen-up events [47]. Sketches can be created in our system using any of a variety of devices that closely mimic the pen–paper paradigm. We use a Wacom Cintiq 21UX digitizer with stylus, tablet PCs, and a traditional mouse. Both Wacom and tablet PCs are particularly suited to natural interaction, enabling the user to sketch directly on a computer display. Figure 2 shows the pipeline through which the input strokes are processed in our system. In FEAsy, the strokes are input in either the “geometry” mode or in the “symbol” mode. Accordingly, the raw input strokes representing geometry are colored in black and those representing symbols are colored in red. Each stroke input in geometry mode is beautified, i.e., decomposed into low-level geometric primitives with minimal error. The system then identifies the spatial relationships between the primitives. These relationships are represented as geometric constraints which are then solved by a geometry constraint solver. The output from the solver is the beautified version of the input which is updated on the screen automatically. The strokes input in symbol mode are processed unlike in geometry mode. The red-colored strokes are first clustered into stroke groups. Then, the stroke groups are classified as either text or symbol and recognized. Finally, the symbols, text, and geometry are interpreted together for understanding the various contexts in the sketch. The sketch is now ready for finite element problem study. Sections 4–6 describe each of these steps in detail.

###### An Example.

Figure 3 shows a step-by-step process of analyzing a bracket from its freehand sketch. The user starts the geometry creation process by sketching a freehand stroke in the geometry mode as shown in Fig. 3(a). The lower circle is the starting point of the stroke, and the upper circle depicts the end point. The blue arrows show the direction of the stroke. At the completion of the stroke, the system automatically beautifies the input. The beautified output is shown in Fig. 3(b). Next, the user adds a freehand stroke to the sketch. In this case, it is a hole in the bracket (see Fig. 3(c)). The final result after beautification is shown in Fig. 3(d). The geometry creation process for the bracket is now complete. The user then switches modes to insert symbols. Figure 3(e) shows a beautified sketch with input symbols. In the next step, the symbols are recognized and the sketch is interpreted. The output is updated on the screen as shown in Fig. 3(f)). Once the sketch is complete and processed, the user specifies the material properties, element description, and meshing parameters for “finite element integration” (Fig. 3(g)), and all the information is exported as a set of commands suitable for import in ansys (Fig. 3(h)). These commands are then run and solved in ansys. Figure 3(i) shows the deformation results of the bracket.

## Beautification

Beautification aims at simplifying the representation of the input where the various points of the strokes are interpreted and represented in a more meaningful manner. Our approach for transforming the input to formalized representations (i.e., beautification) is based on the architecture shown in Fig. 4. There are five steps in the pipeline, namely, resampling, segmentation, recognition, merging, and geometry constraint solving. Figure 4 shows the various steps along with the actual outputs generated in the system. However, it is to be noted that only the final beautified sketch is visible to the users and other intermediate outputs are generated for illustration purposes. Figure 4(a) shows a user-drawn freehand stroke. This is an example of a single stroke representing multiple primitives connected together. Figure 4(b) shows the raw data points (blue circles) as sampled by the hardware, and Fig. 4(c) illustrates the uniformly spaced points after resampling (green circles). The segmentation step explained in Sec. 4.2 identifies the critical points (red circles) shown in Fig. 4(d). Then, the segments between the adjacent critical points are recognized and fit with primitives (Fig. 4(e)). The status of the freehand sketch after merging is shown in Fig. 4(f). Finally, the sketch is beautified considering the geometric constraints (Fig. 4(g)). The aforementioned steps are explained in detail in Secs. 4.14.6. For simplicity, we limit the discussion to a single stroke in a sketch. All the other strokes are processed similarly.

###### Stroke Resampling.

The sampling frequency of the mechanical hardware coupled with the drawing speed of the user results in nonuniform samples of the raw freehand input. Evenly spaced points are important for the segmentation algorithm to work efficiently. To achieve uniform sampling, we resample the points of the input stroke such that they are evenly spaced. We used a fixed interspacing distance, Id of 200 HIMETRIC units (1 HIMETRIC = 0.01 mm = 0.0378 pixels). The resampling algorithm discards any sample within the Id of earlier samples and interpolates between samples that are separated by more than Id. The start and end points of the stroke are by default added to the resampled set of points. Figure 4(c) shows the result of resampling for the stroke.

###### Segmentation.

In our system, a single freehand stroke can represent any number of primitives connected together. The task of the segmentation routine is to find those critical points that divide the stroke into its constituent primitives. These critical points are “corners” of the piecewise linear strokes and also the places where curve and line (curve) segments connect. Our segmentation algorithm builds upon the approach described in Ref. [20], which works well for strokes composed of only line segments. One of the drawbacks of this method is that the algorithm often misses identification of corners at heavily obtuse angles. We address this drawback and also improve their algorithm to accommodate curves in addition to line segments. We are interested in improving this algorithm especially for its simplicity (easy to program) in implementation, high efficiency, and at the same time not being computationally intensive. They described a measure called straw (chord length) which in essence is a naive representation of the curvature of the stroke. The straw at each point pi is computed as $strawi=|pi−w,pi+w|$, where w is a constant window and $|pi−w,pi+w|$ is the Euclidean distance between the points $pi−w$ and $pi+w$. As the stroke turns around a corner, the straw length starts to decrease and a local minimum value corresponds to a likely critical point. However, when there is smooth continuity between a line and an arc or between two arcs, the straw length does not vary much and it fails to identify the transition in such regions. Hence, we use another such measure, chord angle, which is effective in identifying these gradual changes in addition to finding the corners.

After resampling the stroke, we compute chord angle for the resampled points pw to $pn−w$, where “n” is the total number of resampled points and w is a constant window. The chord angle at each point pi is computed as follows: Display Formula

(1)$∡pi=arccos(pipi−w⋅pipi+w|pipi−w||pipi+w|)$

The likely critical points of the stroke are those indices where the “chord angle” is a local minimum, which is lesser than a threshold (“t”). Figure 5 shows the computation of the chord angle. The blue circles represent the resampled points, and “$θ$” represents the chord angle computed using Eq. (1). To avoid the problem posed by choosing a fixed threshold, we set the threshold to be equal to the median of all the chord angle values. For the stroke in Fig. 4(a), the initial set of critical points obtained is shown in Fig. 4(d). By default, the start and end points of a stroke are considered as critical points. A window of uniformly spaced points is used to compute the curvature (chord angle), which smoothens out the noise, if any in the input stroke. The larger the window, the larger the smoothing effect resulting in missed critical points. As in Ref. [17], we found that setting the window size, w = 3 to be effective irrespective of the user or the input device used.

###### Recognition.

The next task after segmentation is to classify and fit the segments between adjacent critical points as low-level geometric primitives. The current implementation of our system recognizes lines, circular arcs, and circles. Our recognition method is based on least-squares analysis [48], but the computation of parameters of best fit line and circular arc differ from the traditional approach. Usually, the least-square fit of lines and arcs results in the end points of the primitives to be moved to new locations as shown in Fig. 6. These new positions do not coincide with the original critical points of the stroke, and hence cause discontinuities between adjacent primitives of the stroke. To prevent such discontinuities, we fix the end points of the primitives to coincide with original critical points and then perform the analysis. Figure 6 shows the actual result of our recognition algorithm which has no discontinuities.

###### Fitting a Straight Line.

Let $SN={pi=(xi,yi)|i=1,2,.....,N}$ be the given points of the segment and let $P1(x1,y1)$ and $PN(xN,yN)$ be the end points of $SN$. These N points are fitted by a straight line, $y=mx+c$, where m and c represent the slope and the intercept, respectively. As the end points of the line segments are fixed, the slope and the intercept can be estimated as follows: Display Formula

(2)$m=yN−y1xN−x1$
Display Formula
(3)$c=12[(yN−mxN)+(y1−mx1)]$

The average distance from the points (xi, yi) to the fitted straight line, El, can be calculated using the following formula: Display Formula

(4)$El=∑i=1N|(mxi+c)−yi|N$

###### Fitting a Circular Arc.

$SN$ can also be fitted as a circular arc, $(x−a)2+(y−b)2=R2$, where C(a, b) is the center of the arc and R is the radius.

As the start and end points of the arc are fixed, the center of the arc should lie on the perpendicular line that passes through the midpoint of the line connecting the end points of the arc (Fig. 7). Let $P1(x1,y1)$ and $PN(xN,yN)$ be the end points of the arc, $C′(a′,b′)$ be the mid point of line joining P1 and PN, and $n̂=(nx,ny)$ be the normal to the line joining P1 and PN. Therefore

Display Formula

(5)$C′=12(P1+PN)$
Display Formula
(6)$C=C′+tn̂, t ε ℜ$

The center C(a, b) is the solution to the problem Display Formula

(7)$Find C minimizing ∑i=2N−1‖Pi−C‖2$

using Eq. (6), the solution to Eq. (7) is Display Formula

(8)$t=1N−2∑i=2N−1(xi−a′)nx+(yi−b′)ny$

and hence the radius, $R=‖PN−C‖$. The average distance from the points (xi, yi), i = 1 to N, Ea, can be calculated using the following equation: Display Formula

(9)$Ea=∑i=1N|(xi−a)2+(yi−b)2−R|N$

After finding the errors, the segment is typically classified by the primitive that matches with the least error. However, the line segments can always be fit with high accuracy as an arc with a very large radius. In such cases, if the arc length is less than 15 deg, we classify it as a line. Similarly, an arc is classified as a circle if its arc length is close to 2π.

###### Merging.

The initial critical points set obtained through segmentation routine may contain some false positives. The merging procedure repeatedly merges adjacent segments, if the fit for the merged segment is lower than a certain threshold. For every ith segment, we try merging it with (i−1)th and (i+1)th segment. Let these new segments be seg1 and seg2. The fit errors for seg1 and seg2 are calculated according to Sec. 4.3. For the segment with least error among seg1 and seg2, merging occurs if and only if the error is less than the sum of the corresponding errors of the original segments. For example, in Fig. 4(e), the two lines and an arc on the right were merged into one single arc (see Fig. 4(f)).

###### Geometry Constraint Solving.

Geometric constraints are usually classified as either (1) explicit constraints, which refer to the constraints that are explicitly specified by the user such as dimensions—distance between a point and a line or angle between two lines, or (2) implicit constraints, which refer to the constraints that are inherently present in the sketch such as concentricity and tangency. It is natural for users to express geometric constraints implicitly when they are sketching. Our system infers and satisfies the constraints automatically without much intervention from user using the method described in Ref. [49]. Figure 8 lists the different kind of constraints inferred in our system between points, lines, circular arcs, and circles. We have integrated the LGS2D [50] geometry constraint solver with our system for constraint solving purposes. The set of primitives along with the constraints are input to the solver, and after satisfying the constraints, the solver returns the modified primitives with their new locations. Figures 4(f) and 4(g) show the primitives of the sketch before and after constraint solving, respectively. The core technology of LGS2D is a combination of symbolic and numerical methods for solving systems of geometrical constraints. The main symbolic method used in LGS2D is a variation of constraint graph analysis, based on abstract degree-of-freedom approach [51].

###### Resolving Ambiguities With Interaction.

Any recognition system is not devoid of ambiguities. Our system provides the interface to correct the errors through simple interactions. Errors in segmentation include missed and unnecessary critical points. In our system, when the user taps on or near a critical point with the stylus, the system first removes that critical point and the corresponding two primitives that share this point. This results in an unrecognized segment which is then classified and refit. The user can also add a segmentation point in a similar manner. The nearest point on the stroke to the clicked location is used as the input point where the existing primitive is broken into two primitives. Errors in segment recognition correspond to primitive misclassification. An input stroke drawn by holding down a button on the stylus is recognized as a pulling gesture. The primitive that is closest to the starting point of this gesture is the one to be pulled, and accordingly, its classification is altered, i.e., if the primitive was a line, it is refit as a circular arc and vice versa. Additionally, the user can erase a primitive, a stroke, or a part of stroke using the eraser end of the stylus, just as using a pencil eraser.

## Symbol Recognition and Sketch Interpretation

The symbols drawn in finite element domain, both in academia and research, have well-defined and standardized forms. The list of symbols commonly used in finite element domain (i.e., for loading and boundary conditions) is shown along with other symbols recognized in our system in Fig. 9. Figure 10(a) shows an example of beautified 2D bracket drawn in geometry mode. The sketch consists of seven line segments (L1–L7), two circular arcs (A1 and A2), and a circle (C1). For visual clarity, once the geometry is beautified, the recognized lines are drawn in black and the arcs (circles) in green. Figure 10(b) shows the various red-colored strokes input in symbol mode that represent dimensions, loading, and boundary conditions. Sections 5.15.3 describe the various steps in processing these strokes for symbol recognition and sketch interpretation (Fig. 10).

###### Clustering.

The first step in processing this collection of symbol strokes is to cluster them into smaller groups. We use both a temporal and a spatial proximity strategy to group strokes. This stems from the observation that a group of strokes comprising a symbol are generally drawn close to each other and continuously. In addition, the system should not constrain the user to complete a symbol before moving on to the next one. For example, if the user specifies “P = 100” as a loading condition initially and later wishes to change it to “P = 1000,” the operations required must be as simple as adding a zero to the input rather than have to erase and rewrite the whole text again. Hence, the criteria for clustering require the strokes to be within a spatial threshold distance of 100 HIMETRIC units and (or) the time gap between continuous strokes is less than 500 ms. Figure 10(c) shows the results of the clustering, where a dashed bounding box is drawn around each group.

###### Text and Symbol Recognition.

The next step is recognition of each stroke group, where each stroke group is comprised of either text or symbols. We use the height (<1.2 cm) and width of bounding box (<2.2 cm) as the criteria to distinguish between text and symbols. The stroke groups that are classified as text are next recognized using the built-in handwriting recognizer (Microsoft Tablet PC SDK). The texts in the sketch are primarily of two types: (1) loading conditions (force, temperature, or pressure) with alphabets—F, T, or P on the left-hand side of an “equal to” symbol and numbers on the right-hand side, and (2) dimensions, which are made up of only numbers. This observation helps in robust recognition of text and also helps in correcting misclassification of texts and symbols. After the identification of texts, the next step is to recognize the remaining stroke groups. On quick observation, one can see that almost all of the symbols are comprised of either lines or circles, and only the “moment” symbol consists of an arc. Also, some symbols like “roller” have different variations, where there is a difference in the number of circles drawn. Though these symbols seem different, there are certain distinct properties for each symbol or group of symbols that are different from other symbols (or groups). For example, the “fully constrained” symbol is different from roller symbol, as it can be distinguished with the presence or absence of circles. In this case, the number of circles does not matter for the differentiation. We have created similar heuristic-based rules to recognize different symbols. The reason behind using such an approach is that the number of symbols in this set is finite and each symbol has some distinct properties that can be used to differentiate from the other symbols in spite of the possible variations. Also, there is no training required. For the recognition of various symbols, we have built custom recognizers by extending simple gesture recognition library (SIGER) [52] using vector strings and regular expressions.

###### Sketch Interpretation.

The text, geometry, and symbols in a sketch have an inherent structure and they all combine only in some specific ways. For example, a dimensional value can never be associated with a straight single headed arrow; a loading condition can never be associated with a fully constrained symbol; and any arrow cannot exist on its own without an associated text group. This kind of reasoning helps to correct errors automatically allowing for robust sketch interpretation.

## Finite Element Integration

The final step is to setup the problem for finite element analysis. Our system provides the interface to (see Fig. 3(g)) input the material information, element type and description, and mesh size (if necessary). Our current implementation of the system supports three types of elements which are commonly used in structural, thermal, and static finite element analysis. Similarly, the users can also specify what results they wish to view after the analysis. Currently, the system allows users to choose from von Misses stress, reaction forces, deflections, and temperature. Figures 3(g)3(i) show the finite element integration for the bracket in Fig. 10. Here, the three-dimensional bracket is modeled as a two-dimensional problem with uniform thickness = 0.5 in. The finite element specific parameters (ansys) include material: steel, $ε=30×106, ν=0.3$; element type: PLANE42; element size: 0.5. After specifying the necessary input, the system exports the model geometry, boundary conditions, loads, material, element, and meshing information to a unified file specific for ansys (APDL commands). Figure 3(h) shows the generated ansys specific code, and Fig. 3(i) shows the “displacement vector sum results” plotted results in ansys.

## User Study

We conducted a preliminary user study to test the system. Through this study, we aimed to find out if the users were able to finish the task given and whether they were able to accomplish it with fewer interactions and strokes than a system that supports only single-primitive strokes. Also, we wanted to receive feedback from participants about the tool for future improvements.

System: Our prototype was ported on to a PC with Wacom Cintiq 21UX LCD display. This display offers the users a way to work naturally and intuitively by using a digital pen, directly on the surface of an LCD display.

Subjects: Six graduate students in mechanical engineering participated in this study and all of them were familiar with sketching aspects of CAD programs (such as autocad and pro/engineer), and hence were well aware of use of geometric constraints in making diagrams. They were also familiar with using ansys for finite element analysis. In addition, they had used digitizing media like tablet PCs and (or) personal digital assistants before but not the Wacom line of products.

Measurement: The measures in this study included critical points segmentation accuracy, primitives recognition accuracy, symbol recognition accuracy, and context interpretation accuracy.

Training Process: The participants were trained for 10 min with the capabilities of the system, i.e., the two modes of input—geometry and symbol; beautification of the freehand strokes in geometry mode; symbol recognition and sketch interpretation; and finally, finite element integration. In addition to illustrating the work flow, we also demonstrated its limitations, i.e., the system recognizes only lines, arcs, and circles and does not handle overtracing (making several overlapping strokes, such that the strokes are perceived as a single object collectively); interaction techniques (such as clicking on a critical point) for correcting errors during beautification and symbol recognition; and finally, symbol recognition might fail when two different symbols overlap each other. In addition, the participants were given 15 min to get acquainted with the system.

Task: On the completion of the training process, the participants were asked to sketch and solve the four problems shown in Fig. 11. The total amount of time given was 1 h. The problems were carefully chosen in such a way that they tested all the different capabilities of our system. In addition, these examples illustrate a good range of problems that can be solved using our system. Each problem had a verbal description of the boundary conditions, loads, and dimensions (collectively termed as nongeometric information) accompanied by a graphic that represented just the geometry, devoid of dimensions and symbols. The reason behind such a formulation was to analyze how the users input the nongeometric information in the symbol mode and also to remove any bias on how the information should be input. The problem types were: (1) a static plane stress structural problem (Fig. 11(a)), (2) a static two-dimensional truss problem (Fig. 11(b)), (3) a three-dimensional structural problem modeled as a static, two-dimensional problem with constant thickness (Fig. 11(c)), and (4) a steady-state heat conduction problem (Fig. 11(d)). The problem descriptions are as follows:

Problem 1. Plot the von Mises stress for the shape in Fig. 11(a) and the following loading conditions. A flat rectangular plate is made of steel (ε = 210,000 MPa and ν = 0.3) with two holes and a constant thickness of 0.75 cm. The width of rectangular plate = 20 cm and height = 10 cm. The two holes must be completely inside the plate and on the same imaginary horizontal line, but should not touch the edges of the plate or each other. The left end of the rectangular plate is welded (fully constrained), and a uniform pressure of 0.1 MPa acts along the right end of the plate. Use PLANE 42 element in plane stress with thickness and a mesh size of 0.25.

Problem 2. Plot the displacement vector sum for the shape in Fig. 11(b) and loading conditions. The material properties are: ε = 210,000 MPa and ν = 0.3. The radius of inner arc = 10 cm and outer arc = 15 cm. The arcs are concentric and the length of a horizontal line segment = 20 cm. The top left edge is fully constrained, and a point load = 750 N acts downward at the bottom left point. Use PLANE 42 element in plane stress and a mesh size of 0.5.

Problem 3. Plot the displacement vector sum for the truss in Fig. 11(c) consisting of six joints and nine links. Here, links L1, L2, L4, and L5 are horizontal (parallel to X-axis). Similarly, (L6, L8, and L3) and (L7 and L9) are parallel. Node N1 is fully constrained, while node N2 is constrained only in the y-direction and free in the x-direction. A load of 100 N acts along negative y-direction on N5, and a load of 200 N acts along the positive x-direction on N4. The material properties are same as previous examples. Use LINK element with cross-sectional area of 0.5 square units.

Problem 4. Plot the temperature contour plot for the shape in Fig. 11(d) and boundary conditions: a square plate (width = 10 and thickness = 1) with a circular hole (diameter = 5) at the center. The top end of the plate is constrained at a temperature = 500 °C and the bottom edge at 100 °C. The left and right edges are maintained at 0 °C (fully constrained). The thermal conductivity (k) of the material is 10 W/mC. Use PLANE 55 element in plane stress and a uniform mesh size of 0.25.

## Results and Discussion

The six participants all solved the four problems within the allocated time, providing a total of 24 sketches. Some of the sample sketches drawn by the participants are shown in Fig. 12. Each row represents the work flow snapshots taken by the system for each of the problems. Figure 13 summarizes the results obtained after beautification in geometry mode. A total of 88 geometry strokes and 13 interactions were recorded for the geometry part of the problems. The input strokes comprised of both single primitive and multiprimitive types. A single-primitive stroke means a stroke can represent only one kind of primitive, i.e., a line, an arc, or a circle. On the other hand, a multiprimitive stroke can represent any number and any kind of primitives connected together. In all, 101 operations were required for successfully completing the geometry for all students and all problems. In contrast, if the system allowed only single-primitive strokes to be input to create the geometry, then the minimum number of total strokes required would be equal to 168, approximately 40% more number of strokes for geometry creation. In addition, this number does not reflect the number of operations that would be required to specify geometry constraints. Our system correctly segmented 236 critical points out of 238 with 99.2% accuracy and correctly recognized 173 out of 174 primitives, achieving 99.4% primitive recognition accuracy. These results indicate that we have a robust beautification algorithm, and the participants were able to draw the given shapes successfully and at the same time with minimal interactions and lesser time.

Figure 14 shows the results of text and symbol recognition algorithm implemented in the system. Our system correctly clustered and recognized 223 out of 228 stroke groups with an accuracy of 97.8%. The various symbols recognized and their individual accuracies across both problems and types are shown in Fig. 14. Problem 4 had five dimensions, two boundary conditions, and two loading (temperature constraints) conditions, which when specified in a single iteration can lead to a crowded sketch and a high chance for overlapping symbols. In one such instance, the fully constrained condition on the left end of the symbol overlapped with the temperature constraint on the bottom edge. The participant had to manually delete the overlapping stroke(s) and process it again. To avoid over crowdedness, three of the six participants resorted to two iterations of context interpretation, where they specified all the dimensional constraints in the first iteration and all the loading and boundary conditions in the second iteration. This process is similar to traditional finite element systems where users usually finish the problem geometry before specifying other constraints.

Figure 15 shows the results of various contexts interpreted in the user study. Our system correctly interpreted 129 out of 132 contexts in the sketch with an accuracy of 98.5%. The misinterpreted contexts were due to the overlapped symbols as explained in Sec. 7. The results suggest that our recognition and interpretation algorithms work robustly for the domain of static finite element analysis.

A two-factor analysis of variance test was performed to see if there were any variations in results by problem or by user for the four response variables, namely, critical point segmentation accuracy, primitive recognition accuracy, symbol recognition accuracy, and context interpretation accuracy. The test showed no significant main effect for the problem factor, $F(3,15)=3.29,p>0.05$, and no significant main effect for the user factor, $F(5,15)=2.90,p>0.05$, for each of the response variables.

At the end of the user study, each participant was asked if they (a) liked the interface and (b) had any suggestions for improvement. All of the participants reported that the system was easy to use and expressed a positive attitude toward drawing using freehand sketching. The participants were very appreciative that the system could infer the implicit constraints automatically and satisfy them simultaneously without the need for manually specifying them. Of the six participants, four of them suggested that the system infer symmetry and expand the geometric constraints set that can be either detected automatically or specified manually like equal radii constraints and equal lengths. For example, in even a relatively simple geometry such as in Problem 4, five-dimensional constraints were required to construct the square and place a circle at the center. A possible solution is to modify some of the dimensions directly in the left tree. This particular system is currently best suited for exploratory studies in early design where the actual dimensions are not that important in comparison to the shape; for in-classroom demonstrations, where the location of stress concentration or the deflection of a truss member is the focal point of discussion rather than the actual values.

## Conclusions

In this paper, we described FEAsy, a sketch-based interface that integrated freehand sketching with finite element analysis. We presented a beautification method that transforms ambiguous freehand input to more formal structured representations considering the spatial relationships implied in the freehand sketches. We also described algorithms for symbol and text recognition, and interpretation of various contexts in finite element domain. The results from the pilot study indicate that our algorithms are efficient and robust. However, more elaborate studies with a large sample size have to be done to see if such sketch-based interfaces are really a viable alternative. Our immediate future work is to integrate a finite element solver and provide visualization capabilities in the system making it a unified tool for finite element analysis.

## Acknowledgements

This work was partly supported by the NSF IIS CHS (Award No. 1422341), partly by NSF CMMI CPS: Synergy (Award No. #1329979), and the Donald W. Feddersen Chaired Professorship from School of Mechanical Engineering at Purdue University. Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

## References

Ullman, D. G. , Wood, S. , and Craig, D. , 1990, “ The Importance of Drawing in the Mechanical Design Process,” Comput. Graphics, 14(2), pp. 263–274.
Yang, M. C. , and Cham, J. G. , 2007, “ An Analysis of Sketching Skill and Its Role in Early Stage Engineering Design,” ASME J. Mech. Des., 129(5), pp. 476–482.
Shpitalni, M. , and Lipson, H. , 1997, “ Classification of Sketch Strokes and Corner Detection Using Conic Sections and Adaptive Clustering,” ASME J. Mech. Des., 119(1), pp. 131–135.
Yang, M. , 2009, “ Observations on Concept Generation and Sketching in Engineering Design,” Res. Eng. Des., 20(1), pp. 1–11.
Igarashi, T. , Kawachiya, S. , Tanaka, H. , and Matsuoka, S. , 1998, “ Pegasus: A Drawing System for Rapid Geometric Design,” CHI’98 ACM Conference on Human Factors and Computing Systems, Los Angeles, CA, Apr. 18–23, ACM, New York, pp. 24–25.
Plimmer, B. , and Grundy, J. , 2005, “ Beautifying Sketching-Based Design Tool Content: Issues and Experiences,” Sixth Australasian Conference on User interface, AUIC '05, Australian Computer Society, Inc., Darlinghurst, Australia, pp. 31–38.
Paulson, B. , and Hammond, T. , 2008, “ PaleoSketch: Accurate Primitive Sketch Recognition and Beautification,” 13th International Conference on Intelligent User Interfaces, IUI’08, ACM, New York, pp. 1–10.
Pu, J. , and Ramani, K. , 2007, “ Implicit Geometric Constraint Detection in Freehand Sketches Using Relative Shape Histogram,” 4th Eurographics Workshop on Sketch-Based Interfaces and Modeling, SBIM’07, ACM, New York, pp. 107–113.
Veselova, O. , and Davis, R. , 2006, “ Perceptually Based Learning of Shape Descriptions for Sketch Recognition,” SIGGRAPH’06 Special Interest Group on Computer Graphics and Interactive Techniques Conference, ACM, New York, p. 28.
Murugappan, S. , and Ramani, K. , 2009, “ FEAsy: A Sketch-Based Interface Integrating Structural Analysis in Early Design,” ASME Paper No. DETC2009-87727.
Forbus, K. D. , Lockwood, K. , Klenk, M. , Tomai, E. , and Usher, J. , 2004, “ Open-Domain Sketch Understanding: The nuSketch Approach,” AAAI Fall Symposium on Making Pen-Based Interaction Intelligent and Natural, Washington, DC, Oct. 21–24, AAAI Press, Menlo Park, CA, pp. 58–63.
Landay, J. A. , and Myers, B. A. , 2001, “ Sketching Interfaces: Toward More Human Interface Design,” Computer, 34(3), pp. 56–64.
Lin, J. , Newman, M. W. , Hong, J. I. , and Landay, J. A. , 2001, “ DENIM: An Informal Tool for Early Stage Web Site Design,” CHI’01 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, pp. 205–206.
McCrae, J. , and Singh, K. , 2011, “ Neatening Sketched Strokes Using Piecewise French Curves,” Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling, ACM, New York, pp. 141–148.
McCrae, J. , and Singh, K. , 2009, “ Sketching Piecewise Clothoid Curves,” Comput. Graphics, 33(4), pp. 452–461.
Sezgin, T. M. , Stahovich, T. , and Davis, R. , 2001, “ Sketch Based Interfaces: Early Processing for Sketch Understanding,” Workshop on Perceptive User Interfaces, PUI’01, ACM, New York, pp. 1–8.
Calhoun, C. , Stahovich, T. F. , Kurtoglu, T. , and Kara, L. B. , 2002, “ Recognizing Multi-Stroke Symbols,” AAAI Spring Symposium—Sketch Understanding, Palo Alto, CA, Mar. 25–27, AAAI Press, Menlo Park, CA, pp. 15–23.
Thiel, Y. , Singh, K. , and Balakrishnan, R. , 2011, “ Elasticurves: Exploiting Stroke Dynamics and Inertia for the Real-Time Neatening of Sketched 2D Curves,” 24th Annual ACM Symposium on User Interface Software and Technology, ACM, New York, pp. 383–392.
Kim, D. H. , and Kim, M.-J. , 2006, “ A Curvature Estimation for Pen Input Segmentation in Sketch-Based Modeling,” Comput.-Aided Des., 38(3), pp. 238–248.
Wolin, A. , Eoff, B. D. , and Hammond, T. A. , 2008, “ ShortStraw: A Simple and Effective Corner Finder for Polylines,” Eurographics 2008—Sketch-Based Interfaces and Modeling (SBIM), pp. 33–40.
Hse, H. , Shilman, M. , and Newton, A. R. , 2004, “ Robust Sketched Symbol Fragmentation Using Templates,” 9th International Conference on Intelligent User Interfaces, IUI’04, ACM, New York, pp. 156–160.
Gennari, L. , Kara, L. B. , Stahovich, T. F. , and Shimada, K. , 2005, “ Combining Geometry and Domain Knowledge to Interpret Hand-Drawn Diagrams,” Comput. Graphics, 29(4), pp. 547–562.
Zhang, X. , Song, J. , Dai, G. , and Lyu, M. , 2006, “ Extraction of Line Segments and Circular Arcs From Freehand Strokes Based on Segmental Homogeneity Features,” IEEE Trans. Syst., Man, Cybern., Part B, 36(2), pp. 300–311.
Xiong, Y. , and LaViola, J. J., Jr. , 2010, “ A Shortstraw-Based Algorithm for Corner Finding in Sketch-Based Interfaces,” Comput. Graphics, 34(5), pp. 513–527.
Taele, P. , and Hammond, T. , 2014, “ Developing Sketch Recognition and Interaction Techniques for Intelligent Surfaceless Sketching User Interfaces,” 19th International Conference on Intelligent User Interfaces, ACM, New York, pp. 53–56.
Babu, S. S. S. , Jaiswal, P. , Esfahani, E. T. , and Rai, R. , 2014, “ Sketching in Air: A Single Stroke Classification Framework,” ASME Paper No. DETC2014-34065.
Wang, C. , 2007, “ Drawing on Air: Input Techniques for Controlled 3D Line Illustration,” IEEE Trans. Visualization Comput. Graphics, 13(5), pp. 1067–1081.
Eggli, L. , ChingYao, H. , Bruderlin, B. , and Elber, G. , 1997, “ Inferring 3D Models From Freehand Sketches and Constraints,” Comput.-Aided Des., 29(2), pp. 101–112.
Zeleznik, R. C. , Herndon, K. P. , and Hughes, J. F. , 1996, “ SKETCH: An Interface for Sketching 3D Scenes,” 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH’96, ACM, New York, pp. 163–170.
Arisoy, E. B. , Orbay, G. , and Kara, L. B. , 2014, “ Predictive Modeling for 2D Form Design,” Computational Modeling of Objects Presented in Images. Fundamentals, Methods, and Applications, Springer, Heidelberg, Germany, pp. 286–291.
Damm, C. H. , Hansen, K. M. , and Thomsen, M. , 2000, “ Tool Support for Cooperative Object-Oriented Design: Gesture Based Modelling on an Electronic Whiteboard,” SIGCHI Conference on Human Factors in Computing Systems, CHI’00, ACM, New York, pp. 518–525.
Plimmer, B. , and Apperley, M. , 2004, “ Interacting With Sketched Interface Designs: An Evaluation Study,” CHI’04 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, pp. 1337–1340.
Lee, Y. J. , Zitnick, C. L. , and Cohen, M. F. , 2011, “ ShadowDraw: Real-Time User Guidance for Freehand Drawing,” ACM Trans. Graphics, 30, p. 27.
Benjamin, W. , Chandrasegaran, S. , Ramanujan, D. , Elmqvist, N. , Vishwanathan, S. , and Ramani, K. , 2014, “ Juxtapoze: Supporting Serendipity and Creative Expression in Clipart Compositions,” SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, pp. 341–350.
Gross, M. D. , and Do, E. Y.-L. , 1996, “ Ambiguous Intentions: A Paper-Like Interface for Creative Design,” 9th Annual ACM Symposium on User Interface Software and Technology, UIST’96, ACM, New York, pp. 183–192.
Landay, J. A. , and Myers, B. A. , 1995, “ Interactive Sketching for the Early Stages of User Interface Design,” SIGCHI Conference on Human Factors in Computing Systems, CHI’95, ACM, New York, pp. 43–50.
Bae, S.-H. , Balakrishnan, R. , and Singh, K. , 2008, “ ILoveSketch: As-Natural-As-Possible Sketching System for Creating 3D Curve Models,” 21st Annual ACM Symposium on User Interface Software and Technology, ACM, New York, pp. 151–160.
Schmidt, R. , Khan, A. , Singh, K. , and Kurtenbach, G. , 2009, “ Analytic Drawing of 3D Scaffolds,” ACM Trans. Graphics, 28, p. 149.
Naya, F. , Contero, M. , Aleixos, N. , and Company, P. , 2007, “ ParSketch: A Sketch-Based Interface for a 2D Parametric Geometry Editor,” Human-Computer Interaction. Interaction Platforms and Techniques (Lecture Notes in Computer Science, Vol. 4551), J. A. Jacko , ed., Springer, Heidelberg, Germany, pp. 115–124.
LaViola, J. J., Jr. , and Zeleznik, R. C. , 2006, “ MathPad2: A System for the Creation and Exploration of Mathematical Sketches,” SIGGRAPH’06 Special Interest Group on Computer Graphics and Interactive Techniques Conference, ACM, New York, p. 33.
Kara, L. B. , Gennari, L. , and Stahovich, T. F. , 2008, “ A Sketch-Based Tool for Analyzing Vibratory Mechanical Systems,” ASME J. Mech. Des., 130(10), p. 101101.
de Silva, R. , Bischel, D. T. , Lee, W. , Peterson, E. J. , Calfee, R. C. , and Stahovich, T. F. , 2007, “ Kirchhoff's Pen: A Pen-Based Circuit Analysis Tutor,” 4th Eurographics Workshop on Sketch-Based interfaces and Modeling, ACM, New York, pp. 75–82.
Hutchinson, T. C. , Kuester, F. , and Phair, M. E. , 2007, “ Sketching Finite-Element Models Within a Unified Two-Dimensional Framework,” J. Comput. Civ. Eng., 21(3), pp. 175–186.
Fonseca, M. J. , and Jorge, J. A. , 2001, “ Experimental Evaluation of an On-Line Scribble Recognizer,” Pattern Recognit. Lett., 22(12), pp. 1311–1319.
Kara, L. B. , and Stahovich, T. F. , 2005, “ An Image-Based, Trainable Symbol Recognizer for Hand-Drawn Sketches,” Comput. Graphics, 29(4), pp. 501–517.
Johnson, G. , Gross, M. , Do, E. Y.-L. , and Hong, J. , 2012, “ Sketch It, Make It: Sketching Precise Drawings for Laser Cutting,” CHI’12 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, pp. 1079–1082.
Wenyin, L. , 2003, “ On-Line Graphics Recognition: State-of-the-Art,” Graphics Recognition. Recent Advances and Perspectives (Lecture Notes in Computer Science, Vol. 3088), Springer-Verlag, Heidelberg, Germany, pp. 291–304.
Chen, K.-Z. , Zhang, X.-W. , Ou, Z.-Y. , and Feng, X.-A. , 2003, “ Recognition of Digital Curves Scanned From Paper Drawings Using Genetic Algorithms,” Pattern Recognit., 36(1), pp. 123–130.
Murugappan, S. , Sellamani, S. , and Ramani, K. , 2009, “ Towards Beautification of Freehand Sketches Using Suggestions,” 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling, ACM, New York, pp. 69–76.
LEDAS, 2009, “ LGS 2D,” LEDAS Ltd., Novosibirsk, Russia, Last accessed Feb. 25, 2009,
Bouma, W. , Fudos, I. , Hoffmann, C. , Cai, J. , and Paige, R. , 1995, “ Geometric Constraint Solver,” Comput.-Aided Des., 27(6), pp. 487–501.
Swigart, S. , 2005, “ Easily Write Custom Gesture Recognizers for Your Tablet PC Applications,” Tablet PC Technical Article, Swigart Consulting, Portland, OR.
View article in PDF format.

## References

Ullman, D. G. , Wood, S. , and Craig, D. , 1990, “ The Importance of Drawing in the Mechanical Design Process,” Comput. Graphics, 14(2), pp. 263–274.
Yang, M. C. , and Cham, J. G. , 2007, “ An Analysis of Sketching Skill and Its Role in Early Stage Engineering Design,” ASME J. Mech. Des., 129(5), pp. 476–482.
Shpitalni, M. , and Lipson, H. , 1997, “ Classification of Sketch Strokes and Corner Detection Using Conic Sections and Adaptive Clustering,” ASME J. Mech. Des., 119(1), pp. 131–135.
Yang, M. , 2009, “ Observations on Concept Generation and Sketching in Engineering Design,” Res. Eng. Des., 20(1), pp. 1–11.
Igarashi, T. , Kawachiya, S. , Tanaka, H. , and Matsuoka, S. , 1998, “ Pegasus: A Drawing System for Rapid Geometric Design,” CHI’98 ACM Conference on Human Factors and Computing Systems, Los Angeles, CA, Apr. 18–23, ACM, New York, pp. 24–25.
Plimmer, B. , and Grundy, J. , 2005, “ Beautifying Sketching-Based Design Tool Content: Issues and Experiences,” Sixth Australasian Conference on User interface, AUIC '05, Australian Computer Society, Inc., Darlinghurst, Australia, pp. 31–38.
Paulson, B. , and Hammond, T. , 2008, “ PaleoSketch: Accurate Primitive Sketch Recognition and Beautification,” 13th International Conference on Intelligent User Interfaces, IUI’08, ACM, New York, pp. 1–10.
Pu, J. , and Ramani, K. , 2007, “ Implicit Geometric Constraint Detection in Freehand Sketches Using Relative Shape Histogram,” 4th Eurographics Workshop on Sketch-Based Interfaces and Modeling, SBIM’07, ACM, New York, pp. 107–113.
Veselova, O. , and Davis, R. , 2006, “ Perceptually Based Learning of Shape Descriptions for Sketch Recognition,” SIGGRAPH’06 Special Interest Group on Computer Graphics and Interactive Techniques Conference, ACM, New York, p. 28.
Murugappan, S. , and Ramani, K. , 2009, “ FEAsy: A Sketch-Based Interface Integrating Structural Analysis in Early Design,” ASME Paper No. DETC2009-87727.
Forbus, K. D. , Lockwood, K. , Klenk, M. , Tomai, E. , and Usher, J. , 2004, “ Open-Domain Sketch Understanding: The nuSketch Approach,” AAAI Fall Symposium on Making Pen-Based Interaction Intelligent and Natural, Washington, DC, Oct. 21–24, AAAI Press, Menlo Park, CA, pp. 58–63.
Landay, J. A. , and Myers, B. A. , 2001, “ Sketching Interfaces: Toward More Human Interface Design,” Computer, 34(3), pp. 56–64.
Lin, J. , Newman, M. W. , Hong, J. I. , and Landay, J. A. , 2001, “ DENIM: An Informal Tool for Early Stage Web Site Design,” CHI’01 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, pp. 205–206.
McCrae, J. , and Singh, K. , 2011, “ Neatening Sketched Strokes Using Piecewise French Curves,” Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling, ACM, New York, pp. 141–148.
McCrae, J. , and Singh, K. , 2009, “ Sketching Piecewise Clothoid Curves,” Comput. Graphics, 33(4), pp. 452–461.
Sezgin, T. M. , Stahovich, T. , and Davis, R. , 2001, “ Sketch Based Interfaces: Early Processing for Sketch Understanding,” Workshop on Perceptive User Interfaces, PUI’01, ACM, New York, pp. 1–8.
Calhoun, C. , Stahovich, T. F. , Kurtoglu, T. , and Kara, L. B. , 2002, “ Recognizing Multi-Stroke Symbols,” AAAI Spring Symposium—Sketch Understanding, Palo Alto, CA, Mar. 25–27, AAAI Press, Menlo Park, CA, pp. 15–23.
Thiel, Y. , Singh, K. , and Balakrishnan, R. , 2011, “ Elasticurves: Exploiting Stroke Dynamics and Inertia for the Real-Time Neatening of Sketched 2D Curves,” 24th Annual ACM Symposium on User Interface Software and Technology, ACM, New York, pp. 383–392.
Kim, D. H. , and Kim, M.-J. , 2006, “ A Curvature Estimation for Pen Input Segmentation in Sketch-Based Modeling,” Comput.-Aided Des., 38(3), pp. 238–248.
Wolin, A. , Eoff, B. D. , and Hammond, T. A. , 2008, “ ShortStraw: A Simple and Effective Corner Finder for Polylines,” Eurographics 2008—Sketch-Based Interfaces and Modeling (SBIM), pp. 33–40.
Hse, H. , Shilman, M. , and Newton, A. R. , 2004, “ Robust Sketched Symbol Fragmentation Using Templates,” 9th International Conference on Intelligent User Interfaces, IUI’04, ACM, New York, pp. 156–160.
Gennari, L. , Kara, L. B. , Stahovich, T. F. , and Shimada, K. , 2005, “ Combining Geometry and Domain Knowledge to Interpret Hand-Drawn Diagrams,” Comput. Graphics, 29(4), pp. 547–562.
Zhang, X. , Song, J. , Dai, G. , and Lyu, M. , 2006, “ Extraction of Line Segments and Circular Arcs From Freehand Strokes Based on Segmental Homogeneity Features,” IEEE Trans. Syst., Man, Cybern., Part B, 36(2), pp. 300–311.
Xiong, Y. , and LaViola, J. J., Jr. , 2010, “ A Shortstraw-Based Algorithm for Corner Finding in Sketch-Based Interfaces,” Comput. Graphics, 34(5), pp. 513–527.
Taele, P. , and Hammond, T. , 2014, “ Developing Sketch Recognition and Interaction Techniques for Intelligent Surfaceless Sketching User Interfaces,” 19th International Conference on Intelligent User Interfaces, ACM, New York, pp. 53–56.
Babu, S. S. S. , Jaiswal, P. , Esfahani, E. T. , and Rai, R. , 2014, “ Sketching in Air: A Single Stroke Classification Framework,” ASME Paper No. DETC2014-34065.
Wang, C. , 2007, “ Drawing on Air: Input Techniques for Controlled 3D Line Illustration,” IEEE Trans. Visualization Comput. Graphics, 13(5), pp. 1067–1081.
Eggli, L. , ChingYao, H. , Bruderlin, B. , and Elber, G. , 1997, “ Inferring 3D Models From Freehand Sketches and Constraints,” Comput.-Aided Des., 29(2), pp. 101–112.
Zeleznik, R. C. , Herndon, K. P. , and Hughes, J. F. , 1996, “ SKETCH: An Interface for Sketching 3D Scenes,” 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH’96, ACM, New York, pp. 163–170.
Arisoy, E. B. , Orbay, G. , and Kara, L. B. , 2014, “ Predictive Modeling for 2D Form Design,” Computational Modeling of Objects Presented in Images. Fundamentals, Methods, and Applications, Springer, Heidelberg, Germany, pp. 286–291.
Damm, C. H. , Hansen, K. M. , and Thomsen, M. , 2000, “ Tool Support for Cooperative Object-Oriented Design: Gesture Based Modelling on an Electronic Whiteboard,” SIGCHI Conference on Human Factors in Computing Systems, CHI’00, ACM, New York, pp. 518–525.
Plimmer, B. , and Apperley, M. , 2004, “ Interacting With Sketched Interface Designs: An Evaluation Study,” CHI’04 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, pp. 1337–1340.
Lee, Y. J. , Zitnick, C. L. , and Cohen, M. F. , 2011, “ ShadowDraw: Real-Time User Guidance for Freehand Drawing,” ACM Trans. Graphics, 30, p. 27.
Benjamin, W. , Chandrasegaran, S. , Ramanujan, D. , Elmqvist, N. , Vishwanathan, S. , and Ramani, K. , 2014, “ Juxtapoze: Supporting Serendipity and Creative Expression in Clipart Compositions,” SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, pp. 341–350.
Gross, M. D. , and Do, E. Y.-L. , 1996, “ Ambiguous Intentions: A Paper-Like Interface for Creative Design,” 9th Annual ACM Symposium on User Interface Software and Technology, UIST’96, ACM, New York, pp. 183–192.
Landay, J. A. , and Myers, B. A. , 1995, “ Interactive Sketching for the Early Stages of User Interface Design,” SIGCHI Conference on Human Factors in Computing Systems, CHI’95, ACM, New York, pp. 43–50.
Bae, S.-H. , Balakrishnan, R. , and Singh, K. , 2008, “ ILoveSketch: As-Natural-As-Possible Sketching System for Creating 3D Curve Models,” 21st Annual ACM Symposium on User Interface Software and Technology, ACM, New York, pp. 151–160.
Schmidt, R. , Khan, A. , Singh, K. , and Kurtenbach, G. , 2009, “ Analytic Drawing of 3D Scaffolds,” ACM Trans. Graphics, 28, p. 149.
Naya, F. , Contero, M. , Aleixos, N. , and Company, P. , 2007, “ ParSketch: A Sketch-Based Interface for a 2D Parametric Geometry Editor,” Human-Computer Interaction. Interaction Platforms and Techniques (Lecture Notes in Computer Science, Vol. 4551), J. A. Jacko , ed., Springer, Heidelberg, Germany, pp. 115–124.
LaViola, J. J., Jr. , and Zeleznik, R. C. , 2006, “ MathPad2: A System for the Creation and Exploration of Mathematical Sketches,” SIGGRAPH’06 Special Interest Group on Computer Graphics and Interactive Techniques Conference, ACM, New York, p. 33.
Kara, L. B. , Gennari, L. , and Stahovich, T. F. , 2008, “ A Sketch-Based Tool for Analyzing Vibratory Mechanical Systems,” ASME J. Mech. Des., 130(10), p. 101101.
de Silva, R. , Bischel, D. T. , Lee, W. , Peterson, E. J. , Calfee, R. C. , and Stahovich, T. F. , 2007, “ Kirchhoff's Pen: A Pen-Based Circuit Analysis Tutor,” 4th Eurographics Workshop on Sketch-Based interfaces and Modeling, ACM, New York, pp. 75–82.
Hutchinson, T. C. , Kuester, F. , and Phair, M. E. , 2007, “ Sketching Finite-Element Models Within a Unified Two-Dimensional Framework,” J. Comput. Civ. Eng., 21(3), pp. 175–186.
Fonseca, M. J. , and Jorge, J. A. , 2001, “ Experimental Evaluation of an On-Line Scribble Recognizer,” Pattern Recognit. Lett., 22(12), pp. 1311–1319.
Kara, L. B. , and Stahovich, T. F. , 2005, “ An Image-Based, Trainable Symbol Recognizer for Hand-Drawn Sketches,” Comput. Graphics, 29(4), pp. 501–517.
Johnson, G. , Gross, M. , Do, E. Y.-L. , and Hong, J. , 2012, “ Sketch It, Make It: Sketching Precise Drawings for Laser Cutting,” CHI’12 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, pp. 1079–1082.
Wenyin, L. , 2003, “ On-Line Graphics Recognition: State-of-the-Art,” Graphics Recognition. Recent Advances and Perspectives (Lecture Notes in Computer Science, Vol. 3088), Springer-Verlag, Heidelberg, Germany, pp. 291–304.
Chen, K.-Z. , Zhang, X.-W. , Ou, Z.-Y. , and Feng, X.-A. , 2003, “ Recognition of Digital Curves Scanned From Paper Drawings Using Genetic Algorithms,” Pattern Recognit., 36(1), pp. 123–130.
Murugappan, S. , Sellamani, S. , and Ramani, K. , 2009, “ Towards Beautification of Freehand Sketches Using Suggestions,” 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling, ACM, New York, pp. 69–76.
LEDAS, 2009, “ LGS 2D,” LEDAS Ltd., Novosibirsk, Russia, Last accessed Feb. 25, 2009,
Bouma, W. , Fudos, I. , Hoffmann, C. , Cai, J. , and Paige, R. , 1995, “ Geometric Constraint Solver,” Comput.-Aided Des., 27(6), pp. 487–501.
Swigart, S. , 2005, “ Easily Write Custom Gesture Recognizers for Your Tablet PC Applications,” Tablet PC Technical Article, Swigart Consulting, Portland, OR.

## Figures

Fig. 4

Beautification of a freehand stroke: (a) freehand stroke, (b) raw points, (c) resampled points, (d) critical points, (e) primitives, (f) primitives, and (g) beautified sketch

Fig. 3

Step-by-step process of analyzing a bracket: (a) user drawn freehand stroke in geometry mode, (b) beautified output, (c) a freehand stroke representing a circle is added to the sketch, (d) result after beautification, (e) symbols input in symbol mode, (f) final sketch after sketch interpretation, (g) finite element integration dialog, (h) generated ansys specific commands, and (i) deformation results in ansys

Fig. 2

The system pipeline with two modes of input—geometry and symbol

Fig. 1

The FEAsy interface showing (a) a hand-drawn sketch of an example 2D bracket and (b) the deformation results in ansys

Fig. 5

Chord angle computation. The circles represent the resampled points. θ represents the chord angle computed for the resampled point using a window size of 3.

Fig. 6

Recognition: (a) shows the freehand stroke with critical points. (b) Shows the results of least-squares fitting. There are discontinuities between adjacent segments. (c) Shows the results of our algorithm.

Fig. 7

Least-squares arc fitting

Fig. 11

Problems used in user study

Fig. 9

The list of finite element symbols recognized in our system

Fig. 10

Symbol recognition and sketch interpretation. (a) A beautified sketch at the end of geometry mode. (b) Newly added strokes represent the dimensions, loading, and boundary conditions drawn in symbol mode. (c) Clustering of strokes into stroke groups represented by dashed bounding boxes. (d) Classification of stroke groups into text and symbols. (e) Final sketch after sketch interpretation.

Fig. 8

Implicit geometric constraints inferred in our system for beautification

Fig. 12

Sample sketches from participants for the four problems. From left to right, each sub image inset in (a)–(d) shows the freehand sketch drawn in geometry mode, the beautified sketch, sketch with input symbols and text drawn in symbol mode, sketch after text and symbol recognition, and context interpretation, in that order.

Fig. 13

Results of user study—beautification

Fig. 14

Results of user study—symbol and text recognition. (Legend: CR—total number of correctly recognized symbols and T—total number of symbols.)

Fig. 15

Results of user study—context interpretation. (Legend: CI—total number of correctly interpreted contexts and T—total number of contexts.)

## Discussions

Some tools below are only available to our subscribers or users with an online account.

### Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related Proceedings Articles
Related eBook Content
Topic Collections