Research Papers

Modeling Participation Behaviors in Design Crowdsourcing Using a Bipartite Network-Based Approach

[+] Author and Article Information
Zhenghui Sha

Department of Mechanical Engineering,
University of Arkansas,
Fayetteville, AR 72701
e-mail: zsha@uark.edu

Ashish M. Chaudhari, Jitesh H. Panchal

School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907

1Corresponding author.

Contributed by the Computers and Information Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received September 14, 2018; final manuscript received January 17, 2019; published online March 21, 2019. Assoc. Editor: Yan Wang.

J. Comput. Inf. Sci. Eng 19(3), 031010 (Mar 21, 2019) (10 pages) Paper No: JCISE-18-1246; doi: 10.1115/1.4042639 History: Received September 14, 2018; Revised January 17, 2019

This paper analyzes participation behaviors in design crowdsourcing by modeling interactions between participants and design contests as a bipartite network. Such a network consists of two types of nodes, participant nodes and design contest nodes, and the links indicating participation decisions. The exponential random graph models (ERGMs) are utilized to test the interdependence between participants' decisions. ERGMs enable the utilization of different network configurations (e.g., stars and triangles) to characterize different forms of dependencies and to identify the factors that influence the link formation. A case study of an online design crowdsourcing platform is carried out. Our results indicate that designer, contest, incentive, and factors of dependent relations have significant effects on participation in online contests. The results reveal some unique features about the effects of incentives, e.g., the fraction of total prize allocated to the first prize negatively influences participation. Further, we observe that the contest popularity modeled by the alternating k-star network statistic has a significant influence on participation, whereas associations between participants modeled by the alternating two-path network statistic do not. These insights are useful to system designers for initiating effective crowdsourcing mechanisms to support product design and development. The approach is validated by applying the estimated ERGMs to predict participants' decisions and comparing with their actual decisions.

Copyright © 2019 by ASME
Topics: Design , Modeling
Your Session has timed out. Please sign back in to continue.


Taylor, C. , 1995, “ Digging for Golden Carrots: An Analysis of Research Tournaments,” Am. Econ. Rev., 85(4), pp. 872–890. https://www.jstor.org/stable/2118237
Che, Y.-K. , and Gale, I. , 2003, “ Optimal Design of Research Contests,” Am. Econ. Rev., 93(3), pp. 646–671. [CrossRef]
Sisak, D. , 2009, “ Multiple-Prize Contests—The Optimal Allocation of Prizes,” J. Econ. Surv., 23(1), pp. 82–114. [CrossRef]
Sheremata, R. M. , 2011, “ Contest Design: An Experimental Investigation,” Econ. Inquiry, 49(2), pp. 573–590. [CrossRef]
Sha, Z. , Kannan, K. N. , and Panchal, J. H. , 2015, “ Behavioral Experimentation and Game Theory in Engineering Systems Design,” ASME J. Mech. Des., 137(5), p. 051405. [CrossRef]
Chaudhari, A. , Thekinen, J. , and Panchal, J. , 2016, “ Using Contests for Engineering Systems Design: A Study of Auctions and Fixed-Prize Tournaments,” DS 84: DESIGN 2016: 14th International Design Conference, Durbovnik, Croatia, May 16–19, pp. 947–956. https://www.designsociety.org/publication/38905/USING+CONTESTS+FOR+ENGINEERING+SYSTEMS+DESIGN%3A+A+STUDY+OF+AUCTIONS+AND+FIXED-PRIZE+TOURNAMENTS
Panchal, J. H. , Sha, Z. , and Kannan, K. N. , 2017, “ Understanding Design Decisions Under Competition Using Games With Information Acquisition and a Behavioral Experiment,” ASME J. Mech. Des., 139(9), p. 091402. [CrossRef]
Chaudhari, A. M. , Sha, Z. , and Panchal, J. H. , 2018, “ Analyzing Participant Behaviors in Design Crowdsourcing Contests Using Causal Inference on Field Data,” ASME J. Mech. Des., 140(9), p. 091401. [CrossRef]
GrabCAD.com, 2018, “ Engineering & Design Challenges,” GrabCAD Inc., Cambridge, MA, accessed May 11, 2018, https://www.grabcad.com/challenges
Amazon Mechanical Turk, 2018, “ Human Intelligence Through an API,” Amazon Mechanical Turk Inc., Seattle, WA, accessed May 11, 2018, https://www.mturk.com
Panchal, J. , 2015, “ Using Crowds in Engineering Design—Towards a Holistic Framework,” International Conference on Engineering Design, Design Society (ICED), Milan, Italy, July 27–30, pp. 27–30. https://engineering.purdue.edu/DELP/sites/default/files/Panchal_ICED_2015_0.pdf
Simula, H. , and Ahola, T. , 2014, “ A Network Perspective on Idea and Innovation Crowdsourcing in Industrial Firms,” Ind. Mark. Manage., 43(3), pp. 400–408. [CrossRef]
Szymanski, S. , and Valletti, T. M. , 2005, “ Incentive Effects of Second Prizes,” Eur. J. Political Econ., 21(2), pp. 467–481. [CrossRef]
Clark, D. J. , and Riis, C. , 1998, “ Influence and the Discretionary Allocation of Several Prizes,” Eur. J. Political Econ., 14(4), pp. 605–625. [CrossRef]
Fullerton, R. , Lincster, B. , McKee, M. , and Slate, S. , 2002, “ Using Auctions to Reward Tournament Winners: Theory and Experimental Investigation,” RAND J. Econ., 33(1), pp. 62–84. [CrossRef]
Schottner, A. , 2008, “ Fixed-Prize Tournaments Versus First-Price Auctions in Innovation Contests,” Econ. Theory, 35(1), pp. 55–71. [CrossRef]
Zhao, Y. C. , and Zhu, Q. , 2014, “ Effects of Extrinsic and Intrinsic Motivation on Participation in Crowdsourcing Contest,” Online Inf. Rev., 38(7), p. 896. [CrossRef]
Zheng, H. , Li, D. , and Hou, W. , 2011, “ Task Design, Motivation, and Participation in Crowdsourcing Contests,” Int. J. Electron. Commerce, 15(4), pp. 57–88. [CrossRef]
Roberts, J. A. , Hann, I.-H. , and Slaughter, S. A. , 2006, “ Understanding the Motivations, Participation, and Performance of Open Source Software Developers: A Longitudinal Study of the Apache Projects,” Manage. Sci., 52(7), pp. 984–999. [CrossRef]
Füller, J. , 2010, “ Refining Virtual Co-Creation From a Consumer Perspective,” California Manage. Rev., 52(2), pp. 98–122. [CrossRef]
Leimeister, J. M. , Huber, M. , Bretschneider, U. , and Krcmar, H. , 2009, “ Leveraging Crowdsourcing: Activation-Supporting Components for It-Based Ideas Competition,” J. Manage. Inf. Syst., 26(1), pp. 197–224. [CrossRef]
Ke, W. , and Zhang, P. , 2009, “ Motivations in Open Source Software Communities: The Mediating Role of Effort Intensity and Goal Commitment,” Int. J. Electron. Commerce, 13(4), pp. 39–66. [CrossRef]
Rogstadius, J. , Kostakos, V. , Kittur, A. , Smus, B. , Laredo, J. , and Vukovic, M. , 2011, “ An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets,” ICWSM, 11, pp. 17–21. https://www.aaai.org/ocs/index.php/ICWSM/ICWSM11/paper/viewPaper/2778
Archak, N. , 2010, “ Money, Glory and Cheap Talk: Analyzing Strategic Behavior of Contestants in Simultaneous Crowdsourcing Contests on TopCoder.com,” 19th International Conference on World Wide Web, Raleigh, NC, Apr. 26–30, pp. 21–30.
Aker, A. , El-Haj, M. , Albakour, M.-D. , and Kruschwitz, U. , 2012, “ Assessing Crowdsourcing Quality Through Objective Tasks,” Language Resources and Evaluation (LREC), Istambul, Turkey, May 23–27, pp. 1456–1461.
Li, K. , Xiao, J. , Wang, Y. , and Wang, Q. , 2013, “ Analysis of the Key Factors for Software Quality in Crowdsourcing Development: An Empirical Study on TopCoder.com,” IEEE 37th Annual Computer Software and Applications Conference, Kyoto, Japan, July 22–26, pp. 812–817.
Kim, D. J. , Ferrin, D. L. , and Rao, H. R. , 2008, “ A Trust-Based Consumer Decision-Making Model in Electronic Commerce: The Role of Trust, Perceived Risk, and Their Antecedents,” Decis. Support Syst., 44(2), pp. 544–564. [CrossRef]
Zheng, H. , Xie, Z. , Hou, W. , and Li, D. , 2014, “ Antecedents of Solution Quality in Crowdsourcing: The Sponsor's Perspective,” J. Electron. Commerce Res., 15(3), p. 212. http://ojs.jecr.org/jecr/sites/default/files/Paper6.pdf
Algesheimer, R. , Borle, S. , Dholakia, U. M. , and Singh, S. S. , 2010, “ The Impact of Customer Community Participation on Customer Behaviors: An Empirical Investigation,” Mark. Sci., 29(4), pp. 756–769. [CrossRef]
Kazai, G. , 2011, “ In Search of Quality in Crowdsourcing for Search Engine Evaluation,” European Conference on Information Retrieval (ECIR), Dublin, Ireland, Apr. 18–21, pp. 165–176. https://dl.acm.org/citation.cfm?id=1996911
Jeppesen, L. B. , and Lakhani, K. R. , 2010, “ Marginality and Problem-Solving Effectiveness in Broadcast Search,” Org. Sci., 21(5), pp. 1016–1033. [CrossRef]
Burnap, A. , Ren, Y. , Gerth, R. , Papazoglou, G. , Gonzalez, R. , and Papalambros, P. Y. , 2015, “ When Crowdsourcing Fails: A Study of Expertise on Crowdsourced Design Evaluation,” ASME J. Mech. Des., 137(3), p. 031101. [CrossRef]
Valente, T. W. , Fujimoto, K. , Chou, C.-P. , and Spruijt-Metz, D. , 2009, “ Adolescent Affiliations and Adiposity: A Social Network Analysis of Friendships and Obesity,” J. Adolesc. Health, 45(2), pp. 202–204. [CrossRef] [PubMed]
Lusher, D. , Robins, G. , and Kremer, P. , 2010, “ The Application of Social Network Analysis to Team Sports,” Meas. Phys. Educ. Exercise Sci., 14(4), pp. 211–224. [CrossRef]
Contractor, N. S. , DeChurch, L. A. , Carson, J. , Carter, D. R. , and Keegan, B. , 2012, “ The Topology of Collective Leadership,” Leadership Q., 23(6), pp. 994–1011. [CrossRef]
Holland, P. W. , and Leinhardt, S. , 1981, “ An Exponential Family of Probability Distributions for Directed Graphs,” J. Am. Stat. Assoc., 76(373), pp. 33–50. [CrossRef]
Frank, O. , and Strauss, D. , 1986, “ Markov Graphs,” J. Am. Stat. Assoc., 81(395), pp. 832–842. [CrossRef]
Pattison, P. , and Robins, G. , 2002, “ Neighborhood–Based Models for Social Networks,” Sociol. Methodol., 32(1), pp. 301–337. [CrossRef]
Snijders, T. A. , Pattison, P. E. , Robins, G. L. , and Handcock, M. S. , 2006, “ New Specifications for Exponential Random Graph Models,” Sociol. Methodol., 36(1), pp. 99–153. [CrossRef]
Wang, P. , Pattison, P. , and Robins, G. , 2013, “ Exponential Random Graph Model Specifications for Bipartite Networks—A Dependence Hierarchy,” Soc. Networks, 35(2), pp. 211–222. [CrossRef]
Wang, P. , Robins, G. , Pattison, P. , and Lazega, E. , 2013, “ Exponential Random Graph Models for Multilevel Networks,” Soc. Networks, 35(1), pp. 96–115. [CrossRef]
Wang, M. , Chen, W. , Huang, Y. , Contractor, N. S. , and Fu, Y. , 2016, “ Modeling Customer Preferences Using Multidimensional Network Analysis in Engineering Design,” Des. Sci., 2, p. e11. [CrossRef]
Wang, M. , Sha, Z. , Huang, Y. , Contractor, N. , Fu, Y. , and Chen, W. , 2018, “ Predicting Product Co-Consideration and Market Competitions for Technology-Driven Product Design: Anetwork-Based Approach,” Des. Sci., 4, p. e9. [CrossRef]
Sha, Z. , Huang, Y. , Fu, J. S. , Wang, M. , Fu, Y. , Contractor, N. S. , and Chen, W. , 2018, “ A Network-Based Approach to Modeling and Predicting Product Coconsideration Relations,” Complexity, 2018, p. 2753638. [CrossRef]
Fu, J. S. , Sha, Z. , Huang, Y. , Wang, M. , Fu, Y. , and Chen, W. , 2017, “ Two-Stage Modeling of Customer Choice Preferences in Engineering Design Using Bipartite Network Analysis,” ASME Paper No. DETC2017-68099.
Handcock, M. S. , Hunter, D. R. , Butts, C. T. , Goodreau, S. M. , and Morris, M. , 2008, “ Statnet: Software Tools for the Representation, Visualization, Analysis and Simulation of Network Data,” J. Stat. Software, 24(1), p. 1548. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2447931/
Snijders, T. A. B. , 2002, “ Markov Chain Monte Carlo Estimation of Exponential Random Graph Models,” J. Soc. Struct., 3(2), pp. 1–40. https://www.cmu.edu/joss/content/articles/volume3/Snijders.pdf
Hunter, D. R. , 2007, “ Curved Exponential Family Models for Social Networks,” Soc. Networks, 29(2), pp. 216–230. [CrossRef] [PubMed]
Scrapy.org, 2018, “ A Fast and Powerful Scraping and Web Crawling Framework,” Scrapinghub, Ballincollig, Ireland, accessed May 11, 2018, https://www.scrapy.org
Albert, R. , and Barabási, A.-L. , 2002, “ Statistical Mechanics of Complex Networks,” Rev. Modern Phys., 74(1), p. 47. [CrossRef]
Autodesk Inc., 2013, “ Autodesk Robot Gripper Arm Design Challenge,” GrabCAD.com, Cambridge, MA, accessed May 11, 2018, https://grabcad.com/challenges/autodesk-robot-gripper-arm-design-challenge
Rabaconda, 2012, “ Dirtbike Tire Changing Tool Challenge,” GrabCAD.com, Cambridge, MA, accessed May 11, 2018, https://grabcad.com/challenges/dirtbike-tire-changing-tool-challenge
Terwiesch, C. , and Xu, Y. , 2008, “ Innovation Contests, Open Innovation, and Multiagent Problem Solving,” Manage. Sci., 54(9), pp. 1529–1543. [CrossRef]
Alcoa corp., 2016, “ Airplane Bearing Bracket Challenge,” GrabCAD.com, Cambridge, MA, accessed May 11, 2018, https://grabcad.com/challenges/airplane-bearing-bracket-challenge
Autodesk Inc., 2013, “ Robot Gripper Test Object Challenge,” GrabCAD.com, Cambridge, MA, accessed May 11, 2018, https://grabcad.com/challenges/robot-gripper-test-object-challenge
Lazear, E. P. , and Rosen, S. , 1981, “ Rank-Order Tournaments as Optimum Labor Contracts,” J. Political Econ., 89(5), pp. 841–864. [CrossRef]
Bloom, M. , and Michel, J. G. , 2002, “ The Relationships Among Organizational Context, Pay Dispersion, and Among Managerial Turnover,” Acad. Manage. J., 45(1), pp. 33–42.
Harrison, D. A. , and Klein, K. J. , 2007, “ What's the Difference? Diversity Constructs as Separation, Variety, or Disparity in Organizations,” Acad. Manage. Rev., 32(4), pp. 1199–1228. [CrossRef]
Hvide, H. K. , 2002, “ Tournament Rewards and Risk Taking,” J. Labor Econ., 20(4), pp. 877–898. [CrossRef]
Barabási, A.-L. , and Albert, R. , 1999, “ Emergence of Scaling in Random Networks,” Science, 286(5439), pp. 509–512. [CrossRef] [PubMed]
Fawcett, T. , 2006, “ An Introduction to Roc Analysis,” Pattern Recognit. Lett., 27(8), pp. 861–874. [CrossRef]
Saito, T. , and Rehmsmeier, M. , 2015, “ The Precision-Recall Plot is More Informative Than the Roc Plot When Evaluating Binary Classifiers on Imbalanced Datasets,” PLoS One, 10(3), p. e0118432.
Sha, Z. , Chaudhari, A. M. , and Panchal, J. H. , 2018, “ Modeling Participation Behaviors in Design Crowdsourcing Using a Bipartite Network-Based Approach,” ASME Paper No. DETC2018-85686.


Grahic Jump Location
Fig. 1

An example of bipartite network representation for multiple number of contests (c1,c2,…cn) and multiple number of participants (p1,p2,…pm). Links joining contest nodes to participate nodes represent respective participation.

Grahic Jump Location
Fig. 2

Two types of network configurations for modeling interdependence of participation decisions

Grahic Jump Location
Fig. 3

A visualization of the GrabCAD bipartite network

Grahic Jump Location
Fig. 4

Degree distribution of design contests and participants



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In