Sharad Oberoi, Susan Finger and Eric Rosé

Carnegie Mellon University

Online Implementation of the Delta Design Game for analyzing Collaborative Team

Practices

DETC2013-13319 2

Collaborative design

• No single actor in complex design tasks has all the knowledge needed to realize the project goal.

• Collaborative design requires individual design expertise as well as collaboration skills.

• Team members have to ensure that the relevant knowledge is shared and integrated effectively.

• To achieve this outcome, team members need to have skills for facilitating the integration of specialized knowledge across boundaries.

DETC2013-13319 3

Delta Design Game

• Created by MIT Professor Louis Bucciarelli in the 1980s

• Aim:– To engage undergraduate engineering students in a team

design project requiring communication and negotiation

• Interdependence of the objectives of the roles requires the players to reveal their constraints in order to achieve a satisfactory outcome

DETC2013-13319 4

Delta Design Game: Design Objectives

• Planet DeltaP is a planned flat world with the design done in a diagonal 2-D space.

• The team designs the residence for the citizens of DeltaP using red and blue deltas shaped like equilateral triangles. – Red deltas provide heat– Blue deltas have a cooling effect

• Each team is asked to design a house that meets all the structural, thermal and aesthetic constraints.

DETC2013-13319 5

Delta Design Collaboration

• Actors– Architect– Project manager,– Thermal engineer – Structural engineer

• DeltaP’s unique design environment and its skewed coordinate system levels the playing field for all players.

DETC2013-13319 6

Delta Design Collaboration

• Each player only knows the rules for his/her own role.

• The rules are designed so that certain roles can easily form alliances, while others have conflicting objectives.

• Unless the team realizes that they need to see the big picture and compromise, they cannot be successful in the game.

DETC2013-13319 7

Delta Design Game: Board version

• Computations required to evaluate the objective function for each role are lengthy.

• Metrics need to be recomputed manually each time a board tile is moved.

• Even successful teams barely manage to achieve most of their design objectives in the allotted time.

• Lasts between 1-1.5 hours• Key attributes of collaboration and the negotiations

within the team cannot be captured adequately.

→Researchers cannot analyze collaborative team practices during the game.

DETC2013-13319 8

Delta Design Game: Online version

• Computations required to evaluate the objective function for each role are done automatically in the background.

• Metrics recomputed automatically each time a board tile is moved.

• Most teams manage to achieve their design objectives in the allotted time.

• Lasts between 1-1.5 hours• Key attributes of collaboration and the negotiations

within the team are captured through text chat and board moves.

→Researchers can analyze collaborative team practices!

DETC2013-13319 9

Delta Design Application

DETC2013-13319 10

Delta Design Application

• Rules of the board-based Delta Design game have been adapted as functional requirements for the online game.

• Every action taken on the shared virtual board and all text chat between participants are recorded.

• The players can roll back the canvas to a prior configuration if they realize that they have made a wrong move → allows the players to consider various strategies to maximize their scores.

• Both the automatic computation of objective functions and the ability to roll back moves change the nature of the collaboration from the physical game play.

DETC2013-13319 11

Delta Design Application Constraints

• Only one person is allowed to work on the board at a time.

• In order to move a tile the player needs to lock the board first (up to 3 minutes).

• When a player has locked the board, the other team members cannot talk to him in person, but can send comments and requests in the chat window.

• In order to win, everyone on a team must meet his/her goals.

DETC2013-13319 12

System Architecture

• The Delta Design game is a network application.

• Four players located on separate computers interact with each other by way of a shared game surface, a shared drawing surface, and a chat application.

• A central server maintains the current state of all running games.

• Each player who logs into one of these games has a pre-assigned role.

DETC2013-13319 13

System Architecture

DETC2013-13319 14

Event Model of Delta Design

• Every interaction with the simulation involves sending an event to the server and the receipt of a response which may contain an arbitrary amount of data.

DETC2013-13319 15

Delta Design Game Implementation in Classroom

• Authors conducted a series of user studies with this online implementation over a period of 4 years

• Data collected from 38 four-player teams (mostly undergraduate capstone design classes)

• Provides a rich resource to explore different aspects of collaborative team practices.

DETC2013-13319 16

Evaluation Criteria

• Metrics used to evaluate success of a collaborative strategy:– Did the teams strategize before the game?– Did the players share their constraints before the game?– How long did the teams take for getting all green lights (or their

highest score during the game)?

• Proposed hypothesis: – The teams that are more communicative during the Delta Design

game can better negotiate among themselves and hence should be more successful.

DETC2013-13319 17

Results

• Out of 38 teams analyzed, only 12 achieved a score of 3 or more: – 3 green + 1 yellow light– All 4 green lights

DETC2013-13319 18

Results

• Among the 12 teams (with score >=3):– 50% teams devised a strategy for moving the tiles on the canvas

and discussed beforehand what each person’s goals were– 16% teams neither strategized nor discussed their goals

beforehand.

• Whether or not the players strategized before the beginning of the game does not have any marked effect on the game’s outcome.

• Whether or not the players were transparent about their role’s constraints and functional requirements does not have any marked effect on the game’s outcome.

DETC2013-13319 19

Results

• Teams which strategize about their game-plan are also better correlated to share their constraints with each other.

• However, this does not seem to make them any more successful.

DETC2013-13319 20

Conclusions

• The complex rules of Delta Design make it a suitable candidate for analyzing collaboration strategies in team-based design projects.

• More transparent negotiation tactics and board-moving strategies, do not improve the teams’ chances of success.

• The proposed hypothesis has been proved false.– The teams that are more communicative during the Delta Design

game are not found to be significantly more successful.

DETC2013-13319 21

Questions?

DETC2013-13319 22

Acknowledgments

• This research was supported in part by National Science Foundation grant 0935127.

• We would like to thank:– Prof. Louis Bucciarelli, MIT– Prof. Daniel P. Siewiorek, CMU– Prof. Asim Smailagic, CMU– Prof. Carolyn P. Rosé, CMU– Peter A. Simon, CMU– Gahgene Gweon, CMU

DETC2013-13319 23

Delta Design Application: Lights

• Every player has met his goal when all the stop lights on the tabs are green.

• All players can see the lights for the other players, but not their specific calculations.

• A player within 10% of the goal for any one of his conditions who has met all the other conditions is assigned a yellow light.

• This allows other members of the team to see how their moves on the canvas affect the performance of the team as a whole.

DETC2013-13319 24

Coordinate System

• Virtual board is composed of diamond shaped units laid out on a coordinate system on a 150° angle.

• This can be seen as a grid composed of triangles called “quads,” each of which bisect one of the diamonds.

• Each delta consists of four smaller quads.• The “origin” or center of a delta is considered to be the

middle quad of the four • Each delta can be represented using three components: x-

coordinate, y-coordinate, and orientation.• The placement of deltas on this grid follows certain rules

which vary with the orientation.

DETC2013-13319 25

Coordinate System

DETC2013-13319 26

MySQL Database

• The database stores the following:– The details of the game (start time, duration, agents, playing

surface state)– A log of all of the events which take place to modify the playing

board– A log of all of the chat communication between the players– A snapshot of the state of the game after every action which

modifies the board– A record of any events affecting the Delta Design server