82

Hybrid Fabrication

1 A Scenario of a Hybrid Fabrication freeform pavilion with irregular foam bricks

Chengyu Sun

Zhaohua Zheng

Yuze Wang

Tongyu Sun

Tongji University, College of Architecture and Urban PlanningA Freeform Building Process with High Onsite Flexibility and Ac-

ceptable Accumulative Error

1

ABSTRACTAlthough digital fabrication has a booming development in the building industry, especially

in freeform building, its further application in onsite operations is still limited because of

the huge flexibility required in programming. On the contrary, traditional manual fabrica-

tion onsite deals perfectly with problems that always accompany fatal accumulative errors

in freeform building. This study explores a hybrid fabrication paradigm to take advantage

of both in an onsite freeform building project, in which there is a cycling human–computer

interactive process consisting of manual operation and computer guidance in real time.

A Hololens-Kinect system in a framework of typical project camera systems is used in the

demonstration. When human builders perceive, decide, and operate the irregular foam

bricks in a complex onsite environment, the computer keeps updating the current form

through 3D scanning and prompting the position and orientation of the next brick through

augmented display. From a starting vault, the computer always fine tunes its control

surface according to the gradually installed bricks and keeps following a catenary formula.

Thus, the hybrid fabrication actually benefits from the flexibility based on human judgment

and operation, and an acceptable level of accumulative error can be handled through

computer guidance concerning the structural performance and formal accuracy.

83

4

5

INTRODUCTIONIn the early development of digital fabrication, robotic arms

were used to fabricate components and install them to the

right positions and orientations with much better accuracy

than traditional manual fabrication. However, designed for

controlled factory environments, these systems can hardly

provide the human builders’ flexibility dealing with various

uncertainties onsite (Bock 2007).

Thus, how to improve the paradigm for onsite fabrication

becomes one of the hot spots in digital fabrication research.

There seem to be three tracks of explorations to improve

either the accuracy, the flexibility, or even both, namely

Guided Manual Fabrication, Adaptive Robotic Fabrication,

and Hybrid Fabrication (Figure 2).

In a Guided Manual Fabrication paradigm, human builders

will have visual guidance about the current operation on

real objects through augmented displays. It is still the

builders who solve all the onsite uncertainties. The perfor-

mance of manual operations can be improved through the

guidance. Its early applications (Figure 3) can be found

in some onsite assembling projects (Reiners et al. 1999;

Tang et al. 2003). Although the guidance is very helpful in

many cases, without real-time evaluation on the effects of

manual operations for its generator, they can only provide

rough ones such as objects’ relationship in a layout, direc-

tions, orders, etc. rather than high precision spatial hints

essentially for accumulative error control in the case of a

sequential fabrication process.

In an Adaptive Robotic Fabrication paradigm, robotic

machines controlled by a man-made program deal with all

the uncertainties onsite through sensors evaluating the

operation effects and the changes of the environment in

real time. With a 3D scanner attached on the robotic arm

(Figure 4), such a system can deal with the uncertainties

caused by slight size differences among the regular bricks

(Dörfler et al. 2016). Although the system manages to

reduce onsite uncertainties to some extent, its flexibility is

still greatly constrained by the complexity of programming.

In a Hybrid Fabrication paradigm, human builders and

programmed machines interact with each other during the

whole process. The former deals with the onsite uncertain-

ties with manual decisions and operations. The latter keeps

offering visual guidance in a real-time cycle according to

any aims preset in the program, such as to keep accumula-

tive errors as low as possible. The visual guidance is based

on a set of technologies such as 3D scanning, point cloud

matching, parametric model refreshing, artificial decision

making, spatial projection, various performance simulation,

4 Mobile Robotic Brickwork (Dörfler et al. 2016)

5 STIK Pavilion (Yoshida et al. 2015)

etc. Typically, in a sequential fabrication process, the above

interaction makes it possible to compensate for errors

from previous manual operations with guidance for the

following operations (Zoran et al. 2013). As a preliminary

sample in building fabrication, Yoshida’s system (Yoshida

2

2 Comparison of various digital fabrication methods

3 AR Screw-fixing instruction (Reiners et.al 1999)

3

COMPUTATIONAL INFIDELITIES

84

et al. 2015) keeps scanning the whole pavilion of chop-

sticks (Figure 5), refreshing its stress distribution, and

projecting guidance of 2D patterns in a manual-invoke rate,

when human builders can still make their own decisions

according to both the guidance and onsite uncertainties.

In this paper, a hybrid fabrication project with a high rate

of interaction is introduced, which tries to build a freeform

pavilion of irregular foam bricks according to structural

optimization results. Firstly, a Kinect-Hololens system with a

WiFi-based workflow provides an almost real time interac-

tion and a more friendly 3D visual guidance. Secondly, due

to the involvement of the builders, the huge uncertainty from

irregular bricks is solved easily. Thirdly, a control surface

keeps refreshing itself according to the bricks installed and

the structural performance, which generates the guidance

and finally leads to an acceptable accumulative error.

METHODS

Human builders are more flexible than programmed

computers in dealing with uncertainties onsite, while

computers are more powerful than human builders in

finding the most structurally efficient form and keeping its

accumulative error at an acceptable level. To explore the

above Hybrid Fabrication paradigm, a team of students set

up a human–computer interactive process to make use

of irregular foam bricks standing in for building waste to

build a structurally optimized vault pavilion under computer

guidance in almost real time (Figure 1).

During the interaction process (Figure 6), the computer

system calculates the most suitable brick to install

currently available from a brick pool and indicates its visual

guidance. After seeing the guidance through a Hololens

helmet, a human builder makes their own decision on

installation details and executes the operation subjectively.

With every manual operation, the computer scans all the

bricks installed through a Kinect automatically and recalcu-

lates the current form of the design as a parametric model.

The updated form will be used in the following brick selec-

tion and visual guidance generation.

Notably, unlike design a in traditional building process,

which is fixed before fabrication, the parametric model

in a hybrid fabrication process is only a start point of

design, which holds all the designer’s aims and the site’s

constrains. During the interactive process, the design can

update itself according to the parametric model and results

gradually made by the builders when following the aims and

constraints tightly. In this way, the human builder’s flexibility

upon uncertainties onsite and the computer’s accuracy in

accumulative error control are both approached.

Designing the Vault Pavilion and Matching the

Augmented View

1) Generating the 3D form of the vault pavilion with

RhinoVAULT

The project starts with a 2D pattern given by the designer,

which describes the supports and the openings. Afterward

a 3D form of the vault pavilion is generated by RhinoVAULT.

The vault pavilion stands on four feet, which support the

weight of the vault itself, including three big feet and a

smaller one in the center (Figure 7). The construction of

the vault starts from the feet simultaneously and finally

converges at the top. In order to orient the coordinate

of the point cloud by scanner from the camera space to

the world space, seven infrared-perceivable markers are

placed around the vault. At one time, in whatever perspec-

tive, at least one of them should be detected by the Kinect.

2) Matching the digital vault with the real site through

Hololens

Before fixing the markers on the ground, the builders could

preview the vault’s shape overlapping with the surrounding

6

Hybrid Fabrication Sun, Zheng, Wang, Sun

85

environment, projected in Hololens (Figure 8), through

which the builders can have a rough perception of the

relationship between the vault and the site. The markers

are located with a full-size printed plan on the ground

according to the coordinates in Rhino.

Calculating the Guidance About the Next Brick

1) Finding the position of the next brick on the vault

The foam bricks are proposed to be installed in a layer-

by-layer manner. There is always a marginal layer under

operation, which is at the top of the last built layer or the

ground. The rough position of the next brick is calculated

as the initial position for the following physical simulation,

which follows a set of conditions that the brick should be

closest to the builder according to his position read from

the helmet and it should be next to a brick installed in the

marginal layer if one exists. If the current marginal layer is

full of bricks, it becomes a built layer and a new marginal

layer is ready for installation.

2) Scanning the bricks for a sample pool

A set of foam bricks are scanned with a constructed light

3D scanner, and they serve as a sample pool for the brick

selection in the next step. The size of the sample pool should

be balanced with the accuracy, the fluctuation of the size of

the bricks, and the efficiency.

3) Selecting the brick from a sample pool with ICP algorithm

For each position to install a brick, a sample pool of 24

prescanned foam bricks is searched for a fittest brick. An

algorithm called Iterative Closest Point (ICP) is used to

evaluate the fitness level between the installed surrounding

bricks and the alternative brick from the pool. Finally, the

brick with best fitness will be selected.

4) Calculating the orientation of the brick with Kangaroo

With a selected foam brick in a position, its orientation is

calculated through physical simulation in Rhino Kangaroo.

When it keeps the brick unchanged and pushes the brick

to the installed surrounding bricks to a stable status, the

orientation of the next brick is found.

Installing the Irregular Foam Bricks

1) Perceiving the guidance and the site conditions through

Hololens

The builders can perceive the hints and the real condi-

tions of the site through the helmet at the same time. A

program for Hololens display is coded in a Unity3D project

which shares the same coordinate with the digital model in

Rhinoceros. The updated geometry in computer would be

transmitted to the helmet though a server.

7 8

6 Process of Hybrid Fabrication

7 The Vault Generated

8 A View in Hololens

9 Installation of a Brick: In the left 3 x 3 grid, the pink brick is the next brick to install; The blue ones are bricks installed on the marginal layer; The white ones are other layers installed. In the right figure, the builder is installing one brick according to the visual guidance in Hololens.

9

COMPUTATIONAL INFIDELITIES

86

10

10 Recalculation and Updating

11 The Final Vault

12 Deviation Analysis

2) Making subjective decisions on the operation

It is the human builder who installs the bricks according to

the computer guidance (Figure 9). The positions of the brick

could be slightly adjusted by the builder in case there are

subjective judgments according to the onsite conditions,

which are not included in the calculation.

3) Operating the irregular foam bricks

The builder can slightly move or rotate the brick to the

fittest position. Acute changing of the position and the

orientation would break the loop.

Recalculating the Vault

1) Scanning the actual positions and orientations of the

installed bricks through Kinect

After one layer is installed, the bricks would be scanned

by the Kinect together with its closest markers, so that the

point cloud coordinates could be oriented from the camera

space to the Rhino space. An ICP calculation of the point

cloud with the corresponding bricks is executed to update

the bricks’ positions. The height of the marker should also

be updated according to the height of the bricks in order to

descend the deviations caused by deflection.

2) Checking the deviation and updating the vault

11

12

Hybrid Fabrication Sun, Zheng, Wang, Sun

87

After being scanned, the digital model is checked with its

deviation from the real vault installed. If a threshold is

reached, the 2D pattern would be trimmed by the projection

of the current marginal bricks. The original vault would

be replaced with the updated vault recalculated from the

trimmed pattern with RhinoVAULT following a catenary

formula (Figure 10).

RESULTSThe final form (Figure 11) indicates that the hybrid fabrica-

tion paradigm proposed in this paper is feasible, and the

final vault is stable with non-rigid connection between the

adjacent bricks. Its deviation does not obviously increase

as the height grows (Figure 12), which means the accumu-

lative error is almost constant within an acceptable level.

CONCLUSIONS AND FUTURE WORKS Through an interactive process, a project following Hybrid

Fabrication paradigm takes advantage of both the high flex-

ibility from human operations and acceptable accumulative

error controlled through computer guidance. The position of

the foam brick is calculated by computer, and the guidance is

suggested to the builder though an augmented helmet. After

one layer installation, the bricks on site would be scanned

and the design would be updated to adapt the renewed site

as well as the builders’ adjustments. This paradigm effec-

tively restrains the accumulative error and increases the

speed of calculation. Meanwhile, through recursive updates,

the final vault pavilion achieves the best approximation

between real construction and its digital counterpart.

However, there are still some phenomena to discuss, such

as the difference between passive deviations caused by

human operation and the active adjustments, which may be

a key question for the Hybrid Fabrication paradigm.

In the future, some comparative experiments will be

conducted to investigate the limitation of the paradigm.

Meanwhile, real small-scale projects and a multi-thread

arrangement of the building team will be explored.

ACKNOWLEDGEMENTSThis study is supported by the National Key Research

& Development Program of China (Grant No.

2016YFC0700200) and a project of National Natural

Science Foundation of China (Grant No.51778417) .

REFERENCESBock, Thomas. 2007. “Construction Robotics.” Autonomous Robots

22 (3): 201–9.

Dörfler Kathrin, Timothy Sandy, Markus Giftthaler, Fabio Gramazio,

Matthias Kohler, and Jonas Buchli. 2016. “Mobile Robotic

Brickwork.” In Robotic Fabrication in Architecture, Art and Design

2016, edited by D. Reinhardt, R. Saunders, and J. Burry, 204–17.

Cham, Switzerland: Springer.

Peng, Huaishu, Jimmy Briggs, Cheng-Yao Wang, Kevin Guo,

Joseph Kider, Stefanie Mueller, Patrick Baudisch, and François

Guimbretière. 2018. “RoMA: Interactive Fabrication with

Augmented Reality and a Robotic 3D Printer.” In Proceedings of the

2018 CHI Conference on Human Factors in Computing Systems,

579. Montreal, QC: CHI.

Reiners, Dirk, Didier Stricker, Gudrun Klinker, and Stefan Müller.

1998. “Augmented Reality for Construction Tasks: Doorlock

Assembly.” In Proceedings of the International Workshop on

Augmented Reality: Placing Artificial Objects in Real Scenes, 31–46.

Bellevue, WA: IWAR.

Tang, Arthur, Charles Owen, Frank Biocca, and Weimin Mou.

2003. “Comparative Effectiveness of Augmented Reality in Object

Assembly.” In Proceedings of the SIGCHI Conference on Human

factors in Computing Systems, 73–80. Ft. Lauderdale, FL: CHI.

Yoshida, Hironori, Takeo Igarashi, Yusuke Obuchi, Yosuke Takami,

Jun Sato, Mika Araki, Masaaki Miki, Kosuke Nagata, Kazuhide Sakai,

and Syunsuke Igarashi. 2015. “Architecture-Scale Human-Assisted

Additive Manufacturing.” ACM Transactions on Graphics 34 (4): 88.

Zoran, Amit, Roy Shilkrot, and Joseph Paradiso. 2013. “Human–

Computer Interaction for Hybrid Carving.” In Proceedings of the

26th Annual ACM Symposium on User Interface Software and

Technology, 433–40. St. Andrews, Scotland: UIST.

IMAGE CREDITSFigure 2: © Reiners et al. 1998.

Figure 3: © Dörfler et al. 2016.

Figure 4: © Yoshida et al. 2015.

All other drawings and images by the authors.

Chengyu Sun

Associate Professor of Architecture. His research covers a set of

hybrid design methods in architecture.

Zhaohua Zheng

Master Student of Architecture.

Yuze Wang

Master Student of Architecture.

Tongyu Sun

Professor of Architecture. His researches focus on urban design.

COMPUTATIONAL INFIDELITIES