Articulo Pav

8/11/2019 Articulo Pav

1/12

Review

Chronological development history ofXYtable based pavement crack sealers and

research ndings for practical use in the eld

Young S. Kim a, Hyun S. Yoo a, Jeong H. Lee b,, Seung W. Han a

a Department of Architectural Engineering, Inha University, South Koreab Center for Cost Engineering Research, Inha University, South Korea

a b s t r a c ta r t i c l e i n f o

Article history:

Accepted 19 February 2009

Keywords:

Automation

Pavement

Crack sealing

Machine vision

Robotics

Teleoperation

During the last two decades, several teleoperated and machine-vision assisted systems have been developed

to automate the overall process of routing and sealing pavement cracks. Productivity improvement, improved

safety and quality, and reduced road user costs have motivated these developments. This paper presents the

chronological development history ofxy table based pavement crack sealers, which have been developed

and demonstrated since the early 1990s, and compares their technical advances. This paper also discusses

primary research ndings in machine vision software and hardware designs of an automated pavement crack

sealer to be newly developed for practical use in the eld. Finally, conclusions and recommendations are

made concerning the value of implementing and practically using the automated pavement crack sealer.

2009 Elsevier B.V. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513

2. Automation needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5143. Automated pavement crack sealing systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514

3.1. Chronological development history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514

3.2. Evolution of the control paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514

3.3. Evolution of machine vision software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515

3.4. Evolution of hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516

4. Primary researchndings for practical application in the eld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517

4.1. Software design requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518

4.1.1. Path planning (full automation) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518

4.1.2. Graphical user interface design for crack sealer control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518

4.2. Hardware design requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520

4.2.1. Types of crack to be sealed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520

4.2.2. Crack image collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520

4.2.3. Entire crack sealing system architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520

4.2.4. Manipulator and end-effector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521

5. Conclusions and recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523

1. Introduction

Crack sealing, a routine and necessary part of pavement main-

tenance, is a dangerous,costly,and labor-intensiveoperation.During the

last two decades, several systems based onxytable for automatically

Automation in Construction 18 (2009) 513524

Corresponding author. Center for Cost Engineering Research, Inha University, 253

Yonghyun-dong, Nam-gu, Incheon, 402-751,South Korea. Tel.:+82 32 8729757; fax: +82 32

866 4624.

E-mail address:[email protected](J.H. Lee).

0926-5805/$ see front matter 2009 Elsevier B.V. All rights reserved.

doi:10.1016/j.autcon.2009.02.007

Contents lists available at ScienceDirect

Automation in Construction

j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / a u t c o n
mailto:[email protected]://dx.doi.org/10.1016/j.autcon.2009.02.007http://www.sciencedirect.com/science/journal/09265805http://www.sciencedirect.com/science/journal/09265805http://dx.doi.org/10.1016/j.autcon.2009.02.007mailto:[email protected]


2/12

routing and sealing pavement cracks have been developed. Examples

include: 1) CMU laboratory prototype (1990) [1,2], 2) CMU-UT eld

prototype (1992)[2,3], 3) UT Automated Road Maintenance Machine

(ARMM) (1997)[36], and4) AutomatedPavement Crack Sealer (APCS)

(2004) [7,8]. Since automating pavement crack sealing can improve

safety, productivity, and quality, and also reduce road user costs, there

has been extremely large demand for practical use of automated

pavement crack sealers in the areas of road construction and

maintenance. While early works sought to completely automate crackmapping and sealing activities, experience and the resulting deeper

understanding of the enabling technologies have highlighted the

importance ofnding a desirable balance between human and machine

functions in the control of automated pavement crack sealers.

Through trial and error and about 20 years of perseverance, the

APCS, the most recent deliverable research, has achieved a desirable

balance between manual and automated functions for automated

pavement crack sealing. Recent eld trials of the full scale APCS have

also indicated that automated pavement crack sealing is now

technically, economically, and nancially feasible. Despite such

numerous efforts to automate conventional crack sealing operations,

lessons learned from previous system developments and eld trials

have indicated that several improvements in both software and

hardware designs are still required for their practical application in

the eld. The primary objective of this paper is to present overall

software and hardware design requirements for practical use of an

automated pavement crack sealer in an effort to fulll the aforemen-

tioned demand in road construction and maintenance. This paper

presents the chronological development history of xy table based

pavement crack sealers, which have been developed and demon-

strated since the early 1990s, and compares their technological

advances. This paper then proposes primary research ndings in

machine vision software and hardware designs of an automated

pavement crack sealer to be newly developed for practical use in the

eld. A conceptual hardware design for a new model is proposed in

this paper as well. Finally, conclusions and recommendations are

made concerning the value of implementing and practically using the

automated pavement crack sealer.

2. Automation needs

Crack sealing is a maintenance technique commonly used to

prevent water and debris penetration and reduce future pavement

degradation. The conventional crack sealing operations are, however,

dangerous, costly, and labor-intensive. With respect to crack sealing

crews, labor turnover and training are also increasing problems.

Automation of the crack sealing process would improve productivity

and quality, and offers safety benets by getting workers off the road.

The reduction in crew size and the increase in productivity of the

automated sealing process are expected to be translated directly into

signicant cost savings.

For example, APCS eld test results indicated that the daily

productivity would be 1.59 km/day. Compared with the productivityof a conventional crack sealing method (1.21 km/day), that of the

APCS was as much as 0.39 km/day higher. On-site tests and a

performance analysis of the APCS demonstrated that its use would

allow a 50% reduction of the labor force and 32.5% enhanced

productivity [8]. Furthermore, when considering nighttimeoperations

and possible hardware improvements of the developed APCS, the

productivity of the APCS would be even higher.

The results of an economic feasibility analysis of the APCS also

revealed that automating the conventional crack sealing operation is

highly feasible and would potentially bring enormous cost savings.

Information for the analysis was gathered to estimate costs and

benets, analysis perspectives were chosen, and the market was

studied. Rate of return, benetcost ratio, break-even point, and

sensitivity analyses were used to verify the economic feasibility of

implementing the automated method in place of the conventional

method. Under assumptions such as an APCS purchase cost of US

$72,000, 100 working days per year, use of 1 APCS, 10% MARR, a

10 year planning horizon, a 50% reduction in labor force, etc., it was

anticipated that a contractor would be able to cut conventional

maintenance costs by 43.6%. With the above assumptions, the

economic analysis results of the APCS also showed a value of 122.5%,

5.5, a 15 month in rate of return period, benetcost ratio, break-even

point, respectively, thus making the use of APCS highly attractive. Theresults of a sensitivity analysis and predictions pertaining to reduction

of road user costs obtained using Paramics simulation software have

been presented elsewhere[8].

3. Automated pavement crack sealing systems

In this chapter,xytable based systems developed since the early

1990s for automatically routing and sealing pavement cracks are

briey described. Technological details of the systems are presented,

and related research accomplishments, concerns, and technical

advances are identied. Visual appearances of each prototype system

are illustrated as well.

3.1. Chronological development history

Table 1 briey describes the accomplishments and major limita-

tions of previous research works. The hardware of early xy table

based pavement crack sealing systems was incomplete in the early

stage. At the same time, the software for mapping and modeling the

crack network to be sealed and path planning were not efcient in

terms of productivity, quality, and accuracy for practical use in the

eld. In addition, the hardware and software were not integrated

properly, causing inaccurate and inefcient movement of the auto-

mated pavement crack sealing systems. The experience acquired from

such early attempts and recent advances in relevant robotic

technologies motivated authors to develop more advanced pavement

crack sealer (APCS). Although the APCS employs a unique man-

machine interfaced control process and provides innovative technical

advances compared to previous research works [16,10], they havenot been practically used on sites due to limitations in their hardware

and software. Detailed comparisons and several limitations regarding

the control paradigm, machine vision software, and hardware in the

xy table based pavement crack sealers developed in previous

research works are presented inSections 3.23.4.

3.2. Evolution of the control paradigm

Complete autonomy[1,3]could be achieved for the whole process

(image acquisition, crack detection and mapping, path planning,

blowing, sealing, and squeegeeing) of automated pavement crack

sealing, but usually at a cost, speed, and accuracy that is impractical

and unacceptable. Complex evolution of the control paradigm has

resulted in a functional production prototype system [46,9,10]thatachieves a good balance between manual functions and automated

functions by taking advantage of the respective strengths of man and

machine in the whole process. Teleoperation based on remote video,

man-machine interfaces, machine vision, and graphical programming

alone can achieve benets of automation in the unstructured

pavement crack sealing work environment. Lessons learned from

system developments and eld trials have also indicated that

computer assistance in the form of man-machine interfaced graphical

programming and machine vision is essential and can cost-effectively

help to achieve improvements in the productivity and quality of

automated methods. Considering recent successful developments in

teleoperated construction eld robots, it is thought that evolution

toward teleoperation as a control paradigm of the automated

pavement crack sealer is highly desirable. The ARMM and the APCS

514 Y.S. Kim et al. / Automation in Construction 18 (2009) 513524


3/12

employ some forms of man-machine interfaced control process for

automatically sealing pavement cracks.

3.3. Evolution of machine vision software

In general, machine vision software in automated pavement crack

sealers can be classied into the following four categories:

Image acquisition

In previouslydevelopedpavementcrack sealers, a computer imaging

system is typically used to view cracks to be sealed on the surface. The

use of remote video cameras is widely employed to provide visual

feedback in the machine. To capture and digitize video images of the

cracks in the machine's work space, a commercial image processing

board is installed in the PC that controls the automated crack sealers.

One or two security cameras mounted on a super-structure of the XY-

table manipulator ((1)(4) inTable 1) acquire live pavement surface

images displayed on the PC's monitor.

Initial crack detection, mapping, and representation

To automatically seal cracks, exact crack locations are the most

important information.In developinga new image processingalgorithm

for automatically sensing, mapping, and representing cracks to be sealed

in pavements, the quality of the segmentation must take the highest

priority. The overall efciency of the algorithm is also important for

project success,if thecracks areautomatically detected andmapped bya

computer.

In CMU laboratory and CMU-UT eld prototypes, a video camera is

used to acquire images. These images are then digitized and subse-

quently combined with laser range data of surface contours using a

specially designed multi-layer quadtree model and image analysis

algorithms, via the process of sensor fusion [1

3]. However, it wasconcluded that detection and mapping pavement cracks to be sealed by

computer functions alone may not be desirable. This is due to the fact

that most pavement cracks are highly noisy and unstructured in their

digitizedform at the pixel level. Thelow contrast betweenthe distresses

and the background also further complicate the pavement analysis. As a

result, the lessons learned revealed that a human operator should

interact directly with the computer in thecrackdetection, mapping, and

representation processes, as in the ARMM and the APCS.

As an example, such a man-machine interface was rst tested to

detect, map, and represent pavement cracks to be sealed in the ARMM

[46,9,10]. Under proper conditions of illumination, humans can see

color, brightness, and form, thus easily distinguishing real cracks from

pavement background noise such as sealed cracks or oil marks or skid

marks. As such, allowing the operator to point out the existence and

location of pavementcracks using a styluson a touchsensitive monitoris

an effective approach. A graphical program can be used to generate a

computer-based model of the surface cracks to be sealed and to provide

visual feedback to the system operator. A good model can be achieved

since the work environment of the automated pavement crack sealer is

fairly static. Despite such advantages, this approach has a drawback

that can signicantly degrade the quality of the resultant seal. The

imperfection of human hand-eye coordination causes errors even in a

good work environment. Also, operator arm fatigue can increase such

errorswhen tracing cracks on themonitor. In theARMM,machine vision

based line snapping or manual editing wasthus required to compensate

human hand-eye coordination errors [5,6]. In a subsequent research

effort, a similar man-machine interface was applied to the ARMM for

detecting, mapping, and representing the pavement cracks to be sealed.

However, in the APCS, crack network detection, mapping, andrepresentation are operated in two modes: full automation and

semi-automation. A unique and innovative machine vision algorithm

has been developed for the automation of pavement crack sealing.

Detailed descriptions of both fully and semi-automated crack network

detection, mapping, and modeling algorithm have been presented

elsewhere[9].

Path planning

Compared to conventional crack sealing operations, the efciency of

movement (pathplan) of an automated crack sealer is a key factor with

regard to performance. Efciency of movement is governed by the

length of the paththat the end-effectors follow to cover the whole crack

network. Other performance factors include manipulator speed and

accuracy, and the duration of crack detection, mapping, and representa-tion. If cracks are fully monitored and mapped by thehuman naked eye,

a path planis notrequired. However,if cracksare mappedmanually on a

computer monitor, an implicit pathplan is generated by the sequence of

strokes made by the operator, including the beginning and end points.

Meanwhile, automated path planning is necessary when cracks are

automatically mapped by the computer. In the CMU-UT eld prototype

and the ARMM, a series of greedy path planning algorithms using a

double linked data structure and an array data structure were proposed

in an effort to automatically generate a path to effectively traverse the

cracknetwork to be sealed[10]. Thetest results showedthat,on average,

the time taken to compute a greedy path was much less than the time

saved by executing the greedy path. Recently, optimal and more

advanced greedy path planning algorithms were successfully developed

and applied to the APCS [7]. In an effort to minimizethe idlepath inthe

Table 1

Previous research works related to thexytable based pavement crack sealing systems.

Previous research works Accomplishments and major concerns

(1) CMU laboratory

prototype (1990)

Conceptual design of S/W and H/W for

full automation

Crack detection, mapping and

representation using digital image

processing

Incomplete and unstable fabrication of

H/W(X

Ymanipulator) Excessive processing time in the crack

detection, mapping and modeling

(2) CMU-UT eld

prototype (1992)

Development of a machine vision

algorithm using sensor (vision and

laser range scanner) fusion

Development of path planning algorithm

using linked data structure

Fabrication of more robust H/W

(XY-manipulator) and partial integration of

S/W and H/W

Partial verication of fully automated

crack sealing process

Excessive processing time due to slow

range scanning

(3) ARMM (1997) Suggestion of a man-machine balanced

control process for effectively sealing

pavement cracks

Development of a machine vision

algorithm (manual mapping, line snapping

and manual editing) using graphical

programming and user-friendly control

software

Development of greedy path planning

algorithm using array data structure

Need to improve the design of turret and

machine vision algorithm[2]

(4) APCS (2004) Suggestion of a man-machine balanced

control process for effectively sealing

pavement cracks

Development of an innovative machine

vision algorithm which can both

automatically and semi-automatically seal

pavement crack network using user-friendly

graphical control software

Development of greedy and optimal pathplanning algorithms

Nighttime operation using lighting

system mounted on the super-structure of

the machine

515Y.S. Kim et al. / Automation in Construction 18 (2009) 513524


4/12

crack network to be traversed, as well as to maximize the efciency

(productivity) of the APCS, the developed optimal and greedy path

planning algorithms are intelligently and selectively used, based on the

complexity of crack networks to be traversed in a given workspace.

Table 2 briey summarizes the technical advances related to the control

paradigm and machine vision software made in previous research

works.

3.4. Evolution of hardware

In previous research works, the system architectures for the xy

table based pavement crack sealing consisted of multi-units (tow

truck, sealant melter, and crack sealer). The CMU laboratory proto-

type, CMU-UT eld prototype, the ARMM, and the APCS use multi-

units as shown in Table 1. XY-table manipulators with a turret

structure as an end-effector and multi-DOF manipulator arms were

employed as the main bodies of the crack sealing hardware. Table 2

compares the technical aspects of the previous automated pavement

crack sealing hardware.

The APCS is an extended version of the ARMM with innovative

improvements to both hardware and software. The hardware of the

APCS is composed of thefollowing three units:1) a towtruck, inwhich

a systemoperator controlsthe APCS using a PC;2) a sealant melter that

supplies 170180C sealant tothe APCS; and 3) a crack sealing unit for

blowing, sealing, and squeegeeing cracks in pavements. The major

components of the cracksealing unit of the APCS areas follows: 1)XY-

manipulator with cart, gantry, turret, and CCD camera, 2) control box,

3) industrial PC with a frame grabber and a touch sensitive monitor,

and 4) power supplies. In designing and fabricating the hardware of

the APCS, the following factors were considered:

(1) Designing anXY-table manipulator that does not twist against

the pavement level and various working situations;

(2) Adding rotating and telescoping functions to the end-effector

(turret structure) to effectively blow, seal, and squeegee the

crack network to be sealed in rutted pavement;

(3) Adding a heating function to theinside of the turret structure to

inject sealant with a certain degree of viscosity without rapid

solidication;

(4) Designing a turret structure to effectively squeegee the sealantinjected into the pavement cracks (turret with a squeezing

device having a Vor Ushape); and

(5) Mounting a lighting system for night work to minimize road-

user-costs and increase productivity.

Based on theidentied major considerations, theAPCS wasdesigned

and fabricated as shown in Fig.1. The APCS usesanXY-manipulator with

a rotating turret to blow, seal, and squeegee pavement cracks in one

sealing travel. While themanipulatoris movingwithin itsworkspace, its

frame is stationary. Sealing cracks in one work area and then moving to

the next work area is considered one work cycle. To control the APCS

through a work cycle, the following six steps are required:

(1) Acquire a pavement image including the crack network using a

digital CCD camera installed on the superstructure of the APCS

and a frame grabber board installed in the PC;

(2) Stop the APCS if any crack network is detected on the operator's

touch sensitive monitor;

(3) Automatically eliminate noises for mapping and modeling the

cracknetwork(This process is required only forthe case of fully

automated crack network mapping and modeling, to be

explained in Section 3.3);

Table 2

Comparison result of control paradigm, machine vision software, and crack sealing hardware developed in previous research works.

Control paradigm (1) (2) (3) (4)

Full automation

Teleoperation

Machine vision software (1) (2) (3) (4)

Vision Image acquisition Fully automated (vision sensor)

Fully manual (naked eyes)

Initial crack identication on video image Fully automated


Crack mapping and representation Fully automated

Semi-automated (MMI)

Manual editing


Graphical user interface

Path planning Fully automated Optimal

Greedy

Not considered

Hardware (1) (2) (3) (4)

Entire system architecture Single unitMulti-unit

Manipulator and end-effector XY-table manipulator with turret

Multi-DOF manipulator arm

Independent sealing apparatus

Functions equipped Blow

Seal

Squeegee

Telescope

Heat

Crack image collection Remote video

Laser range scanner

Human driver(sys. operator)

Lighting system for nighttime operation

Types of cracks sealed Transverse cracking

Longitudinal cracking

Block cracking

(1) CMU laboratory prototype, (2) CMU-UT eld prototype, (3) ARMM, (4) APCS.



5/12

(4) Extract the exact spine of the crack network using the

developed crack network mapping and modeling algorithm

(In the APCS, the system operator can map and model the crack

network to be sealed in a given work space by both automated

and semi-automated methods selectively);

(5) Automatically perform optimal path planning, based on the

results of mapping and modeling for the crack network; and

(6) Automatically blow air, inject sealant into the crack network,

and squeeze the sealant based on the calculated path plan.

The eld test results showed that the APCS was able to more

intelligently detect cracks through video images, and subsequently

automatically blow dust, inject sealant materials into cracks, and moreefciently squeegee the sealed cracks than the previous version (the

ARMM). Longitudinal, transverse, and block (random) cracks could be

effectively repaired by this machine. Although the current prototype

showed signicant improvements in hardware and software system

development, too much time was required to set up the whole system

including the tow truck, sealant melter, and crack sealing unit. Also, it

was difcult for the driver to precisely control the directional change

of three different units connected sequentially. For practical use under

a more controlled work environment, a new design is thus required to

reduce the preparation time and improve the mobility.Table 3briey

summarizes the advantages and disadvantages of hardware design

used for the APCS. This state-of-the art automated pavement crack

sealing research prototype is the most advanced research deliverable

in the form ofxytable manipulator.

4. Primary research ndings for practical application in the eld

It has taken approximately 20 years to achieve a balance between

human and machine functions for automated pavement crack sealing,

instead of pursuing an all-in-one full automation system. Over the

course of developing four physical crack sealing prototype systems

usingxytables (Table 1), a series of unique man-machine balanced

control loops for successful automation of crack sealing have been

developed for computer assisted teleoperation (Fig. 1). A complex

evolution in both software and hardware design has resulted in

functional production prototype system (APCS) that veries the

Fig. 1.Automated pavement crack sealer (APCS).

Table 3

Advantages and disadvantages in hardware design of the APCS.

APCS

Advantages

_ More effective design of end-effector (turret)

_ High accuracy for crack sealing

_ Cracks in pavement with rutting can be effectively sealed.

_ Potential productivity improvement and cost savings due to night

time operation

Disadvantages

_ Poor mobility: three units a tow truck, a sealant melter, and the

XY-table manipulator with turret

_ Only cover the area within theXYtable. It may require multi-paths

to cover one full lane, if the workspace ofXY-table is small.

_

Only used for technology feasibility demonstration



6/12

prevalence of the automated crack sealing method. The following are

lessons learned and primary research ndings in software and

hardware designs obtained from previous works over the last two

decades.

4.1. Software design requirement

(1) Image acquisition (full automation): As aforementioned, tele-

operation was recommended as a preferred control option forcrack sealing over complete automation. Quite often, the

teleoperation user is in a location that provides either poor or

no visual feedback. A remote camera is thereforegenerally used

to provide visual feedback. A security or infra-red camera can

be used for automatically acquiring live pavement images. The

images acquired by means of the remote camera are then

digitizedby a framegrabber, and theresultscan be displayed on

the monitor.

(2) Initial crack identication on video images (manual): Since

humans can instantly distinguish real cracks from pavement

background noise such as sealed cracks or oil or skid marks

under proper conditions of illumination, a human operator

should interact in the process of initial crack identication. As

shown in the test results of the most recent xy table based

pavement crack sealer (APCS), lessons learned also show that

initially identicationof a crackis betterperformed by a human

than by a computer in a very time-effective manner.

(3) Crack detection, mapping, and modeling (full or semi-automa-

tion): Automatically detecting, mapping, and modeling cracks in

pavement are desirable and technically feasible. However, there

is considerable concern about the accuracy and time component

of the processed results, because most pavement cracks are

highly noisy and unstructured in their digitized form at the pixel

level. Allowing the operator to point out the existence and

location of a pavement crack using a mouse or a stylus on a touch

sensitive monitor is economical and benecial. A graphical

program can be used to generate a computer-based model of

the surface cracks to be sealed and to provide visual feedback to

the operator. A good model can be achieved since the workenvironment of the automated pavement crack sealer is fairly

static. Despite such advantages, this approach has a drawback

that can signicantly degrade the quality in the resultant seal.

That is, the imperfection of human hand-eye coordination and

arm fatigue when tracing cracks on the monitor can cause errors,

even in a good work environment.

A good model for crack network detection, mapping, and represen-

tation was proposed for the APCS. In the APCS, crack network detection,

mapping, and representation are operated in two modes: full automa-

tion and semi-automation. A unique and innovative machine vision

algorithm has been developed for automation of pavement crack

sealing. Fully automated crack network detection, mapping, and

representation process successively includes: binarizing, noise elimina-tion, dilation, edge thinning and linking, and complete crack network

modeling, as shown inFig. 2. Sixty pavement crack images with shades

and various intensity of pavement surface were experimented to

measure the accuracy (86.6%) and efciency (0.46sec./image) of the

developed algorithm. Fig. 2 illustrates the semi-automated crack

network detection, mapping, and representation process of the APCS

in a very simplied form. Detailed descriptions of both fully and semi-

automated crack network mapping and modeling algorithms have been

presented elsewhere[7,8].

The test results showed that the semi-automated crack network

mapping andmodelingalgorithm wassuperior in terms of accuracyover

the fully automated crack network mapping and modeling algorithm,

but inferior in terms of productivity. However, both methods could

eventually guarantee 100% accuracy, because a manual editing function

witha rubber banding capability(Fig.2b)could beusedin boththe fully

automated and semi-automated methods. Complex and innovative

technical advances have been made in machine vision-based crack

network mapping and modeling algorithms compared to previous

researchefforts. Duringeldtrials, it wasfoundthat thesystem operator

usually uses the automated crack network and modeling method, as its

accuracy was sufcient for practical use. Furthermore, if there were any

errors in the extracted model, the resultant crack network model could

be easilyadjustedusing themanualediting function. Finally,the authorsestimate that the automated and semi-automated crack mapping and

modeling algorithms and processes developed for the APCS can be

directly applied to the automated pavement crack sealer to be newly

developed.

4.1.1. Path planning (full automation)

If there are several lines in the crack network extracted from a

pavement image, the number of paths is 2nn!. In the greedy path

planning algorithm, the nearest nodepoint from the home point among

the end node points of each crack network is the start node point for

sealing. The end-effector of the automated pavementcrack sealer injects

sealant along thecracknetworkuntilit reaches theotherend nodepoint

in the greedy path planning algorithm (Fig. 3a). Previous studies

proposed some greedy path planning algorithms for automated

pavement crack sealing. However, the paths planned by such greedy

algorithms often were not the optimal path.

An optimal path planning algorithm that can always provide the

shortest path in a given crack network was developed for the APCS. In

the optimal path planning algorithm, a computer calculates the lengths

of all paths so that the shortest path can be selected ( Fig. 3b). Ex-

perimentaltest results showed that the optimal pathplanning algorithm

is effective if the number of crack lines in a network is 6 or less, and the

greedy path planning algorithm is useful if the number of crack lines in

a network is more than 6, from a computational time perspective. Once

the algorithm automatically identies the number of crack lines in a

given crack network, the path planning method (optimal or greedy

path plan) is then selectively chosen and their xand ycoordinates in

2D workspace, which are ultimately required for control of the auto-

mated pavement crack sealer, are generated. Therefore, the pathplanning algorithm proposed for the APCS can be directly applied to

the automated pavement crack sealer to be newly developed as well.

4.1.2. Graphical user interface design for crack sealer control

In any man-machine system, an effective and user-friendly designof

the graphical user interface for controllinghardware is veryimportant in

terms ofnal acceptance in the market. Poor user interface design or

difcult system operation signicantly degrades the system perfor-

manceand productivity.Mostworkers in theeldare seldomexposedto

complicated computer control systems for road construction and

maintenance tasks and labor turnover rate is relatively high. As such,

ease of operation based on a user-friendly graphical interface design

would be a critical factorin attracting prospective end-usersas well as in

practically using the man-machine system in the eld. It is alsoestimated that overall efciency of the automated pavement crack

sealer will depend as much on the user-friendliness of software and

hardware controls as on the capabilitiesof themachine visionalgorithm

and crack sealing speed.

Findings in previous research efforts have indicated that the method

of clicking graphical menu buttons on a touch sensitive monitor is the

most appropriate design option for effective control of an automated

pavement cracksealer.The graphicaluser interfaceof theAPCS shown in

Fig. 4is a good example of this. Under such a user-friendly graphical

interface, the system operator should be able to easily understand the

man-machineinterfaced pavement crack sealing process and effectively

control the machine. The system operator can also observe the current

working status via the monitor in real time and instantly command

actions to be performed by clicking the menu buttons on the touch



7/12


8/12

sensitive monitor. This user-friendly graphical interface design would

also signicantlyreduce possible problems related to labor turnover and

training of crack sealing crews. Therefore, it is anticipated that the

graphicaluser interfaceof theAPCS shown in Fig.4 can beadopted inthe

present form, or used as a reference in developing a new model of the

automated pavement crack sealer.

4.2. Hardware design requirements

4.2.1. Types of crack to be sealed

In general, crack patterns existing on pavements are classied into

the following four categories: 1) longitudinal, 2) transverse, 3) block,

and 4) alligator cracks. Longitudinal, transverse, and block cracks are

mainly repaired by sealing, while overlay or patching is used for

repairing alligator cracks. In order to appeal to private contractors and

government departments, as well as for practical use in the eld, it

must be guaranteed that the automated pavement crack sealer can

seal any type of longitudinal, transverse, and block cracks on normal

and/or rutted pavement surfaces.

4.2.2. Crack image collection

As aforementioned, the system operator of the automated pavement

crack sealer is in a location (e.g., the tow vehicle's cab) where visual

feedback of the workspace is not available. In this situation a typical

method for providing visual feedback is through the use of remote video

cameras. The video cameras provide several different views of the

equipment in its workspace, as well as the crack sealing status in real

time.This operatingenvironment provides the system operator with the

necessary visual feedback to control the automated pavement crack

sealer. The remote video signal is also processed by a computer using a

machine visionalgorithmto aidin thecompletionof theimage collection

task for crack identication. Single or multi-CCD (or infrared) camerascan be used based on the hardware architecture and the size of the

workspace designed. A series of lightingsystemsmust alsobe considered

for nighttime operation of the automated pavement crack sealer.

4.2.3. Entire crack sealing system architecture

Since a multi-unit crack sealer (e.g., APCS), which consists of a tow

truck, sealant melter, and crack sealer, would require excessive

mobilization and demobilization time, a single self-contained vehicle

[11,12]would be a better alternative for the entire system architecture

(appearance) of an automated pavement crack sealer. The reduction in

time to connect/disconnect the three independent units and to hook/

unhook all the cables for mobilization and demobilization would be

directly translated into signicant potential improvements in daily

productivity of the crack sealer.

Fig. 3.Comparison of greedy and optimal path plan results in the APCS.



9/12

4.2.4. Manipulator and end-effector

An automated pavement crack sealer to be newly developed needs

a minimum of 4 degrees of freedom to effectively move and seal along

thespineof cracknetworks.X-axis and Y-axis movements are required

to move in any direction on a 2D surface. Z-axis movement

(telescoping function) is also required to guarantee that cracks in

rutted pavement are sealed properly. In addition, the turret needs to

rotate to effectively blow, seal, and squeegee along the spine of the

crack network. Therefore, a motion control system is required to direct

the crack sealer along the X-, Y-, and Z-axes and to rotate the end-

effector of the automated pavement crack sealer. From previous

studies, ow ability of the melted sealant has been identied as an

important factor. To maintain the sealant in an appropriately viscous

state without any internal congestion in the hose, the length of the

Fig. 5.Software and hardware design requirements for enhancing the eld applicability.

Fig. 4.User-friendly graphical interface design of the APCS [7].



10/12

Fig. 6.Conceptual hardware design of automated proposed pavement crack sealer.


11/12


12/12

[7] J.H. Lee, H.S. Yoo, Y.S. Kim, M.Y. Cho, J.B. Lee, The development of a machine visionassisted, teleoperated pavement crack sealer, Proc. of the 21st ISARC, Jeju, Korea,September 2004, pp. 149158.

[8] J.H.Lee, Y.S. Kim, J.B. Lee, M.H. Jeong,Developmentof an automatedpavement cracksealingmachine and its economic feasibility analysis, Korea Institute of ConstructionEngineering and Management(KICEM), Seoul, Korea 7 (6) (2006) 151164.

[9] Y. Kim, H. Yoo, J. Seo, Machine vision algorithm for automation of pavement cracksealing, Submitted to Journal of Computer-Aided Civil and InfrastructureEngineering on February (2007).

[10] Y.Kim, C. Haas, R. Greer,Path planning for a machine vision assisted, tele-operatedpavement crack sealer, ASCE Journal ofTransportation Engineering 124(2) (1998)137143.

[11] S.A. Velinsky, Fabrication and testing of an automated crack sealing machine,SHRP-H-659, National Research Council, Washington, D.C., 1993.

[12] S. Velinsky, B. Rabani, Longitudinal crack sealing and random crack sealing inintroduction to advanced highway maintenance and construction technology onhttp://www.ahmct.ucdavis.edu(2008).


Articulo Pav

Documents

Transcript of Articulo Pav