Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology,...

11
7 th Asia-Pacific Workshop on Structural Health Monitoring November 12-15, 2018 Hong Kong SAR, P.R. China Advancements in Structural Health Monitoring Using Vision-Based and Optical Techniques A. Sabato 1 *, A. Sarrafi 1 , Z. Mao 1 , C. Niezrecki 1 1 Department of Mechanical Engineering, University of Massachusetts Lowell, 1 University Avenue, Lowell, MA 01854, USA Email: [email protected]; [email protected]; [email protected]; [email protected] KEYWORDS: Digital Image Correlation; Motion Estimation; Optical Techniques; Structure from Motion; Video Magnification ABSTRACT Engineering structures continue to be used despite that they have reached or are approaching their design lifetime. Therefore, developing practical approaches that can be used to evaluate the condition and identify damage before in-service failure occurs is essential. Often, Structural Health Monitoring (SHM) relies on contact-type sensing techniques such as accelerometers, strain gauges, and potentiometers for condition assessment. The use of distributed and discrete sensors has some drawbacks. They can be labour intensive to install, require large channel counts, require power and signal transmission through wires, and they provide information at only a few discrete measurement points. Recent advancements achieved in cameras technology, image-processing algorithms, and computer-aided vision systems, have made optically-based techniques such as three-dimensional Digital Image Correlation (3D-DIC), Structure from Motion (SfM), phase-based motion estimation (PME), and video magnification (VM) effective approaches for structural interrogation and infrastructure inspection. This paper presents an overview of the research experience performed by the University of Massachusetts Lowell’s team in the last decade. It includes inspection of large-sized civil engineering structures such as concrete bridges experiencing cracking and spalling, railroad tracks subjected to centre binding conditions, and utility- scale wind turbine blades during static and dynamics tests. Also, the opportunity to couple vision-based techniques with autonomous systems to expedite the capability of the measurement process while not interfering with the structure’s operations is described. To finish, future research directions are briefly introduced. The results of the activities performed show that computer-vision techniques represent a powerful tool available for SHM inspections and evaluations. 1. Introduction Civil, mechanical, and aerospace structures continue to routinely be used even though they are approaching or have already exceeded their design life [1] . Effective and low-cost non-destructive inspection (NDI) and novel structural health monitoring (SHM) procedures are continuously sought to improve structural safety, reduce the frequency of sudden breakdowns, minimize downtime, and streamline testing procedures [2] . Sensing techniques like visual inspection and wire-based transducers (e.g., strain gages, accelerometers, string potentiometers, acoustic transducers) are the primary techniques used for identifying the static and dynamic responses of a structure as it is subjected to loading [3] . The main drawback of these approaches is that they produce results at only a discrete number * Corresponding author. Creative Commons CC-BY-NC licence https://creativecommons.org/licenses/by/4.0/ More info about this article: http://www.ndt.net/?id=24068

Transcript of Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology,...

Page 1: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

7th Asia-Pacific Workshop on Structural Health Monitoring

November 12-15, 2018 Hong Kong SAR, P.R. China

Advancements in Structural Health Monitoring Using Vision-Based and

Optical Techniques

A. Sabato 1*, A. Sarrafi1, Z. Mao1, C. Niezrecki1

1 Department of Mechanical Engineering, University of Massachusetts Lowell, 1 University Avenue,

Lowell, MA 01854, USA

Email: [email protected]; [email protected]; [email protected];

[email protected]

KEYWORDS: Digital Image Correlation; Motion Estimation; Optical Techniques; Structure from

Motion; Video Magnification

ABSTRACT

Engineering structures continue to be used despite that they have reached or are approaching their design

lifetime. Therefore, developing practical approaches that can be used to evaluate the condition and

identify damage before in-service failure occurs is essential. Often, Structural Health Monitoring (SHM)

relies on contact-type sensing techniques such as accelerometers, strain gauges, and potentiometers for

condition assessment. The use of distributed and discrete sensors has some drawbacks. They can be

labour intensive to install, require large channel counts, require power and signal transmission through

wires, and they provide information at only a few discrete measurement points. Recent advancements

achieved in cameras technology, image-processing algorithms, and computer-aided vision systems, have

made optically-based techniques such as three-dimensional Digital Image Correlation (3D-DIC),

Structure from Motion (SfM), phase-based motion estimation (PME), and video magnification (VM)

effective approaches for structural interrogation and infrastructure inspection. This paper presents an

overview of the research experience performed by the University of Massachusetts Lowell’s team in the

last decade. It includes inspection of large-sized civil engineering structures such as concrete bridges

experiencing cracking and spalling, railroad tracks subjected to centre binding conditions, and utility-

scale wind turbine blades during static and dynamics tests. Also, the opportunity to couple vision-based

techniques with autonomous systems to expedite the capability of the measurement process while not

interfering with the structure’s operations is described. To finish, future research directions are briefly

introduced. The results of the activities performed show that computer-vision techniques represent a

powerful tool available for SHM inspections and evaluations.

1. Introduction

Civil, mechanical, and aerospace structures continue to routinely be used even though they are

approaching or have already exceeded their design life[1]. Effective and low-cost non-destructive

inspection (NDI) and novel structural health monitoring (SHM) procedures are continuously sought to

improve structural safety, reduce the frequency of sudden breakdowns, minimize downtime, and

streamline testing procedures[2]. Sensing techniques like visual inspection and wire-based transducers

(e.g., strain gages, accelerometers, string potentiometers, acoustic transducers) are the primary

techniques used for identifying the static and dynamic responses of a structure as it is subjected to

loading[3]. The main drawback of these approaches is that they produce results at only a discrete number

* Corresponding author.

Creative Commons CC-BY-NC licence https://creativecommons.org/licenses/by/4.0/

Mor

e in

fo a

bout

this

art

icle

: ht

tp://

ww

w.n

dt.n

et/?

id=

2406

8

Page 2: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

of points. Moreover, they can be labour-intensive and expensive to install, have wire and power

constraints, and the correlation between the sensor signal and structural integrity is not always

straightforward.

In recent years, a great deal of attention has been given to non-contact and remote measurement

techniques. Systems such as Acoustic Emission (AE)[4, 5], active and passive infrared (IR) images[6],

Ground Penetrating Radar (GPR)[7], Interferometric Synthetic Aperture Radar (InSAR)[7], Light

Detection And Ranging (LiDAR)[8], Ultrasonic Testing (UT)[9, 10], and Wireless Sensor Networks

(WSNs)[11, 12] have pros and cons for monitoring purposes. Their costs and the difficulties of being

installed and used for in-situ monitoring are severe drawbacks to their use especially for large-area

inspections. Recent developments in camera technology, optical sensors, and image-processing

algorithms have made computer-aided vision systems such as three-dimensional Digital Image

Correlation (3D-DIC), Structure from Motion (SfM), phase-based motion estimation (PME), and video

magnification (VM) appealing tool for SHM and NDI analyses.

In this paper, an overview of the research experience involving the use of vision and optically-based

techniques performed by the University of Massachusetts Lowell’s team in the last decade is presented.

It includes inspection of large-sized civil engineering structures such as concrete bridges experiencing

cracking and spalling, railroad tracks subjected to centre binding conditions, and utility-scale wind

turbine blades during static and dynamic tests. Also, the opportunity to couple vision-based techniques

with autonomous systems to expedite the capability of the measurement process while not interfering

with the structure’s operations is described.

The manuscript is organized as follow: Section 2 (Theoretical Background) introduces an outline of the

fundamental principles on which 3D-DIC, SfM, PME, and VM are based. An overview of some notable

case studies analysed in the recent past is presented in Section 3 (Application of Vision-Based

Techniques to Real World Structures), while conclusion are drawn and future works described in Section

4 (Conclusions).

2. Theoretical Background The scientific community has widely used optically-based measurement techniques for assessing the

condition of various aerospace, civil, and mechanical engineering structures. In this section, a review of

these systems is presented in three parts. The first summarizes the operational principles of 3D-DIC, the

second offers a brief overview of SfM, while the latter provides a description of how PME and VM

work.

2.1 Operational Principles of Three-Dimensional Digital Image Correlation

3D-DIC is a non-contact, full-field, optical measuring technique capable of extracting surface strain,

displacement, and geometry profiles from images acquired through a synchronized stereo camera pair

system to identify coordinates of points, features, and patterns of an object, and use this data to track

their motion thorough different times or stages. This technique was further developed and used for

deformation measurement in the early 1980s[13 - 15].

A stochastic pattern (e.g., black dots on a white background or vice versa) and/or optical targets have to

be applied to the surface of interest, and the relative position of each of them is tracked as the surface

deforms over time. Each image can be considered as a matrix of natural integers where white pixels

have a grayscale level equal to 0 and black pixels grayscale level 100. Since a single value is not a

unique signature point, neighboured pixels are used (i.e., facets). Each facet is a set of distinctive

correlation areas defined across the measuring region. The centre of each facet is a measurement point

that can be thought of as the edge of an extensometer. The position of these facets is tracked through

each of the successively acquired images, and the 3D coordinates of the entire area of interest are

Page 3: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

calculated. The fundamental principle of 3D-DIC is to match the same physical point between a

reference image and several deformed stages based on grayscale variations of continuous patterns[16].

2.2 Operational Principles of Structure from Motion

SfM is a technique that allows building a 3D description of a static scene from a dense sequence of

images[17]. This means that starting from a cloud of two-dimensional (2D) images. The 3D model can

then be used for performing visual inspection remotely, reducing downtime and costs associated with

the investigation. In particular, once the virtual model of the object being tested has been reconstructed

from the 2D images, an operator could “virtually” navigate it and detect any anomaly in the structure.

SfM is a photogrammetry technique based on finding “keypoints” between images. The keypoints are

defined as common points between two or more images by using algorithms for detecting features (e.g.,

corner points and edges with gradients in multiple directions). In practice, as several images of the same

object are taken from different points of view, a cloud of retinal data is generated[18]. By processing the

images, SfM algorithms look for common features from one image to the next. When two keypoints on

two or more different images are found to be the same, they are matched and will generate one 3D point.

As a general rule, it can be stated that the more keypoints there are, the more accurately 3D points can

be computed. Therefore, the recommended overlap for most cases is equal to 75% in the frontal direction

(i.e., frontal overlap) and at least 60% in the transversal direction (i.e., side overlap). For a detailed

description of the mathematical models and algorithms behind this technique, interested readers can

refer to paper[19]. SfM has been widely used for performing large-area aerial inspections in the field for

agriculture, geosciences, environmental disaster management, and heritage documentation [20-23]. The

quality of the generated 3D renderings is high and allows detecting features of the targeted object.

2.3 Operational Principles of Phase-Based Motion Estimation and Video Magnification

Recent advances in computer vision and computational photography have found numerous applications

in structural dynamics identification and structural health monitoring. Phase-based motion estimation is

one of the most recent computer-vision techniques that is able to reveal the small perturbations or

vibrations in videos [1]. The idea of phase-based motion magnification is closely related to the Fourier

transform shift theorem which indicates that any motion in the spatial domain will result to phase

variations in frequency domain and vice versa. Phase-based motion magnification decomposes each

frame of the video to a set of complex steerable filters. Then the phase variations in this complex

representation of images are temporally filtered and amplified by a band-pass filter in a specific

frequency band. Manipulating the phase variations will result to changes to the real motion of the video

in the spatial domain. Therefore, reconstructing the video with the new set of complex representation of

each frame will generate a motion-magnified video with amplified motions in the selected frequency

band of interest.

In structural dynamics, if the canter frequency of the band-pass filter is selected to only include one of

the resonant frequencies of the structure, the motion-magnified video will be representative of the

operating deflection shape of the structure, which is corresponding to the selected resonant frequency.

In the next section some the applications of phase-based motion magnification is available.

3. Application of Vision-Based Techniques to Real World Structures In this section, an overview of recent research performed by the University of Massachusetts Lowell’s

team about the possibility of using optically-based techniques for assessing the structural and dynamic

conditions of large-sized real-world structures. Case-studies described in this paper focus on concrete

bridges experiencing cracking and spalling, railroad tracks subjected to centre binding conditions, and

utility-scale wind turbine blades during static and dynamic tests. Also, the opportunity to couple vision-

based techniques with unmanned aerial vehicles (UAVs) to expedite the capability of the measurement

process while not interfering with the structure’s operations is described as well.

Page 4: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

3.1 Railroad Tracks Inspection and Assessment Using Combined 3D-DIC and SfM

Approaches

Track failures due to cross tie degradation or loss in ballast support may result in many problems ranging

from simple service interruptions to derailments. Optically based, non-contact measurement techniques

may be used for assessing surface defects, rail and tie deflection profiles, and ballast condition. The

design of two camera-based measurement systems is proposed for crossties-ballast condition assessment

and track examination purposes. The first one consists of four pairs of cameras installed on the underside

of a rail car to detect the induced deformation and displacement on the whole length of the track’s cross

tie using 3D DIC measurement techniques in a setup similar to that shown in Figure 2a[25]. The second

consists of another set of cameras using SfM techniques for obtaining a 3D rendering of the

infrastructure from a series of two-dimensional (2D) images to evaluate the state of the track

qualitatively (see Figure 2b)[26].

(a) (b)

Figure 2. Conceptual setup of the camera systems that can be used to observe the tie deflection

profiles attached to the underside of a rail car: (a) 3D-DIC approach[25]; (b) SfM approach[26].

In the tests, because of geometrical limitations due to the features of the train cars commonly used, the

working distance of the cameras (i.e., ~1 meter) allowed for measuring a crosstie area whose dimensions

correspond to nearly 56% of the distance between the two rails. For this reason, the field of view (FOV)

of the full crosstie has been obtained by stitching together the images recorded using four pairs of

cameras, in a setup similar to that shown in Figure 1a. The 3D-DIC system has been evaluated through

extensive experiments on a full-scale railroad model, which experiences displacement levels similar to

what is experienced in the field when a railcar passes over a section of the track.

(a) (b)

Figure 3. Out-of-plane vertical displacement for three different simulated centre binding conditions:

(a) full-field displacement counterplot; (b) crosstie longitudinal profiles[26].

As shown in Figure 3, test results showed that the 3D-DIC system could detect out-of-plane vertical

displacement with an accuracy of 10-4 m and characterize the differences in the crossties’ deflection

Page 5: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

profiles as various ballast conditions are simulated[27]. In this study, a projected pattern was also used to

quantify the deflection of the crossties to cope with the impossibility of applying a painted stochastic

pattern over an entire railroad track[26].

The second approach evaluates the performance of commercially available SfM software (i.e., Pix4D)

reconstruct a 3D model of the railroad, which can be virtually navigated to assess tracks’ health and to

measure the out-of-plane displacement of the crossties. One of the first outcomes of the tests was

performed on a trolley-line in Lowell, MA and shows that SfM is suitable in obtaining qualitative

evaluations of the overall condition of the track. The tests revealed some significant physical features,

proving the method’s ability to identify defects at a macroscopic scale (e.g., damaged crossties, missing

or failing fastening systems) as shown in Figure 4a and Figure 4b. For quantitatively validating the

accuracy of the performed SfM measurements, a back-to-back comparison with a 3D-DIC was

performed as the crosstie is experiencing different loading and centre binding conditions.

(a) (b) (c)

Figure 4. Results of the SfM tests: a) 3D rendering from images on a trolley line in Lowell, MA; b)

damaged crossties detail; c) comparison of the vertical displacement Z measured using the 3D-DIC

and SfM systems[27].

As can be observed from data reported in Figure 4c, the SfM approach can measure the zero

displacement condition with an accuracy within ± 1 mm, likewise it can detect with reasonable accuracy

the shape of the crosstie as it deforms. For small displacements, the accuracy of the SfM system deviates

from DIC measurement and this phenomenon could be because the displacements had the same order

of magnitude of the noise-floor of the SfM software (i.e., ~± 1 mm). It means that SfM cannot detect

displacement with the same accuracy as the 3D-DIC system (which has a noise floor on the order of 10-

5m). As the load is applied and the crosstie experiences higher displacements, the SfM detects

deflections with good accuracy. Errors on average are equal to 0.583 mm and 0.330 mm for the unloaded

and the loaded case respectively as a comparison with the higher accuracy system is performed.

3.2 Bridges’ Cracks Assessment Using Combined 3D-DIC-UAV and SfM Approaches

Bridges are among the most monitored civil structures due to their strategic importance and safety

requirements. Starting in 2013, researchers at the University of Massachusetts Lowell started using 3D-

DIC for periodic inspection of concrete bridges to locate non-visible cracks in concrete, quantify spalling,

and measure bridge deformation[28]. In their research, they used photogrammetric targets as

extensometers to track the opening of joints and cracks over a 4.5-month period and a stochastic pattern

to monitor the strain fields over two bridges in service. Follow those experiences; the researchers

proposed a system for providing accurate crack-displacement measurements, over several areas of

concrete bridges using a set of cameras installed on a UAV for remote data acquisition and performing

off-line 3D-DIC. By installing a stereovision camera system on a drone payload similar to that shown

in Figure 5, they demonstrated the accuracy of this system in detecting structural changes and monitoring

the dynamic behaviour of hairline cracks and expansion joints over time[29].

Page 6: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

In particular, several laboratory tests and an 11 month monitoring campaign on two currently in-service

bridges were performed to evaluate the potential capabilities of UAVs coupled with 3D-DIC

measurement techniques. The first set of experiments were designed to determine the system noise floor

in a controlled environment; while, the measurements executed over the two bridges were used for

validating the performance of the whole system over time in realistic conditions (e.g., identifying useful

capabilities of the designed stereovision payload and the effect of a moving UAV on the quality of the

recorded images, and lighting issues)[30]. The experimental evaluations executed for determining the

performance of the proposed 3D-DIC UAV system demonstrate its accuracy in measuring the evolution

of cracks having widths as small as 1x10-5 m, outperforming the resolution that can be obtained when

visual inspection techniques are employed. Preliminary results obtained using the developed technology

has shown that the proposed 3D-DIC UAV system can be used for complete scanning of walls, columns,

or other large areas of interest having complex geometry and reconstructing their shape and measuring

their full-field displacements. Larger FOVs can be obtained by stitching together adjacent sets of images

in a global coordinate system for when the UAV is deployed in different positions as has been

demonstrated in several other studies[27].

(a) (b)

Figure 5. Combined 3D-DIC UAV approach for bridges inspection: a) PSI InstantEye Gen4 Heavy

Lift Variant with affixed stereovision payload; b) UAV performing 3D-DIC inspection of target areas

under an in-service bridge in Lowell, MA[29].

Figure 6. In-plane X-displacement of a vertical hairline crack measured over a period of nearly 11

months tested on an in-service bridge in Lowell, MA[30].

Images acquired using cameras are also employed for reconstructing a digital 3D model of one of the

two bridges analysed in the study by Reagan et al.[30]. In particular, nearly 150 pictures were combined

using Pix4D, a commercially available software, to generate the SfM reconstruction of the structure.

The reconstructed model can be virtually “navigated” to highlight the presence of macroscopic defects

and also to quantify their severity as shown in the example reported in Figure 7.

Page 7: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

Figure 7. SfM rendering of an in-service bridge in Lowell, MA used for evaluating the severity of the

spalling phenomenon interesting the bridge’s abutment wall.

The results showed that this method is suitable for obtaining qualitative evaluations on the overall

condition of the bridge starting from the analysis of some major features. This could be particularly

useful for those areas difficult to access that may rely on the use of unmanned systems for inspection.

Outcomes of the research demonstrate the efficiency of the system in highlighting defects at a

macroscopic scale, while further steps have to be taken for increasing the accuracy of this technique for

large-sized civil structural monitoring.

3.3 Wind Turbine Blades Assessment using 3D-DIC, PME, and VM Approaches

Utility-scale wind turbine blades (WTBs) require proper certification to ensure that they meet the design

specifications and for safe operation throughout their design life. To couple with the problems associated

with traditional wire-based instrumentation (e.g., costs, time, auxiliary equipment, discrete measurement

points), 3D-DIC can be used to estimate the structural response and modal parameters of WTBs during

static and dynamic tests. Nevertheless, due to the limitations of a single stereo-vision system concerning

useful FOV of the cameras, a single camera cannot be used for obtaining information of the whole

structure. For this reason, a multi-camera based measurement system (i.e., two stereo-vision systems A

and B synchronized together) was implemented and experimentally evaluated to increase the covered

area in a setup similar to that shown in Figure 8. The multi-camera stereo vision system in conjunction

with data stitching algorithm enables the photogrammetry techniques to cover larger areas for expanded

measurements[31].

Figure 8. Concept setup of the pair of stereo-photogrammetry systems used to measure full-field

strain and modal parameters of a utility-scale WTB during dynamic tests.

In the study, point clouds obtained from stereo-vision A and B were then stitched together using the

corresponding points in the overlapping areas. By synchronizing the two cameras sets (each having a

field of view of nearly 4 m wide by 6 m long), it is possible to cover a wider area having dimensions of

approximately 4 x 11 m. The performance of the optical sensing system in identifying the modal

Page 8: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

parameters and the strain fields of the WTB was validated during dynamic tests performed on a ~60-

meter long WTB at the Wind Technology Testing Center (WTTC) in Charlestown, MA. Comparison

with data recorded using traditional wire-based sensors (i.e., strain gages and accelerometers) was

performed to quantify the accuracy of the optical method. From an analysis of processed data, it was

observed that the multi-camera system allows for measuring in-plane displacement as low as 10-3 m and

strain fields with an error below 5% when compared with traditional wired, metal foil strain gages.

Additionally, the system has been shown to be capable of extracting the full-field displacement and

strain over the entire analysed area of the utility-scale blade investigated. To finish, the system proved

to be capable of measuring the first five operating mode shapes with an error below 2% as compared

with traditionally wired accelerometers. For higher modes, still a proper evaluation is possible, but the

accuracy of the system decreases and miscorrelation in the mode shapes up to ~15% resulted.

(a) (b)

(c)

Figure 9. Evaluation of the multi-cameras system in assessing the condition of utility-scale WTB: a)

full-field strain in the X direction obtained from the combined images recorded by the multi-cameras

3D-DIC system; b) comparison of the strain measured with mounted strain-gages (blue lines) and the

recorded strain using the proposed approach using the multi-camera system (red line) for location

Stg_A2 on the surface of the blade; c) comparison of the displacements measured by stereo-vision A

and B at optical targets in the overlapping area.

Figure 10. First flap-wise bending operating deflection shape.

Page 9: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

A video of the same WTB during an impact test has been processed using the technique described in

Section 2.3 to visualize the operating deflection shapes (ODSs) associated with each of the identified

resonant frequencies. Figure 10 and 11 show the overlap of the two frames of the motion magnified

video at its maximum deflection shape at the first and second mode respectively. The two frames are

encoded with red and green colours, therefore the regions without motion will appear as yellow those

figures.

Figure 11. Second flap-wise bending operating deflection shape of a utility-scale wind turbine blade.

Similarly, Sarrafi et al. [32, 33] used PME and VM to perceive the operational deflection shapes and

dominant vibration patterns in a WTB cross-section. The estimated resonant frequencies are validated

by comparison with commercial accelerometers, and results show that optically-based techniques were

able to detect the first six modes with an error below 4%. Moreover, as the ODSs associated with each

of the identified resonant frequencies are reproduced from the original video using VM, it is possible to

observe the motion of the mode shape. An example of this phenomenon is shown in Figure 12, where

both the cavities of the test article are experiencing in phase expansion and contraction cycles in this

specific resonant frequency.

Figure 12. Two frames of the motion-magnified video of the WTB cross-section’s fifth ODS at its

maximum deflection shape[32].

4. Conclusions Monitoring the structural behaviour of large-sized structures plays is of great importance to industry and

society for economic, safety, and strategic reasons. Novel sensing techniques are continuously sought

that allow for easier, cheaper, and more accurate structural assessment. In the last decade, due to

improvements in computer-aided optical methods, techniques such as three-dimensional Digital Image

Correlation (3D-DIC), Structure from Motion (SfM), phase-based motion estimation (PME), and video

magnification (VM) have proved to be effective approaches for structural interrogation and

infrastructure inspection. In this paper, a few case-studies performed by the research team at the

University of Massachusetts Lowell are described with particular emphasis on structures like railroad

tracks, bridges, and wind turbine blades.

Node

Contraction

Contraction

Expansion

Expansion

Page 10: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

The first case study described focuses on the use of 3D-DIC and SfM for assessing the severity of centre

binding condition experienced by railroad crossties. 3D-DIC has proven to be capable of measuring the

full-field deflection of a crosstie with sub-millimetre accuracy. It also highlights the deflection shape of

the crossties as different intensity loads are applied. SfM is a useful tool for generating 3D models that

can be used for highlighting macroscopic defects and the deformed shape of the structure being tested,

but the level of accuracy is still too coarse for being used to quantify the severity of small deformations.

The second case study aims to describe how a 3D-DIC payload installed on an unmanned aerial vehicle

(UAV) can be used for long-term monitoring of crack activity on in-service concrete bridges. Results

showed that the proposed approach could measure cracks having a width on the order of 10-5 m,

outperforming the results that can be obtained as visual inspection is performed. To finish, the last case

study described focuses on the static and dynamic characterization of utility-scale wind turbine blades.

The results summarized in this paper show that optical techniques allow for obtaining full-field

displacement and strain profile as well as mode shapes and dynamic characteristics with accuracy

comparable to that of traditional wire-based systems and have the potential to reduce instrumentation

cost and time.

References

[1] American Society of Civil Engineers (ASCE). “2017 Report Card for America's Infrastructure”,

https://www.infrastructurereportcard.org/. (2017, accessed July 2018).

[2] D Balageas, “Introduction to structural health monitoring”, pp 13-43, 2010.

[3] H Sohn, CR Farrar, FM Hemez and JJCzarnecki, “A Review of Structural Health Review of

Structural Health Monitoring Literature 1996-2001”, Report. No. LA-UR-02-2095, 2002.

[4] M Wevers and K Lambrighs, “Applications of acoustic emission for SHM: A review”,

Encyclopedia of Structural Health Monitoring, 2009.

[5] S Gholizadeh, Z Leman and BTH Baharudin, “A review of the application of acoustic emission

technique in engineering”, Struct Eng Mech, 54(6), pp 1075, 2015.

[6] R Usamentiaga, P Venegas, J Guerediaga, L Vega, J Molleda and FG Bulnes, “Infrared

thermography for temperature measurement and non-destructive testing”, Sensors, 14(7), pp

12305-12348, 2014.

[7] K Vaghefi, RC Oats, DK Harris, TTM Ahlborn, CN Brooks, KA Endsley, and R Dobson,

“Evaluation of commercially available remote sensors for highway bridge condition assessment”,

J of Bridge Eng, 17(6), pp. 886-895, 2011.

[8] M Pieraccini, “Monitoring of civil infrastructures by interferometric radar: A review”, Sci World

J, 2013, ID 786961, 2013.

[9] BW Drinkwater and PD Wilcox, P. D, “Ultrasonic arrays for non-destructive evaluation: A

review”, Ndt&E INT, 39(7), pp 525-541, 2006.

[10] JL Rose, “Success and challenges for ultrasonic testing in NDT and SHM”, Mater Eval, 68(5), pp

494-500, 2010.

[11] JP Lynch and KJ Loh, K. J, “A summary review of wireless sensors and sensor networks for

structural health monitoring”, Shock and Vib Dig, 38(2), pp 91-130, 2006.

[12] A Sabato, C Niezrecki and G Fortino, “Wireless MEMS-based accelerometer sensor boards for

structural vibration monitoring: a review”, Sensors J, 17(2), pp 226-235, 2017.

[13] WH Peters, WF Ranson, “Digital imaging techniques in experimental stress analysis”, Opt Eng

21(3), pp 213427-213427, 1982.

[14] MA Sutton, WJ Wolters, WH Peters, WF Ranson, SR McNeill, “Determination of displacements

using an improved digital correlation method”, Image Vis Comput 1(3), pp 133-139, 1983.

[15] MA Sutton, JJ Orteu, H Schreier, “Image correlation for shape, motion and deformation

measurements: basic concepts, theory and applications”, Springer Science & Business Media,

New York, 2009.

[16] ZL Kahn-Jetter and TC Chu, “Three-dimensional displacement measurements using digital image

correlation and photogrammic analysis”, Exp Mech, 30(1), pp 10-16, 1990.

Page 11: Advancements in Structural Health Monitoring Using Vision ... · achieved in cameras technology, image -processing algorithms, and computer -aided vision systems, have made optically-based

[17] RC Bolles, HH Baker and D. H.Marimont, “Epipolar-plane image analysis: An approach to

determining structure from motion”, Int J Comp Vision 1(1), pp 7-55, 1987.

[18] DG Lowe, “Distinctive image features from scale-invariant keypoints,” Int J Comp Vision, 60(2),

pp 91-110, 2004.

[19] D Terzopoulos, A Witkin and M. Kass, “Constraints on deformable models: Recovering 3D shape

and nonrigid motion”, Artif Intell, 36(1), pp 91-123, 1988.

[20] M Pollefeys, D Nistér, JM Frahm, A Akbarzadeh, P Mordohai, “Detailed real-time urban 3d

reconstruction from video,” Int J Comp Vision, 78(2-3), pp 143-167, 2008.

[21] D Turner, A Lucieer, and C Watson, “An automated technique for generating georectified mosaics

from ultra-high resolution unmanned aerial vehicle (UAV) imagery, based on structure from

motion (SfM) point clouds”, Remote Sens-Basel 4(5), pp 1392-1410, 2012.

[22] MJ Westoby, J Brasington, NF Glasser, MJ Hambrey and JM Reynolds, “Structure-from-

Motion’photogrammetry: A low-cost, effective tool for geoscience applications,” Geomorphology

179, pp 300-314, 2012.

[23] S Jay, G Rabatel, X Hadoux, D Moura and N Gorretta, “In-field crop row phenotyping from 3D

modeling performed using Structure from Motion,” Comput Electron Agr 110, pp 70-77, 2015.

[24] D. Fleet and Y. Weiss, “Optical flow estimation” Handbook of mathematical models in computer

vision, Springer, pp 237-257, 2006.

[25] A Sabato and C Niezrecki, “Feasibility of digital image correlation for railroad tie inspection and

ballast support assessment”, Measurement, 103, pp 93-105, 2017.

[26] A Sabato and C Niezrecki, “Evaluation of using Structure from Motion Optical Techniques for

Structural Health Monitoring of Railroad Track” AIWSH Conf. Proc., 2017.

[27] A Sabato, CH Beale and C Niezrecki, “A novel optical investigation technique for railroad track

inspection and assessment”, SPIE Nondestructive Characterization and Monitoring of Advanced

Materials, Aerospace, and Civil Infrastructure Conf. Proc. 10169, pp 101692C, 2017.

[28] C Nonis, C Niezrecki, TY Yu, S Ahmed, CF Su and T Schmidt T, “Structural health monitoring

of bridges using digital image correlation”, SPIE Health monitoring of structural and biological

systems Conf. Proc. 8695, pp. 869507, 2013.

[29] D Reagan, A Sabato and C Niezrecki, “Unmanned aerial vehicle acquisition of three-dimensional

digital image correlation measurements for structural health monitoring of bridges”, SPIE

Nondestructive Characterization and Monitoring of Advanced Materials, Aerospace, and Civil

Infrastructure Conf. Proc. 10169, pp 1016909, 2017.

[30] D Reagan, A Sabato and C Niezrecki, “Feasibility of using digital image correlation for unmanned

aerial vehicle structural health monitoring of bridges”, Struct. Health Monit, 1475921717735326,

2017.

[31] P Poozesh, J Baqersad, C Niezrecki, P Avitabile, E Harvey and R Yarala, “Large-area

photogrammetry based testing of wind turbine blades” Mech Syst Signal Pr 86, pp 98-115, 2017.

[32] A Sarrafi, P Poozesh, C Niezrecki and Z Mao, “Applying video magnification for vision-based

operating deflection shape evaluation on a wind turbine blade cross-section”, SPIE Health

Monitoring of Structural and Biological Systems XII Conf. Proc. 10600, pp 106000M, 2018.

[33] P Poozesh, A Sarrafi, Z Mao, P Avitabile and C Niezrecki, “Feasibility of extracting operating

shapes using phase-based motion magnification technique and stereo-photogrammetry”, Journal

of Sound Vib, 407, pp 350-366, 2017.