Estimation of Oil Thickness

7/26/2019 Estimation of Oil Thickness

1/41

AEAT-5279 Issue 1

Estimation of oil thickness

A report produced for the Maritime and Coastguard

Agency

Louise Davies

Jenny Corps

Tim LunelKaren Dooley

November 1999


2/41

AEAT-5279 Issue 1

Estimation of oil thickness

A report produced for Maritime and Coastguard

Agency

Louise Davies

Jenny Corps

Tim LunelKaren Dooley

November 1999


3/41

- ssue Estimation of Oil Thickness

AEA Technology ii

Title Estimation of oil thickness

Customer Maritime and Coastguard Agency

Customer reference MSA 10/9/111

Confidentiality,

copyright and

reproduction

Copyright AEA Technology plc 1999

All rights reserved.

Enquiries about copyright and reproduction should be addressed to the

Commercial Manager, AEA Technology plc.

File reference EERA 20738001

Report number AEAT-5279

Report status Issue 1

AEA Technology Environment

National Environmental Technology Centre

Culham

Abingdon

Oxfordshire

OX14 3ED

Telephone +44 (0)1235 463117

Facsimile +44 (0)1235 463030

AEA Technology is the trading name of AEA Technology plc

AEA Technology is certificated to BS EN ISO9001:(1994)

Name Signature Date

Author Louise Davies

Reviewed by Tim Lunel

Approved by Jenny Corps


4/41


AEA Technology iii

Executive Summary

This project was undertaken on behalf of the Maritime and Coastguard Agency to devise a method

of measuring the total volume of oil on the sea surface using airborne remote sensing techniques.

This would provide considerable information for those planning a counter pollution response as itwould allow near real time estimates of the change in oil volume over time as a result of natural

processes and response actions. Such a response tool would revolutionise oil spill response by

providing a real time measure of the success of the response technique employed.

Thermal infra-red (IR) is able to detect a different thermal response from the heterogeneous (non-

uniform) thickness of an oil slick spreading on the sea surface. However, the relationship between oil

thickness and thermal response is complex. To overcome the complexities of the inter-relationships,

the National Environmental Technology Centre (NETCEN) developed a method of analysing IR

data from experimental oil slicks in the North Sea using neural network processing (Wood et al,

1997). This work demonstrated that it was possible to use a neural network approach to predict the

oil thickness of the test data set on the basis of the IR response. This initial development project was

followed with a validation exercise carried out by NETCEN (Davies et al, 1998). The 1998 project

concluded that the neural network was unable to classify oil thickness reliably for the validation data

set.

This report details the re-calibration of the neural network to establish whether it has the ability to

predict oil thickness from a wider range of input variables taking account of factors such as solar

heating of the surface oil slick. The neural network was retrained with two data sets: a primary data

set containing only field trial data from 1994, 1995 and 1997 with oil thickness measured by pad

sampling, and a secondary data set containing the primary data set and a Sea Empress data set with

estimated slick thickness. The secondary data set included thermal IR imagery and parameters from

a wider range of environmental conditions that represent both winter and summer.

The re-calibrated neural network was subjected to a verification process through testing with a

selection of data from each of the data sets that had not been used to re-train the network. This

verification process has established that the neural network is internally consistent and is able to

predict oil thickness representative of the data sets used to train the network in 50-75% of cases,

even under a wide range of environmental conditions.

A committee of networks approach was able to correctly assign the thickness of the slick into one of

four ranges of slick thickness in 76% of cases. In the majority of the misclassifications, the thicknesswas assigned to an adjacent thickness range. Whilst these results are promising, it must be noted that

the number of data available for training the network is relatively small compared to the number of

variables in the network and the network is over-fitted. Therefore the networks derived during this

project should not be considered as being sufficiently robust to be relied upon for operational use at

present. In addition it is important to emphasise that there is no guarantee that the data sets being

used to re-train the network have covered the complete range of the parameters (such as the

meteorological conditions and oil type) which will be encountered in the event of a spill. The


5/41


AEA Technology iv

importance of using a training data set which includes the full range of parameter values was

emphasised by the fact that the network trained solely with the primary data set (summer field trial)

was not able to predict the thickness for the Sea Empress imagery taken during the winter.

However, the results are sufficiently encouraging, we believe, to develop a pilot system to deploy as

an experimental system at the next real incident or experimental field trial. This will allow the

technique to continue development at every available opportunity while providing an additional tool

to the MCA in the event of an oil spill.

To develop the current experimental system into a pilot operational system would require the

following relatively minor developments:

Collaboration with Air Atlantique to standardise on image analysis software (for example, using

ER Mapper) and to devise procedures for downloading the thermal IR imagery to a suitable

format for analysis.

Determine suitable techniques for collecting all the variables required by the neural network, i.e.

air and sea temperature, irradiance.

Develop a sorbent pad sampling technique that includes real-time analysis for calibration of the

neural network.

We believe that this pilot system could be developed within one month. If the pilot system is

developed and deployed (potentially using the FLIR system on the SAR helicopters as well as the

MCAs remote sensing planes) at future incidents then the system can be used to give estimates of

volumes, alongside the normal visual estimates, for operational use. The pilot system can also be

evaluated in terms of the practicalities of operating the system and the benefits it provides to MCA in

an oil spill incident.

In addition, the neural network could be continually re-trained with the new sets of field data taken indifferent environmental conditions to maximise the chances of the operational data falling in the range

of environmental parameters used in the training process. This will be particularly useful if pad

sampling is used to provide in-situ calibration of the slick thickness.


6/41


AEA Technology v

Contents

1 Introduction 1

2 Thermal Infra-red 2

2.1 THERMAL IR CAMERA 2

2.2 THERMAL BEHAVIOUR OF SPILLED OIL 2

2.2.1Thin Oil Layers 3

2.2.2Thick oil layers 3

2.2.2Heat exchange and surface temperature effects 4

3 Previous Neural Network Research Programmes 5

3.1 INITIAL DEVELOPMENT OF NEURAL NETWORK SYSTEM 5

3.2 NEURAL NETWORK VALIDATION 8

4 Research Programme 9

5 Data Collation 10

5.1 DATA SELECTION 10

5.2 TRAINING PARAMETERS 11

5.3 PRIMARY DATA SET 135.4 SECONDARY DATA SET 15

6 Neural Network Re-Training 17

6.1 REGRESSION APPROACH 18

6.2 CLASSIFICATION APPROACH 19

6.3 SUMMARY 21

7 Neural Network Verification 22

7.1 PRIMARY DATA SET 227.2 SECONDARY DATA SET 23

7.3 COMMITTEE OF NETWORKS 25

7.4 SUMMARY 28


7/41


AEA Technology vi

8 Conclusions 29

9 Recommendations 30

9 References 33

Appendices

APPENDIX 1 PRIMARY AND SECONDARY DATA SETS

VERIFICATION RESULTS FOR SEA EMPRESS DATA USING

NEURAL NETWORK TRAINED ON PRIMARY DATA

THERMAL IR IMAGE OF 1997 FORTIES BLEND EXPERIMENT


8/41


AEA Technology 1

1 Introduction

The ability to measure the total volume of oil on the sea surface using airborne remote sensing

techniques would provide considerable information for those planning a counter pollution response.This would allow near real time estimates of the change in oil volume over time as a result of natural

processes and response actions. Such figures would also be invaluable in quantifying the success of

the response technique employed. For example, a quantification of the volume of oil remaining on

the sea surface before and after a test run of dispersant spraying would enable the responder to

rapidly assess whether the spilt oil is responding to the dispersant under the given environmental

conditions.

Thermal infra-red (IR) is able to detect a different thermal response from the heterogeneous non-

uniform thickness of an oil slick spreading on the sea surface. However, the relationship between oil

thickness and thermal response is complex. To overcome the complexities of the inter-relationships,

the National Environmental Technology Centre (NETCEN) developed a method of analysing IR

data from experimental oil slicks in the North Sea using neural network processing on behalf of the

Maritime and Coastguard Agency (Wood et al, 1997). The 1997 work demonstrated that it was

possible to use a neural network approach to predict the oil thickness of the test data set on the basis

of the IR response.

This initial development project was followed with a validation exercise carried out by NETCEN and

sponsored by the Maritime and Coastguard Agency (Davies et al, 1998). This exercise consisted of

a set of field experiments to collect the data that was subsequently used to test the validity of the

trained neural network. This project concluded the neural network was unable to classify oil

thickness reliably for the validation data set.

It was also concluded that the failure of the neural network was mainly due to the data being outside

the data range the network was originally trained on for a number of the input variables. The neural

network was also unable to account for the thermal effects of the oil as a result of solar heating of

the surface oil during this set of field trials. The thermal effects in the IR imagery were not observed

in the field data sets used to train the network.

In the report by Davies et al (1998) it was suggested that it might be possible to re-train the neural

network with the new 1997 which included solar heating effects. However it was emphasised in the

conclusions that there was no guarantee that even with this new additional data set that the full rangeof environmental variables would be taken into account.

This report details a study involving the re-calibration of the neural network with all the data sets

obtained during the initial training of the neural network (Wood et al, 1997) and the validation

exercise (Davies et al, 1998). The data used in the re-calibration includes additional core variables

that were considered important for predicting the effect of solar heating on the thermal IR imagery

during the validation exercise. These additional core variables were not previously included as core

variables in the initial training programme. The re-calibrated neural network was subjected to a


9/41


AEA Technology 2

verification process through testing with a selection of data that had not been used to re-train the

network. This verification process has established whether the neural network is internally consistent

and is able to predict oil thickness representative of the data sets used to train the network. This

process does not provide a validation of the network, since the verification data set is a subset of the

training data set.

2 Thermal Infra-red

2.1 THERMAL IR CAMERA

A thermal IR camera, such as the Talytherm camera in the Air Atlantique Cessna 404, detects

thermal IR radiation in the 8-13 m range. The detector measures apparent black body

temperature, which is a function of actual temperature (expressed as C) modified by the emissivity

of the material that is omitting the IR radiation. Emissivity is the property of a material that indicates

the proportion of thermal IR emitted by the material when it is at a particular temperature.

The system detects very small differences in thermal IR emission. For display on a video output, the

signal is amplified (the degree of amplification is expressed as gain) and the signal base-level is

expressed as offset. The gain and offset may be adjusted automatically or set manually. Changing

the offset will alter the response of the camera image to temperature; manually setting the sea to a

mid-grey on the screen establishes the prevailing sea temperature as being in the middle of the

detector range. Increasing the gain causes maximum discrimination in the image. The brightness of a

recorded thermal IR image is therefore a function of the objects temperature, but is an indication of

relative and not actual temperature.

2.2 THERMAL BEHAVIOUR OF SPILLED OIL

A thermal IR image of an oil slick is a representation of apparent temperature differences in the oil

slick and these can give an indication of relative oil layer thickness. An oil surface at the same

temperature as a water surface will have a lower equivalent black body temperature than the water

surface. This will cause the oil to appear cooler in the thermal IR image, although it is at an identical

temperature as the water surface. On the basis of the difference in emissivity values of oil and water,

oil should always appear cooler than water when both are at the same temperature. However, oilslicks tend to appear as two distinct areas in a thermal IR image:

Areas that are apparently cooler than the sea.

Areas that are apparently warmer than the sea.

The apparently cooler areas of the slick are areas of relatively thin oil, from 50 m up to

approximately 250 to 500 m thick and the warmer areas are thicker or emulsified oil, greater than

approximately 250 to 500 m (Goodman, 1994).


10/41


AEA Technology 3

2.2.1 Thin Oil Layers

Thin layers of oil will be in intimate thermal contact with the sea surface. The thermal capacity of a

thin oil film will be very low because there is not much oil per unit area. It is therefore unlikely that the

oil could be at a significantly different temperature from the sea for any length of time before the

temperature equilibrates. Under most circumstances, the thinner layers of oil are likely to be at almost

exactly the same temperature as the sea and appear cooler in the thermal IR image because of the

differences in emissivity between oil and water. There are two factors that may slightly alter the

actual temperature of thin oil films:

In the very early stages of oil weathering, the rapid evaporation of large quantities of volatile

components from the oil layer may cause cooling, leading to the oil film being at a lower

temperature than the sea for a very short period.

Layers of oil of any significant thickness are likely to absorb solar radiation more effectively than

the surrounding sea because they are opaque and dark coloured. This could cause a slight

increase in temperature of the oil film.

Evaporative cooling and solar heating will tend to cancel each other out some time after the oil has

been released onto the sea. If the oil layer is being exposed to solar heating, the rise in temperature

may still be insufficient to cause the oil to appear warmer than the sea in the thermal IR image

because of the effect of emissivity.

2.2.2 Thick oil layers

While the thinner layers of oil are essentially in thermal equilibrium with the sea, the areas of thick oil

or emulsion will not be in thermal equilibrium with the sea and can have a higher temperature. The

higher temperature of the thicker layers of oil will be due to the balance of heat transfer into and out

of the spilled oil layer. The precise temperature rise will depend on several factors:

The temperature rise produced by constant heat input will depend on the thermal capacity of the

oil layer, which is proportional to its volume per unit area (thickness). The actual temperature of

an oil layer is the result of the balance between heat input from solar radiation and heat lost to the

air or sea. Thicker oil layers are therefore capable of sustaining a higher temperature difference

than thinner oil layers.

The thermal capacity of an oil layer depends on its mass per unit area (thickness) and its specific

heat capacity. Oil has a specific heat that is about half that of water. As oil emulsifies its specific

heat will increase in proportion to the water content. Emulsified oil is therefore capable of holding

more heat than non-emulsified oil.

The heat input and output is a result of temperature differences between the oil layer, the sea and

air, and the degree of solar heating. Thick layers of oil or emulsion will absorb heat from the sun


11/41


AEA Technology 4

more effectively than the sea because the oil is opaque or of a dark colour. This will cause a

significant rise in temperature if the rate of heat input is greater than the rate of heat loss to the sea.

2.2.2 Heat exchange and surface temperature effects

Unless the sea and air are at precisely the same temperature there will be heat transfer between

them. If spilled oil is present it will act as a buffer to this heat exchange. The oil will adsorb heat

from the warmer of the two and lose heat to the cooler. The thermal capacity of the oil layer is

minuscule compared to that of the sea and atmosphere and it is inevitable that the temperature of the

oil layer will change. The temperature differentials between air, sea and oil plus the relative thermal

conductivities of the system will determine the rate of change of temperature of the oil layer.

The sea surface temperature will tend to remain almost constant during the period of a day, due to its

massive thermal capacity. However the oil layer temperature will oscillate; the thick layers of oil

warm to quite high temperatures during the course of the day as heat is absorbed from the sun and

cool slowly at night as heat is lost to the sea and air. The oil will be cooler than the sea in the early

morning, but rise to a temperature higher than that of the sea during the day. At nightfall the

temperature of the oil will start to drop, but it will still be higher than that of the sea for some time in

the evening. During the night it will lose heat to the cooler air which is not replaced by solar heating,

and the temperature of the oil will then drop below that of the sea. Therefore typically the greatest

extent of thick oil is detected in imagery recorded around midday to early afternoon when it is the

warmest part of the day.

Similar effects will occur on a shorter time-scale if there is intermittent sunlight or cloud shadow

effects. The oil will heat up rapidly in the sun, but start to cool when the sun is obscured. The major

determinants of the heating and cooling of the oil layers will be the intensity and duration of sunlight

and the relative temperature differences of the air and sea.

The temperature attained by an emulsified oil layer will be proportional to its thickness under the

prevailing conditions; thinner layers of oil will lose heat more rapidly, and therefore have a lower

temperature, than thicker layers. However, this proportionality only holds true for a specific set of

conditions and the actual temperature attained by the oil in thicker layers will depend on the heating

and cooling effects that have occurred previously, rather than those that prevail at the time of

observation.

In a thermally equilibrated system with no heat exchange, surface temperature will be a very good

indicator of bulk temperature as heat will be conducted to the surface from the bulk of the oil.

However, in situations where the temperatures are constantly changing, such as the dynamic heatexchange (heat input and output) which gives rise to temperature differences in spilled oil layers, the

surface temperature might not be a direct indicator of bulk temperature. In these circumstances, the

IR brightness (or relative equivalent black temperature) as measured by a thermal IR camera, will be

an indication of the surface temperature of the oil layer and not of the internal temperature. This can

occur if there are large differences in sea and air temperature. Heat transfer from the inside of the oil

layer to the surface might be slower than heat transfer from the surface of the layer to cold air. The

surface of the oil layer may be chilled by extremely cold air to lower temperatures than that of the

sea, even though the internal temperature of the oil is higher due to previous heat input.


12/41


AEA Technology 5

3 Previous Neural Network Research

Programmes

The National Environmental Technology Centre (NETCEN), have completed two previous research

projects relating the response of thermal IR to oil slick thickness on behalf of the Maritime and

Coastguard Agency. The initial research project developed the neural network approach to

measuring oil slick thickness (Wood et al, 1997) and the following research project validated the

neural network developed in the initial project (Davies et al., 1998).

3.1 INITIAL DEVELOPMENT OF NEURAL NETWORK SYSTEM

Neural networks are computing methods that learn from empirical data rather than being

programmed explicitly. Neural networks are particularly useful in the modelling of complex

relationships where non-linear effects could be involved. Instead of being programmed with a series

of equations, the neural network can be trained to learn a cause-effect relationship. During training,

examples of input and output are fed to the network, which learns the relationships by building up

interconnecting paths in an iterative training process.

A range of object-oriented neural network software exists, some of which are appropriate for the

analysis of IR imagery data. For this type of problem, a neural network may be regarded as a non-

linear regression tool. It predicts output values from a number of inputs. The inputs are weighted in

a similar manner to linear regression, and then passed to a set of non-linear basis functions (each

with a different set of weights). Then a least squares fit is performed on the outputs of the basis

functions.

The initial experimentation (Wood et al., 1997) has trained a Multi-Layer Perceptron (MLP) neural

network architecture (Figure 1). The MLP consists of a number of interconnected layers of

processing units. The first layer (left on the diagram) is called the input layer. Each circular node

represents an input to the network. Each input is a single number. The processing units in the other

layers all perform a similar function, which is to take a weighted sum of the outputs from the

previous layer, add a bias term, and apply an activation functionto the result. In the diagram, the

weights are represented by the connections between the circular nodes.

The data set used to develop the neural network was collected in experimental field trials carried out

in 1994 and 1995 by NETCEN. The inputs to the neural network included the thermal IR response

(brightness of the oil and sea in the imagery), oil type, time of day the data was recorded and various

meteorological parameters. The resulting oil thickness predictions were compared to measured oil

thickness values which were obtained in the field through sorbent pad sampling.


13/41


AEA Technology 6

Figure 1. Schematic Diagram of an MLP

The initial development work carried out by NETCEN (Wood et al., 1997) suggested an optimal

configuration for prediction of oil thickness, which is shown in Figure 2. Bias weights are omitted for

clarity.

Figure 2. Optimal Configuration for Prediction of Oil Thickness

B

Bs

cos (Time)

sin (Time)

Oil T e

Wind Speed

Sea Temp

log( )

brightness of oil B

brightness of sea + oil Bsobrightness of sea Bsoil thickness Note that B B Bso s= .

This optimum configuration was then used to predict oil thickness on a proportion of the field data

set from the 1994 and 1995 experimental field trials. Figure 3 shows the typical performance of the


14/41


AEA Technology 7

network; predicted oil thickness is plotted against the actual oil thickness. The straight line is the line

y=xand corresponds to a perfect prediction. The R2value is 0.846.

Figure 3. Scatter Plot of Predicted Against Actual Thickness for Optimal Configuration

a

-0.5 0 0.5 1 1.5 2 2.5 3 3.5

Actual log( )

-0.5

0

0.5

1

1.5

2

2.5

3

Predicted log( ) R

= 0.846221

Additionally, a classification approach was used to categorise the data into four thickness ranges:

0-100 m

101-500 m

501-1000 m

1001-2500 m.

Table 1 presents typical results from the combined data set obtained in analysing the previous sea

trial data sets (Wood et al. 1997). When all available sea trial data were used to train the network it

was possible to classify correctly 80 % of data into one of four thickness classes for the previous

sea trial data sets.

Table 1. Confusion Matrix for Combined Sea Trial Data Set

Bin 0-100

m

101-500

m

501-1000

m

1001-2500

m

Correct

%

0-100 m

136

3

0

1

97.1%

101-500 m 8 15 3 0 57.7%

501-1000 m 2 1 7 0 70%

1001-2500 m 0 1 0 8 88.9%


15/41


AEA Technology 8

Once the initial neural network system was completed, it was recognised that the validity of this

approach had to be tested against data sets which were totally unrelated to the data sets used to

initially train the network. This led to the validation exercise as described in the following section.

3.2 NEURAL NETWORK VALIDATION

The validation exercise consisted of collecting a new data set though an experimental field trial. This

was carried out in 1997 by NETCEN, a data set was obtained containing thermal IR imagery and

meteorological parameters for input into the neural network trained in the initial project. Oil

thickness was also measured during the field trial using sorbent pad sampling for comparison to the

predicted thickness values. The data sets collated from the 1997 field trial for the validation of the

neural network were processed by the neural network in order to obtain predicted thickness values

(Davies et al., 1998).

As previously stated, the core input variables required by the neural network trained in 1997 were :

Thermal infrared sea and oil brightness Time of day

Wind speed

Sea temperature

Oil type

The 1997 field trial data set consisted of data collected from three oil types; Forties Blend crude oil,

Alaska North Slope crude oil, and IFO-180 heavy fuel oil. The network was originally developed

with data from the 1994 and 1995 field trials obtained from Medium Fuel Oil and Forties Blend

crude oil experiments. The neural network requires specification of oil type so the 1997 Forties data

set was analysed using only the trained network for Forties. The data sets from the Alaska NorthSlope Crude oil slick and IFO-180 were processed by the network with oil type as both MFO and

Forties.

Using these input data, the neural network analysis was unable to accurately predict the measured oil

thickness. For example, the data set from the experimental Forties blend slicks showed that

measured slick thickness data were representative of all four thickness categories. Yet the neural

network analysis categorised all the thickness measurements into just two, non-consecutive,

categories 0-100 m and 501-1000 m. Further, the categorisation was purely a function of time:

before midday all the IR data was categorised into 0-100 m despite a measured thickness of 336-

2625 m; and after midday all the data was categorised into 501-1000 m despite a measuredthickness of 139-2359 m.

The poor performance of the neural network classification was likely to be due to the degree of solar

heating of the surface oil during this set of field trials which was not observed in the field data sets

used to train the network. In addition, in the training data sets the air temperature was above the

seawater temperature in nearly all cases whereas in the validation data set, air temperature was

always below sea temperature thereby reversing the net thermal flux.


16/41


AEA Technology 9

4 Research Programme

The validation exercise demonstrated that the neural network trained in 1997 was incapable of

reliably predicting oil slick thickness. Therefore, a research programme was devised to re-calibrate

the neural network using both the initial training data set and the validation data set. This would

indicate whether the neural network is able to predict oil slick thickness from a more varied data set.

The re-calibrated neural network would then be tested with a subset of the data that was not used

for training as a verification of whether the neural network is internally consistent and therefore has

the ability to predict oil slick thickness.

The research programme was divided into three stages;

1. Data collation

2. Neural network re-training

3. Neural network verification

The data collation stage involved collating all the data sets that had been used in the previous two

neural network research projects. This was followed by a data selection procedure to ensure that

the data used to retrain the neural network was as accurate and robust as possible. This involved re-

examination of the thermal IR imagery and the thickness measurements. Specific meteorological

factors were selected that were considered essential to the prediction of the oil thickness, i.e.

emissivity, irradiance, air and sea temperature. Once the final data set was complete, a proportion of

the data was removed for the verification stage and the remaining data set was used to re-train the

neural network.

The neural network was re-trained using the brightness measurements from the thermal IR imagery,

the thickness measurements and the selected meteorological parameters. The optimum neural

network settings for the data set were devised based on the work undertaken in the initial training of

the neural network. Once the neural network was trained to the best output for the data set, the

verification stage was undertaken.

A spreadsheet was developed to enable the trained network to be tested using the verification data

that was not used for the training. The network was then tested and the results examined todetermine the accuracy of the neural network.

Each of these stages is described in more details in the following three sections.


17/41


AEA Technology 10

5 Data Collation

There are two sub-sections of data that have been used in the neural network research programmes;

the 1994 and 1995 sea trial data, which was used to train the original neural network, and the 1997

sea trial data which was used to validate the trained neural network.These combined data sets consist of a total of 225 data observations from four oil types (Alaska

North Slope crude oil, Forties Blend crude oil, Medium Fuel Oil/Gas Oil blend and IFO-180 Heavy

Fuel Oil). The data sets contain the following parameters for each data observation:

1. Oil slick thickness (as measured by sorbent pad sampling)

2. Thermal IR response (brightness values)

3. Altitude of aircraft

4. Time of day

5. Camera offset and gain6. Oil type

7. Wind speed and direction

8. Air and sea temperature

9. Irradiance

10.Surface current (at 1 metre depth)

All of these parameters were studied in the initial research programme and it was determined that

many of these parameters have no effect on the prediction of oil thickness using the thermal IR

brightness data. The studies undertaken as part of the earlier validation of the neural network

(Davies et al, 1998) highlighted the importance of solar heating on the prediction of oil thicknesstherefore certain parameters were identified as essential to predict the changes in the thermal IR

imagery as a result of heating. This is discussed further in Section 4.2.

These data sets have undergone different levels of data processing and interpretation. The latest data

set, from the 1997 sea trial, is the most accurate and robust as a result of the continuing development

of our data processing and sampling techniques. Therefore it has been vital to the retraining

programme that all the data sets were reassessed to ensure all the data used in the retraining was as

accurate and robust as possible.

5.1 DATA SELECTION

The data selection process was aimed at reducing the likelihood of data being included in the

retraining which represents a false relationship between the oil thickness and IR response.

The first stage of the data selection procedure was to re-calculate the thickness measurements made

in 1994 and 1995 using sorbent pad samples to ensure that the increased thickness of emulsions

was considered.


18/41


AEA Technology 11

The data sets were then subjected to a range of data selection criteria, listed as follows:

1. Thermal IR imagery must have been obtained at an altitude of less than 3000ft (as the resolution

of the image reduces significantly with increasing altitude).

2. Removal of spurious data when:

The brightness value is taken from a poor quality image

The brightness value is not taken in the vicinity of the sampling vessel.

The thickness sampling position is unknown.

3. Data sets of a particular oil type, which because of the nature of the field experiment, over-

represented data of particularly low thickness value were reduced in number to prevent the data

from biasing the training of the neural network toward these very low thickness values.

This data selection process reduced the number of data observations from 225 to 96. The majority

of data removed was that obtained during the 1994 sea trials during the Medium Fuel Oil/Gas Oil

blend continuous release trials, where the data obtained consisted of thickness measurements of less

than 10 m for more than 80 data observations. This level of thickness is difficult to accurately

measure with sorbent pad samples and is at the limit of detection for the thermal IR camera.

Therefore, the relationship between the thermal IR brightness, oil thickness and other variables may

not be represented as accurately as the data observations taken at higher oil thickness levels. So by

including such a large number of samples of less than 10 m, this may have biased the training of the

neural network toward the low oil thickness values, which are of little interest in operational

response. The majority of the other data observations removed were as a result of low resolution

(high altitude) and poor quality thermal IR imagery.

5.2 TRAINING PARAMETERS

The neural network was trained in the initial research programme (Wood et al., 1997) using the

following key parameters:

Brightness of oil

Brightness of oil and sea

Time

Oil type

Wind speed

Sea temperature

During the validation studies in the previous research project, it was observed that there were further

parameters that may be important in predicting oil thickness from thermal IR imagery. It was

observed in the validation data that the extent of solar heating significantly affected the brightness of

the thermal IR imagery, therefore it would be important to include irradiance and air temperature

data as key parameters in the neural network.


19/41


AEA Technology 12

Thermal IR emission is related to both the way a surface emits IR radiation as a function of

temperature and the actual temperature of the surface. This relationship is described by the

emissivity, therefore emissivity is an important parameter when studying thermal IR response. The

emissivity values for the oils were not included in the data sets used to train the original neural

network. Therefore the inclusion of emissivity data as a parameter in the re-training was considered

essential to improve the accuracy of the thickness predictions.

The emissivity value for each of the four oils in the data set was obtained and is detailed as follows.

The directional reflectance of the oil was measured over a wavelength range of 2-25 m. The

thermal IR camera operates over a range of 8-13 m, so the emissivity values over this range were

noted and the average emissivity value taken for each oil type. The emissivity values are shown in

the following table.

Table 2. Emissivity Values for the Oils.

Wavelength

(m)

Alaska North

Slope crude oil

Forties Blend

crude oil

Medium Fuel

oil/Gas oil blend

IFO-180 Heavy

Fuel Oil

8 0.957 0.963 0.960 0.953

9 0.958 0.964 0.960 0.953

10 0.957 0.963 0.960 0.953

11 0.957 0.963 0.960 0.953

12 0.957 0.963 0.959 0.952

13 0.956 0.962 0.958 0.951

Average 0.957 0.963 0.960 0.953

The parameters selected for re-training the neural network are as follows.

1. As used in the original neural network;

Brightness of oil


Sea Temperature

2. Additional core variables identified during the validation exercise; Emissivity

Irradiance

Air Temperature

The additional core variables in the current training parameters were identified during the validation

exercise.


20/41


AEA Technology 13

It is important to keep the number of key training parameters to a minimum to reduce over-fitting in

the neural network training. Note that the emissivity values replaced the oil type input variable in

the neural network. Also the time of day and wind speed parameters have been excluded as these

are considered to have a lesser effect on the brightness and the effects are accounted for through the

other parameters (irradiance, sea and air temperature).

The change in the parameters selected for re-training the neural network in comparison to the

previous training are summarised in the following table.

Table 3. Previous and current training parameters

Previous training parameters Current training parameters

Brightness of oil Brightness of oil

Brightness of oil and sea Brightness of oil and seaSea temperature Sea Temperature

Oil type Emissivity

Time of day Irradiance

Wind speed Air Temperature

5.3 PRIMARY DATA SETAfter the data had undergone the data selection procedure and the key training parameters were

selected, the data, labelled the primary data set, was divided into a training data set and a

verification data set. The selection of the verification data set was carried out by taking a

representative sample from each thickness prediction range to:

Obtain a verification data set that was representative of a range of oil thickness with different oil

types, at different temperatures, different brightness ranges and different irradiance levels.

Ensure the input variables for the verification data were within the limits of the training data set.

The following table details the total number of data observations in each of the thickness classification

ranges and the amount of data taken for training and testing the neural network.

Table 4. Number of thickness observations in each prediction range for the primary data set.

Thickness Range TOTAL Training Verification


21/41


AEA Technology 14

0-100 28 23 5

101-500 33 28 5

501-1000 14 11 3

>1001 21 18 3

TOTAL 96 80 16

This table can be divided into each oil type as follows:

Table 5. Number of thickness observations in each prediction range for each oil type.

Thickness

Range

Alaska North Slope crude oil Forties Blend crude oil

Total Training Verification Total Training Verification

0-100 3 3 0 5 4 1

101-500 9 8 1 5 4 1501-1000 3 2 1 5 4 1

>1001 6 5 1 12 10 2

TOTAL 21 18 3 27 22 5

Thickness

Range

IFO-180 Heavy fuel oil Medium Fuel Oil/Gas Oil

Total Training Verification Total Training Verification

0-100 0 0 0 20 16 4

101-500 2 2 0 17 14 3

501-1000 0 0 0 6 5 1>1001 1 1 0 2 2 0

TOTAL 3 3 0 45 37 8

Tables 4 and 5 demonstrate that the primary data set consists of data observations in each of the

thickness ranges for all of the oil types specified (except for IFO-180 heavy fuel oil). There is a

fairly even spread of data observations for each thickness range, thereby reducing the chance of

biasing the training of the neural network to certain thickness ranges during the re-training.

It is important that the limits of the training set are defined clearly if the trained neural network is to beapplied to other data sets. The neural network can not reliably predict outside the limits of the

training data set. The primary training data set consists of the following upper and lower boundaries

for each of the parameters.

Table 6. Primary Training Set Range of Parameters.


22/41


AEA Technology 15

Parameter All Data

Max Min

Thickness 5079 4

Brightness of oil & sea 230 32

Brightness of sea 215 87

Brightness of oil 85 -126

Sea Temperature 18.15 14.8Air Temperature 18 14.12

Irradiance 940 -11.7

Emissivity 0.963 0.953

5.4 SECONDARY DATA SET

The primary data set consists of data from the 1994, 1995 and 1997 sea trials only. These trials

were all carried out during the summer months (July, August and September) when the ambient

temperature was fairly warm (14-18C). As a result, once trained with this data, the neural network

may only be able to predict oil thickness within these summer temperatures. Therefore the neuralnetwork was also trained with a secondary data set containing data collected at colder temperatures

(i.e. during the winter months). This secondary data set allows the robustness of the neural network

to be established.

The secondary data set consists of the primary data set (summer data) and data from the Sea

Empress incident (winter data). The Sea Empress incident occurred in February 1996 and

throughout the incident, thermal IR imagery and a range of meteorological parameters were

recorded. This information was compiled into a data set to train the neural network. However, it

must be noted that this data should not be considered an accurate training set because of the absence

of in-situ measurements of oil thickness and will only be used to determine the robustness of thetrained neural network.

The data required by the neural network trained in 1999 is:

Thermal IR imagery brightness values

Sea temperature

Air temperature

Irradiance

Emissivity

Oil thickness

The thermal IR imagery, sea and air temperature were recorded during the Sea Empress incident.

The irradiance data were obtained from Met Office for the dates required specifically for this

project. The emissivity of the oil spilt during the Sea Empress (Forties Blend crude oil) was obtained

for the primary data set so further emissivity analysis was not required. The oil thickness was not

measured at specific locations during the incident, however visual estimates were made during the

early stages of the incident so can be used as a guide to the range of oil thickness observed during

the incident.


23/41


AEA Technology 16

A data set was created for the neural network from thermal IR imagery taken from three dates in

February 1996 (22, 27 and 29). As no thickness measurements were made during the incident, the

data set was created through taking the brightness level in the thermal IR image from different areas

of the slick that are most likely to have a different oil thickness. The oil thickness at the specific

location where the brightness level was recorded was estimated into one of four categories as

follows:

1. 0-100 m = specific thickness applied to the data observation was 50 m




These estimates of thickness were based on the relative thickness indicated in the thermal IR imagery

and our knowledge of the behaviour of Forties Blend crude oil. The thickness estimates underwent a

series of checks through independent estimates of the thickness by three experienced staff. This

resulted in more than 60% of the data being classified into the same thickness category. However, it

is important to note that this method of estimating the thickness is by no means accurate and has

been used as a method of allowing the use of the Sea Empress data, because it is invaluable in testing

the neural network at this stage.

The final Sea Empress data set consisted of 84 data observations from the three dates in February.

The data included imagery from thick and thin oil layers, ambient temperatures ranging from 2 to 6C

and a range of irradiance levels. A total of 18 data observations were removed from the data set for

the testing phase of the project. These were representative of the range of the parameters

represented in the whole data set.

The following table details the thickness observations in the Sea Empress data set.

Table 7. Number of observations in each thickness category for the Sea Empress data set.

Thickness Range Forties Blend crude oil (Sea Empress)

Total Training Verification

0-100 7 5 2

101-500 19 15 4

501-1000 31 26 5

>1001 27 20 7

TOTAL 84 66 18

The Sea Empress data set was added to the primary data set to create the secondary data set. This

data set contains a total of 180 data observations of which 146 are for the training and 34 are for the

verification. The following table details the number of data observations for each thickness range for

the secondary data set.


24/41


AEA Technology 17

Table 8. Number of data observations in each thickness category for the secondary data set.

Thickness Range Secondary data set

Total Training Verification

0-100 35 28 7101-500 52 43 9

501-1000 45 37 8

>1001 48 38 10

TOTAL 180 146 34

The secondary data set, including observations of the Forties Blend crude oil spilt at the Sea

Empress incident, significantly extends the range of environmental parameters in comparison to the

primary data set. The following table shows the range of parameters the total of the secondary data

set covers (primary and Sea Empress). The range of parameters for the other oil types are as in the

primary data set detailed in Table 6.

Table 9. Secondary Data Set Range of Parameters for the Secondary Data.

Parameter All Data

Max Min

Thickness 5079 1

Brightness of oil & sea 255 0

Brightness of sea 230 32

Brightness of oil 215 -170

Sea Temperature 18 8Air Temperature 18 2

Irradiance 940 -11.7

Emissivity 0.963 0.953

6 Neural Network Re-Training

The primary and secondary training data sets were used in a Multi-Layer Perceptron (MLP) neural

network architecture, as used in the initial development of the neural network. Our previous work

(Wood et al., 1997) has suggested this is the best system for approaching these types of complex

relationships. This methodology is described in Section 3.1.

A problem anticipated at the start of the neural network training process was the danger of over-

fitting the network - a problem which frequently arises when the number of observations is small

compared to the number of adjustable parameters. The MLP has the property of being a universal


25/41


AEA Technology 18

function approximator, which means that given the right architecture (number of processing units) it

can learn any functional mapping. This is its strength and also its weakness. The ability to learn

mapping means that it can learn to model the noise in the data used to train it. Thus, training

examples can be reproduced well, but the model is influenced by the noise in the measurement data.

The model will not then produce sensible answers when tested on a new data set.

As detailed in Section 5.2, 7 input variables have been selected for the revised training of the

network. The network is forced to define the relationships between each of these variables and each

of the nodes in the inner layers (described in Section 3.1) and between the elements of the hidden

layer and the final output variable. The two sets of data available for training contain just 80 (primary

data set) and 146 (secondary data set) data observations. These small numbers of data were not

expected to be sufficient to avoid the problems of over-fitting.

As discussed previously (Wood et al., 1997) an initial solution is to train the network on only a

specified fraction of the available training data (the training data set) and to retain the remainder of

the training data set for testing the network during the training process (the training test data set).

The ability of the network to reproduce the training data set improves throughout the training

process. However, the parameters used to define the trained network are not those obtained when

the best reproduction of the training data set is achieved, instead they are those obtained when the

best performance against the training test data set is achieved.

Once the best configuration of the neural network was defined for the primary and secondary data

sets, it was then tested against the verification data set. The MLP network allows for two

approaches to be adopted during training i.e. regression and classification. Both of these have been

explored during this project.

6.1 REGRESSION APPROACH

The network was trained using 75% of the available training data from the Primary data set and, as a

separate process, using 75% of the available training data from the Secondary data set. Whilst the

data span the range from 4-5000 m, only 3 out of 146 data are in excess of 2500 m. The

network was, therefore, trained to estimate log (thickness) to prevent errors in the estimation of these

few large values biasing the error analysis.

To determine the optimum number of hidden neurons for the network, 15 training runs (trials) were

carried out with the network configured with 2-10 hidden neurons. The performance of the network

in reproducing the test data set was compared for each of the groups of 15 trials using a T-test. The

results indicated that the network did not perform significantly better with any one number of hiddenneurons - this is again an indication that the problem was over-fitted.

In general, the network was able to provide an acceptable reproduction of the measured thickness in

the training data set. The typical performance of the network is shown in Fig 4 for the network

configured with 7 hidden neurons and trained against the Primary data set. Whilst acceptable, this is

not as good as the performance reported during the initial training of the network (Wood et al.,

1997) which had an r2of 0.85.


26/41


AEA Technology 19

Figure 4. Scatter plot for predicted against actual thickness for the primary data set.

6.2 CLASSIFICATION APPROACH

Instead of treating the prediction of oil thickness as a regression problem, it is also possible to treat it

as a classification problem. Using this approach, the network does not attempt to predict the exact

thickness, but to assign the data into one of a series of bins representing thickness ranges. The

thickness ranges used in this study were selected to be consistent with those used in this previous

study (Wood et al, 1997). Alternative ranges, e.g. based on equal bandwidths, would have been

equally valid.

Training the network on both the Primary and Secondary data sets resulted in the same optimalarchitecture for both data sets. In both cases, 8 hidden neurons gave the optimal performance,

although the numerical values assigned to each of the weights in the network differed. The optimal

architecture is shown below:

Figure 5. Optimal architecture of neural network for both data sets as a classification problem.

Emissivity


0 - 100 m

Brightness of sea

101 - 500 m

Brightness of oil

501 - 1000 m


27/41


AEA Technology 20

Sea temperature

1001 - 6000 m

Air temperature

Irradiance

Hidden

Neurons

The bias weights are omitted for clarity on Figure 5.

The results from a typical training run of the network trained on the Primary data set are summarisedin thematrix below. The numerical values relate to the number of data points predicted to lie in each

combination of predicted and measured thickness categories. Diagonal elements indicate a correct

classification, whilst off-diagonal elements indicate an incorrect classification.

Table 10. Classification matrix using the Primary data set

Predicted

Measured 0-100 m 101-500 m 501-1000 m 1001-6000 m % Correct

0-100 m 17 6 0 0 74%

101-500 m 6 20 1 1 71%

501-1000 m 1 2 6 2 55%1001-6000 m 2 1 0 15 83%

73%

When trained with data from the Primary data set, the network is able to correctly classify over 70%

of the available training data.

The corresponding matrix for the network trained with the Secondary data set is summarised below.

Table 11. Classification matrix for the Secondary data set

PredictedMeasured 0-100 m 101-500 m 501-1000 m 1001-6000 m % Correct

0-100 m 17 7 1 3 61%

101-500 m 2 36 2 3 84%

501-1000 m 3 3 31 0 84%

1001-6000 m 1 1 4 32 84%

79%


28/41


AEA Technology 21

Using data from a wider range of environmental conditions, the network is able to correctly classify

almost 80% of the available training data.

6.3 SUMMARY

In summary, the neural network was trained with two data sets; the primary and secondary data sets.

The primary data set contained reasonably accurate in-situ thickness observations. The secondary

data set consisted of the primary data set and data from the Sea Empress incident which consisted of

estimates of the in-situ thickness by three members of the NETCEN team.

The optimum training approach was the classification method. The neural network was trained with

each of the data sets and the test errors of the training are as follows:

Correct Classification (%)

Primary Data Set 73

Secondary Data Set 79

This indicates the training using the secondary data set was able to produce a higher percentage of

correct classifications than for the primary data set.


29/41


AEA Technology 22

7 Neural Network Verification

This phase consists of testing the trained neural network with a selection of data that was not used to

re-train the network. This exercise can not be used as a true validation of the network since the

verification data set is a subset of the data being used to train the network. This verification processis used to establish whether the neural network is internally consistent and is able to predict oil

thickness representative of the data sets used to train the network.

A spreadsheet was developed to run the neural network model produced in the training phase. The

results of the verification are detailed as follows.

7.1 PRIMARY DATA SET

The following table details the results of the verification of the neural network trained as a

classification problem with the primary data set. The table includes the measured oil thickness values

and those predicted by the trained neural network (the correctly classified thickness values are

shaded in solid grey).

Table 12. Test data and thickness predictions for the primary data set.

Oil type Oil

Brightness

Sea

Temp

(C)

Air

Temp

(C)

Irradiance Measured

Thickness

(m)

Predicted

Thickness

(m)

Alaska North Slope 13 18.01 14.12 742 222 0-100

Alaska North Slope -72 18.05 14.58 184 525 1001-6000


Forties Blend 25 17.68 16.83 410 7 1001-6000

Forties Blend 37 18.10 17.02 375 310 101-500

Forties Blend 27 18.10 16.08 86 954 101-500

Forties Blend 10 14.99 16.93 270 1531 1001-6000

Forties Blend 19 14.80 16.88 20 3316 1001-6000

MFO/GO 15 14.99 17.02 5 7 0-100

MFO/GO 35 16.44 17.75 359 21 101-500MFO/GO 15 16.83 17.71 412 23 0-100

MFO/GO 8 15.29 16.68 120 29 0-100

MFO/GO 12 15.24 16.97 260 104 0-100

MFO/GO 30 16.73 17.71 451 204 101-500

MFO/GO 25 14.99 17.17 840 435 0-100

MFO/GO 10 15.04 17.17 930 537 101-500


30/41


AEA Technology 23

The results of the verification are summarised in the matrix below.

Table 13. Classification matrix for the verification of the network trained on the Primary data set

Predicted

Measured 0-100 m 101-500 m 501-1000 m 1001-6000 m % correct0-100 m 3 1 0 1 60%

101-500 m 3 2 0 0 40%

501-1000 m 0 2 0 1 0%

1001-6000 m 0 0 0 3 100%

50%

The network correctly classifies 50% of the verification data into the appropriate thickness bin.

However the verification data set contains a number of datum for which the measured thickness lies

on the borderline of two bins (e.g. 104 m has been classified as 0-100 m). These are indicated inTable 12 by shading in grey diagonal lines. If these values are also included, the network can be

regarded as correctly classifying 63% of the values into the correct thickness bin.

On examination of the correct and incorrect thickness predictions (Table 13), the neural network

does not appear to be predicting thickness consistently incorrectly for specific oil types, thickness

categories, thermal IR brightness or temperature. There is no systematic bias to the 37-50%

incorrect classification.

It was observed in the validation project (Davies et al, 1998) that once the data was outside the

training range of the neural network, the network could not predict oil thickness reliably. As an

additional verification, the neural network trained with the primary data set was used to predict theoil thickness using the data from the Sea Empress incident. The primary network is trained with data

collected during summer conditions and the Sea Empress data was collected during the winter.

Therefore the Sea Empress data set will not fall in the range of the primary training data set. The

result of this verification is that the neural network classifies the oil thickness for all the Sea Empress

data observations into the 1001-6000 m thickness range despite the estimated thickness ranging

from 50-1750 m. This demonstrates that the neural network cannot reliably predict oil thickness

outside the range of the training data set. The results of this verification are shown in more detail in

Appendix 1.

7.2 SECONDARY DATA SET

The following table details the results of the verification of the neural network trained as a

classification problem with the secondary data set. The correct thickness classifications are

indicated with a solid grey shaded area.


31/41


AEA Technology 24

Table 14. Test data and thickness predictions for the secondary data set.

Oil type Oil

Brightness

Sea

Temp

(C)

Air

Temp

(C)

Irradiance Measured

Thickness

(m)

Predicted

Thickness (m)

Alaska North Slope 13 18.01 14.12 742 222 0-100Alaska North Slope -72 18.05 14.58 184 525 501-1000


Forties Blend 25 17.68 16.83 410 7 101-500

Forties Blend 37 18.10 17.02 375 310 101-500

Forties Blend 27 18.10 16.08 86 954 1001-6000

Forties Blend 10 14.99 16.93 270 1531 501-1000

Forties Blend 19 14.80 16.88 20 3316 1001-6000

MFO/GO 15 14.99 17.02 5 7 0-100

MFO/GO 35 16.44 17.75 359 21 101-500

MFO/GO 15 16.83 17.71 412 23 101-500MFO/GO 8 15.29 16.68 120 29 0-100

MFO/GO 12 15.24 16.97 260 104 0-100

MFO/GO 30 16.73 17.71 451 204 0-100

MFO/GO 25 14.99 17.17 840 435 0-100

MFO/GO 10 15.04 17.17 930 537 101-500

Forties Blend 15 7.75 2.14 127 50 101-500

Forties Blend 24 7.95 7.00 251 50 101-500

Forties Blend 32 7.65 2.14 400 250 501-1000

Forties Blend 38 7.75 2.14 127 250 101-500

Forties Blend 39 7.80 6.69 75 250 101-500

Forties Blend 58 7.95 7.00 251 250 501-1000

Forties Blend 54 7.65 2.14 400 750 501-1000

Forties Blend 44 7.65 2.14 400 750 501-1000

Forties Blend 59 7.75 2.14 127 750 501-1000

Forties Blend 88 7.80 6.69 75 750 501-1000

Forties Blend 60 7.95 7.00 251 750 501-1000

Forties Blend 71 7.65 2.14 400 1750 1001-6000

Forties Blend 59 7.65 2.14 400 1750 1001-6000

Forties Blend 83 7.65 2.14 400 1750 1001-6000

Forties Blend -40 7.75 2.14 127 1750 1001-6000

Forties Blend 98 7.80 6.69 75 1750 501-1000

Forties Blend -62 7.95 7.00 251 1750 501-1000

Forties Blend -51 7.95 7.00 251 1750 1001-6000

The verification results are summarised in the following matrix.


32/41


AEA Technology 25

Table 15. Classification matrix for the network trained on the Secondary data set

Predicted

Measured 0-100 m 101-500 m 501-1000 m 1001-6000 m % correct

0-100 m 2 5 0 0 29%

101-500 m 4 3 2 0 33%

501-1000 m 0 1 6 1 75%1001-6000 m 0 0 3 7 70%

53%

Overall, the network is able to correctly classify 53% of the data into the appropriate thickness bin.

As with the primary data set, the verification data contains a number of datum for which the

measured thickness lies on the borderline of two bins (e.g. 104 m has been classified as 0-100m).

These are indicated in Table 14 by shading in grey diagonal lines. If these values are also included,

the network can be regarded as correctly classifying 59% of the values into the correct thickness bin.

In some situations, the network returns a high probability that the thickness lies in one particular bin,indicating a high confidence that the result lies in that bin. In other cases, a more even distribution of

probabilities may be returned, indicating a lower confidence in the prediction of the network. If the

analysis also takes account of the predictions where the probability that the thickness lying in the

correct bin is only slightly lower than the probability that the thickness lies in an alternative bin, then

the results improve and the network comes close to correctly classifying almost 70% of the data.

The overall percentage error in the verification results for the secondary data set is very similar to that

for the primary data set. In the secondary data set, the verification data classified incorrectly

accounted for 16 of the 34 data observations whereas in the primary data set 8 of the 16 data

observations were incorrectly classified. As discussed previously there is no apparent bias to theincorrect predictions, i.e. they are not oil type, thickness or specific variable dependent.

It is worthy of note that the majority of the data set incorrectly classified by the network trained into

the primary data set were also incorrectly classified by the network trained with secondary data set.

This implies that the network is consistently failing to learn the relationship between certain variables

as a result of the lack of training data. It is also possible that the input variables used in the training

do not adequately represent the relationship between the input variables and the thickness, therefore

it may be that another input variable is required to predict the thickness in some instances.

7.3 COMMITTEE OF NETWORKS


33/41


AEA Technology 26

To further address the problem of over-fitting the data, another approach was adopted in which a

number of networks were trained using the Secondary data set. To make a prediction, the outputs

from these trained networks were combined, summed and normalised. As each network has a

slightly different training set (because only 75% of the available training data is used for training, as

described in Section 6.1) each one will over-fit the data in a different way and hence when averaged

the effects of over-fitting should be averaged out. This approach (termedforming a committee of

networks) is a well-established technique and was also used in the earlier studies (Wood et al.,

1997).

The neural network was re-trained using the committee of networks approach with the secondary

data set. The resultant committee underwent testing with the verification data set and the results are

detailed in the Table 16.


34/41


AEA Technology 27

Table 16. Test data and thickness predictions for the secondary data set using a committee of

networks.

Oil type Oil

Brightness

Sea

Temp

(C)

Air

Temp

(C)

Irradiance Measured

Thickness

(m)

Predicted

Thickness (m)


Alaska North Slope -72 18.05 14.58 184 525 501-1000


Forties Blend 25 17.68 16.83 410 7 101-500

Forties Blend 37 18.10 17.02 375 310 101-500

Forties Blend 27 18.10 16.08 86 954 1001-6000

Forties Blend 10 14.99 16.93 270 1531 0-100

Forties Blend 19 14.80 16.88 20 3316 1001-6000

MFO/GO 15 14.99 17.02 5 7 0-100

MFO/GO 35 16.44 17.75 359 21 500-1000MFO/GO 15 16.83 17.71 412 23 0-100

MFO/GO 8 15.29 16.68 120 29 0-100

MFO/GO 12 15.24 16.97 260 104 0-100

MFO/GO 30 16.73 17.71 451 204 0-100

MFO/GO 25 14.99 17.17 840 435 101-500

MFO/GO 10 15.04 17.17 930 537 101-500

Forties Blend 15 7.75 2.14 127 50 0-100

Forties Blend 24 7.95 7.00 251 50 0-100

Forties Blend 32 7.65 2.14 400 250 101-500

Forties Blend 38 7.75 2.14 127 250 101-500

Forties Blend 39 7.80 6.69 75 250 101-500

Forties Blend 58 7.95 7.00 251 250 101-500

Forties Blend 54 7.65 2.14 400 750 501-1000

Forties Blend 44 7.65 2.14 400 750 501-1000

Forties Blend 59 7.75 2.14 127 750 501-1000

Forties Blend 88 7.80 6.69 75 750 501-1000

Forties Blend 60 7.95 7.00 251 750 501-1000

Forties Blend 71 7.65 2.14 400 1750 1001-6000

Forties Blend 59 7.65 2.14 400 1750 1001-6000

Forties Blend 83 7.65 2.14 400 1750 1001-6000

Forties Blend -40 7.75 2.14 127 1750 1001-6000

Forties Blend 98 7.80 6.69 75 1750 501-1000

Forties Blend -62 7.95 7.00 251 1750 1001-6000

Forties Blend -51 7.95 7.00 251 1750 1001-6000


35/41


AEA Technology 28

The data are summarised into the following matrix.

Table 17. Classification matrix for the committee of networks trained on the Secondary data set

Predicted

Measured 0-100 m 101-500 m 501-1000 m 1001-6000 m % correct

0-100 m 5 1 1 0 71%

101-500 m 2 7 0 0 78%

501-1000 m 0 1 6 1 75%

1001-6000 m 1 0 1 8 80%

76%

The network is able to correctly classify 76% of the data into the appropriate thickness bin.

As previously, the verification data contains a number of datum for which the measured thickness lieson the borderline of two bins (e.g. 104 m has been classified as 0-100 m). These are indicated in

Table 16 by shading in grey diagonal lines. If these values are also included, the network can be

regarded as correctly classifying 82% of the values into the correct thickness bin.

The committee of networks approach should decrease the noise in the network training and thereby

decrease the error in the predictions. On comparison of the results of the verification using the single

classification training (Tables 14 and 15) and using the committee of networks training (Tables 16

and 17), the percentage error in the predictions has decreased as expected. There was a significant

improvement in the number of correct predictions for the lower thickness categories (0-100 m and

101-500 m) and a slight improvement in the high thickness category (1001-6000 m). This is atotal of 26 correct predictions of the 34 data observations in the verification data set.

The incorrect predictions are the same data observations incorrectly predicted in the verification of

the single classification training. This supports the theory that the network is consistently failing to

learn the relationship between certain variables and the oil thickness.

7.4 SUMMARY

The verification tests of three neural networks trained resulted in the following verification results:


36/41


AEA Technology 29

Table 18. Summary of the verification tests for the neural networks

Network Training

Type

Data Set % Correct % Correct within

10% error

Single Classification Primary 50 63

Single Classification Secondary 53 59Committee of

networks classification

Secondary 76 82

The best verification results were obtained from the committee of networks classification with the

secondary data set which was shown to be capable of correctly classifying the thickness of the

verification data in over 76% of cases. A 10% error is included in the % correct to allow for

occasions where the actual thickness is close to the boundary of the thickness categories, for

example an actual thickness of 104m is classified as 0-100m. This raises the percentage correctly

predicted for all the training types, the highest % correct remains the committee of networks at 82%.

It was observed with each of the trained neural networks that the same verification data observations

were being incorrectly classified for the majority of cases. Some of the data observations had similar

input variables, so the network classified them into the same thickness category despite sometimes

having actual thickness values from different thickness categories.

This implies that the network is consistently failing to learn the relationship between certain input

variables and the thickness. This could be due to the size of the training data set, if more data was

available for training the network would have a better chance of fully learning the relationship

between the input variables and the thickness. It is also possible that the input variables used in the

training do not adequately represent the relationship between the input variables and the thickness,

therefore another input variable may be required to predict the thickness in some cases.

8 Conclusions

The re-calibration suggests that the re-trained neural network is internally consistent in predicting oil

thickness based on the input data used. This analysis includes data sets consisting of a wider range

of environmental variables (sea and air temperature) and thermal IR imagery recorded in both winter

and summer that has not been previously included in the training.

The errors identified in the predictions by the network show no clear pattern with oil type, brightness

or thickness categories. This suggests the network does not have a systematic bias. The 1999 re-

trained network demonstrates the ability to account for solar heating which was not the case for the

1997 trained network used for the validation project (Davies et al., 1998). This is evident through

correct predictions from data taken throughout the day (i.e. at different irradiance levels and ambient

temperatures) and from different oil thickness levels.


37/41


AEA Technology 30

However it is important to emphasise that there is no guarantee that the data sets that have been used

to retrain the network will cover the complete range of the parameters which will be encountered in

the event of a spill, such as the meteorological conditions and oil type. In addition, some of the data

used are estimated rather than measured. It is still not clear whether the size of the training data set

required to produce a robust system makes this neural network technique impracticable in the long

term.

Using the committee of networks approach it was possible to obtain the correct classification in 76%

of cases. For the majority of the remaining cases, the misclassification was only by one thickness

category. However, the verification studies carried out in this project indicate that there is still a

significant error in the thickness predictions. This may be improved with:

A larger training data set

Or/and

More input variables

This additional data could be obtained during actual incidents and experimental field trials in the

future. In this way, the neural network can be continually re-trained and updated to take account of

additional validated data taken in different environmental conditions. This would maximise the

chances of the operational data falling in the range of environmental parameters used in the training

process.

9 Recommendations

The ultimate aim of this research is to develop a technique for MCA to measure oil slick thickness.

Having knowledge of the thickness of the slick allows calculation of the volume of oil on the sea

surface in an oil spill incident. This enhances the operational response to a spill by allowing

assessments to be made on the efficiency of a response technique and ensures that response

equipment is used to its maximum effectiveness.

The re-training of the neural network has shown that this technique is partially successful in measuring

oil thickness. However, the networks derived during this project can not be considered as beingsufficiently robust to be reliable during operational use at present. In order to continue the

development of this technique and to provide some benefit of the research to MCA at this stage, we

recommend a pilot system is developed which can be deployed as an experimental system at the

next spill or experimental field trial.

On deployment, the pilot system can provide MCA with an additional tool for the operational

response. The practicalities of the system and its usefulness to the MCA can be evaluated. This

includes the data analysis and collection facilities and procedures. The accuracy of the neural


38/41


AEA Technology 31

network can be studied further and re-training undertaken as and when additional data sets are

collected to continue development of this technique.

To develop the current experimental system into a pilot operational system would require the

following relatively minor developments:

Collaboration with Air Atlantique to standardise on image analysis software (for example, using

ER Mapper) and to devise procedures for downloading the thermal IR imagery to a suitable

format for analysis.

Determine suitable techniques for collecting all the variables required by the neural network, i.e.

air and sea temperature, irradiance.

Develop a sorbent pad sampling technique that includes real-time analysis for calibration of the

neural network.

If the pilot system is developed and deployed (potentially using the FLIR system on the SAR

helicopters as well as the MCAs remote sensing planes) at future incidents then the system can be

used to give estimates of volumes, alongside the normal visual estimates, for operational use.

As an illustration of how the volume could be calculated from the oil thickness predictions from the

neural network, a thermal IR image was taken from the Forties Blend experiment in the 1997

experimental sea trials. This image (shown in Appendix 1) was divided into four brightness bands

and the surface area of the oil slick in each brightness band determined. The thickness relating to a

single brightness value in each brightness band was determined using the trained committee of

networks. For each of the brightness bands, the neural network predicted the thickness in the range

of 1001-6000 m. This is not unreasonable because the release of the Forties Blend was completed

only 10 minutes before the thermal IR was recorded, therefore the oil is likely to be at a fairly

uniform thickness. To calculate the volume of oil, a single thickness value was assigned to the

thickness range. The value for the 1001-6000 m range was taken to be 1750 m based on thespread of the training data in the thickness range. Table 18 details the surface area and predicted

volume using the neural network.

Table 19. Volume prediction using the committee of networks.

Neural Network

prediction

IR

Brightness

Range

Area

(m2)

Thickness

(m)

Volume

(m3)

0-126 155 1750 0.3

126-183 6035 1750 10.6

183-208 10324 1750 18.0

208-255 11116 1750 19.5

TOTAL 27630 TOTAL 48.4


39/41


AEA Technology 32

The actual volume of the oil slick when the image was taken is estimated at 44-51 m3. The neural

network volume estimate is 48 m3, which is within this estimated range.


40/41


AEA Technology 33

9 References

Davies L., Dooley K., Lunel T and Lewis A., Field validation of neural network analysis to measure

oil slick thickness, AEAT-3474, AEA Technology, Culham, Oxfordshire, UK, 1998.

Goodman R., Overview and future trends in oil spill remote sensing, Spill Science and Technology

Bulletin, Vol 1, No 1, pp11-21, 1994.

Wood P, Strachan I, Davies L and Lunel T., Determination of oil thickness by neural network

analysis, AEAT-1151, AEA Technology, Culham, Oxfordshire, UK, 1997.


41/41

Appendix 1

Data Sets

CONTENTS

Table 1 Primary Data Set

Table 2 Secondary Data Set

Table 3 Verification results using Sea Empress data on the neural network

trained with the primary data set

Figure 1 Thermal IR image of Forties Blend experiment from 1997 seatrial

Estimation of Oil Thickness

Documents

Transcript of Estimation of Oil Thickness