1

Doctoral Defense

 Application of Agent-Based Approaches to Enhance Container

Terminal Operations

byOmor Sharif

Director: Dr. Nathan Huynh In Partial Fulfillment of the Requirements for the Degree of

Doctor of Philosophy in Civil EngineeringCollege of Engineering and Computing

Summer, 2013

2

A Very Brief Review of Container Terminal

• An interface between ocean and land transport• Containers are loaded/unloaded to/from a ship• Temporary storage of export and import

containers• Operation involves a large number of decisions

Land Side

Water Side

Container Yard

3

Growth in Container Traffic, Major Challenges

• Limited capacity issues, Congestion• Environmental concerns, emission from

congestion• Increased complexity in planning• Lack of optimized decision making tools• Competition among terminals has considerably

increased• Provide satisfactory customer service, reliability• Tools that reduce cost, increase productivity of

operationsOptimization | Automated Equipment | Information Technology

World container volumes have increased from 113 million TEU to more than 572 million TEU in the period from 1993 to 2011. An annual average growth rate of nearly 10% per year.

1993

1995

1997

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

0100200300400500600700

Total Container Traffic (Million TEU) By Year

Flow of Containers in a Container Terminal

CUSTOMER/DEPOTS

TERMINALGATE

STORAGEAREA

SHORE/BERTH

VESSEL/ SHIP

XTs XTs AGVs /ITs/SCs QCsYCs

ARRIVAL AND STORAGE RETRIEVAL AND LOADINGRETRIEVAL AND PICKUP UNLOADING AND STORAGE

Quay Cranes

Internal Trucks

Yard Cranes

ExternalTrucks

ExternalTrucks

Customers

Storage

Area

Berth

Vessel

Intern

al Tru

ck

5

Decision Problems in Container Terminal

× Berth Allocati

on

× Quay Crane Scheduling

Transport

Containers

Storage Assignment

Yard Crane Scheduling

Gate Operations

Three Research TopicsResearch Focus Areas

Queuing of Trucks at

GatesYard Crane Operations

Container Storage Problem

6Reduce truck processing time | Reduce congestion | Reduce emission Optimize available resources | Improve efficiency of related processes

7

Journal Article I 

 Application of El Farol Model for Managing Marine Terminal Gate

Congestion 

B

byOmor Sharif, Nathan Huynh and Jose Vidal

Research in Transportation Economics, 32 (1), 81-89 (2011)

Excessive Waiting of Trucks at Terminal Gates

• Documentation processing, security checks at gate• At peak hours demand greatly exceeds supply at

gate• Long waiting time results in high truck cost• Source of significant emission, unhappy truckers

Figure: Port Elizabeth, New Jersey - Peak hour congestion at gate

How to Reduce Queuing Time? Experimental Solutions to Reduce Fluctuating

Arrivals

Appointment System with Time

WindowsLive View of GateUsing Webcams

9

10

Agent-based Model for Dispatch Decisions Send

Truck Now???

4 miles

11

El Farol Bar Problem (Arthur, 1994)• A bar in Santa Fe, New Mexico• Every Thursday night, everyone wants to go to the

El Farol• But it's no fun to go there if, say, more than 60 go

to the bar, they'll all have a worse time than if they stayed at home.

• It is necessary for everyone to decide at the same time whether they will go to the bar or not

• If agents use different strategies to decide, the overall attendance fluctuates at 60, which is the Nash Equilibrium

Fig: A simulated 100-week record of attendance at El Farol

Principal Parameters of Model

12

N ≡ Set of Depots (n ∈ N), T ≡ Set of Trucks (t ∈ T)

Tolerance, L ≡ Maximum time the truck willing to waitExpected wait, E (W) ≡ The time truck thinks it will waitMake a Decision, SEND? (n, t) ≡ YES if E (W) ≤ L

NO otherwise

Wait at gate, W(t) = Waiting in Queue + Waiting in ServiceDiscretization interval, I ≡ How often to make a decision?Historyx = Waiting time at terminal gate for last m intervals

Estimate E(W) using ‘predictors’• Predictors are simple rules or

strategies/hypotheses• Agents use predictive strategies to estimate E(W)• Sample predictors-

A random number Positive trend of last 10 values A periodic function Moving average of past 5 values

13

14

Solution Steps

Create: A large number of ‘predictors’ to estimate E(W)Choose: Agent chooses k predictors from predictor space, S Estimate: Estimate E(W) using each predictor

Learn: Agents learns about how predictor has performed in past Rank: Agents assign scores and rank the predictors

Use Best: Use ‘best performing predictor’ from predictors set

Model Implementation in Netlogo

• Simulation model, coded in Netlogo• A modeling environment for agent-based systems• Many useful primitives that expedite the

implementation• Extensive documentation and tutorials, model

libraries• Updated versions are continuously released 15

Experiment Parameters (Process 1440 Trucks)

Parameter Value UnitNumber of Depots 10 NosDispatch rate (θ) 12 trucks/

depot/hrMean transaction time (μ)

3, 4, 5, 6, 7 and 8

minutes

Tolerance (L) 15, 20, 25 and 30

minutes

Total predictors 200 NosPredictors per depot (k)

12 Nos / depot

Update interval (I) 5,10 and 15 minutesMaximum memory (m)

20 intervals

Predictor scoring policy

Original precision

n/a

Alpha 0.5 n/a

16

Results (Impact of Tolerance, Interval on Wait)

Fig: Impact of tolerance on mean wait time of trucks

Fig: Impact of tolerance on max waiting time. 17

Increases fast at low I

Mean Wait Time Fluctuates Around ToleranceFig: Mean wait time of trucks (I =15 minutes, L = 15 minutes)

Fig: Mean wait time of trucks (I =10 minutes, L = 10 minutes)

18

Significantly Better Than ‘Do Nothing’

• 43% and 63% lower mean wait time for I = 5 and 10 mins

• 22% and 40% lower maximum wait time for I = 5 and 10 mins

19

Key Findings and Contributions• Provides steady truck arrival, less queuing time at gate

• Adopt higher ‘I’ for distributing peak hour demand

• Good amount of emission (PM2.5, NOx, GHG) reduction

• First study on gate congestion using real-time information

• Examines how dispatchers can benefit from real-time information.

• Offers many experiments for choosing suitable parameters

20

21

Journal Article II

Inter-Block Yard Crane Scheduling at a Marine Container Terminal

byOmor Sharif, Nathan Huynh, Mashrur Chowdhury, Jose

VidalTransportation Science and Technology, 1 (2), 109-130

(2012)

22

Efficiency of Container Yard Operations is Crucial

• Container yard is the interface of land and waterside operations

• Containers are temporarily stored in Container Yard • A container yard is made up of several blocks of

containers• Workload varies (1) among blocks (2) over operational

hours

Land/ Gate Side

Water/ Vessel Side

23

Yard Cranes Influence Truck and Ship Waiting Time

• Yard Cranes are deployed for container handling• Rubber Tired Gantry Cranes are most popular Yard

Crane• Cranes support the ship loading and unloading

operations• Cranes support external truck operations

24

Objective, Assumptions, Constraints

• Total operational hours is divided in several planning periods

• A workload forecast for blocks is known at start (time-units)

• At most two cranes work at a block at the same time• At most one transfer per planning period• 10 – 15 minutes transfer time for each block traveled

Given: A limited number of cranes and workload at blocksGOAL: Assign and relocate cranes among blocks to minimize the workload that remains unfinished at the end of a period

25

Rules for Initial Assignment of Cranes to Blocks High to low work volume• Sort blocks in decreasing order of initial work

volume (IWb)• Assign NCb

max cranes to each block starting from topmost item

Crane at each block• If nc = nb, assign one crane per block• If nc < nb, sort blocks in decreasing IWb , assign

cranes• If nc > nb, first assign one crane per block, then sort

in decreasing IWb

Reduce transfers • Assign cranes to reduce number of future transfers• A four step approach• Based on pass/fail of a series of inequalites

26

Three Principal Parameters of Model Extra capacity of a crane• E (c) = Tc × NCb

initial − IWb

• Time Period X Initial Number of Cranes – Workload of block

Amount of help needed by a block• H (b) = IWb − Tc × NCb

current

• Workload of block - Time Period X Current Number of Cranes

Transfer time matrix• TTc

od

• Different transfer time for longitudinal and lateral movement

27

Matching using Deferred Acceptance Algorithm• Assigning cranes are viewed as matching crane and

blocks• DAA is a market matching model by Gale and Shapley• Numerous follow up studies in Economics, Computer

science• Find a match between two set of agents• Each agent has a preference order over other set of

agents • ‘One to one’ matching – e.g. Marriage Problem• ‘Many to one’ matching – e.g. College Admission

28

Preference Functions For Crane And Block AgentsMinimum transfer time • Transfer time of a crane from origin to destination

blockPositive difference • Extra Capacity of a crane – Help needed by a

block – Transfer timeAbsolute difference • abs (Extra Capacity of a crane – Help needed by a

block – Transfer time)

Absolute difference squared distance• abs (Extra Capacity of a crane – Help needed by a

block – Transfer time2)

29

Algorithm to assign cranes to blocks1. Each crane j send match request to first block i from its preference list

2. Each block i receiving more than qi requests, ‘holds’ the most preferred qi cranes and rejects all others.

n. Each crane j rejected at step n − 1 removes the block i rejecting the crane from its preference list.

Then the rejected crane j makes a new match request to its next most preferred block i who hasn’t yet rejected it.

Go to step n − 1. (n = 3, 4, 5, …)

30

Implementation Snapshot from Netlogo

• Multi-agent simulation GUI

Setup block layout

Create workload

Create cranes

Initial assignment

Inter-block transfer matrix

Pre-analysis steps

Find Matching

Reassign/update results

31

Test Parameters Based on Real-World ProblemsParameter Value UnitNumber of blocks 10, 20, 30 NosNumber of cranes 1 or 1.5 × Number of blocks NosLength of planning period

240 Minutes

Work volume (1) Moderate (2) Heavy (3) Above capacity

Minutes 

Initial assignment (1) Crane at each block (2) High to low work volume (3) Reduce transfers

-

Preference strategy (1) Minimum transfer time (2) Absolute difference (3) Positive difference (4) Absolute inverse squared distance

-

Algorithm Version (1) Blocks proposing (2) Cranes proposing

-(1)Moderate Workload: 60% of total crane capacity ± 40% of average

workload(2)Heavy Workload: 90% of total crane capacity ± 20% of average

workload(3)Above Capacity Workload: 110% of total crane capacity ± 40% of

average workload

32

Model Effectively Reduces Unfinished WorkloadPercentage incomplete work volume: (Incomplete/Total*100%)Case I - Average number of cranes per block = 1.0

Number of Blocks     10

      20

      30

 

Number of Cranes     10

      20

      30

Work Volume M H  AC M H   AC M H   AC

Minimum Transfer Time 0 0.13 11.34

0 0.10 11.14

0 0.12 11.28

Absolute difference 0 0.16 12.16

0 0.13 12.09

0 0.11 12.16

Positive difference 0 0 13.55

0 0.05 13.08

0 0.04 12.86

Absolute difference squared distance

0 0.13 11.80

0 0.08 11.69

0 0.10 11.81

Mathematical Program 0 0 10.57

0 0 10.57

0 0 10.60

33

Majority of Cases are Optimal/Near-optimalPercentage incomplete work volume: Case II - Average number of cranes per block = 1.5Number of Blocks   10     20     30  

Number of Cranes   15     30     45

Work Volume M H AC M H AC M H AC

Minimum Transfer Time 0 0.02 9.79 0 0.20 10.00

0 0.48 10.04

Absolute difference 0 0.07 10.19

0 0.18 10.45

0 0.43 10.48

Positive difference 0 0.14 10.40

0 0.53 10.73

0 1.00 10.71

Absolute difference squared distance

0 0.01 10.06

0 0 10.31

0 0.01 10.38

Mathematical Program 0 0 9.69 0 0 9.77 0 0 9.79

34

Key Findings and Contributions• In ‘medium’ condition all work can be finished • In ‘heavy’ condition the percentage incomplete is 1% or less• In ‘above capacity’ condition the percentage remaining is

within 3% of the lower bound.• ‘Reduce Transfers’ initial assignment works best.• An agent-based approach for solving inter-block scheduling

problem• Provide several preference functions that are effective in

minimizing• Scalable and time efficient, a problem with 30 blocks can be

solved in 3 seconds• Several minutes to solve using Mixed Integer Programs• Provides various strategies for initial assignment of cranes

to blocks

35

Journal Article III 

 Storage Space Allocation at Marine Container Terminals Using Ant-based

Control

byOmor Sharif and Nathan Huynh

Expert Systems with Applications, 40 (6), 2323-2330 (2013)

36

Assign Newly Arriving Containers to Yard Blocks• Container yard is divided up into yard

blocks• Each yard block has a fixed total capacity• Store export containers brought in by

trucks• Store import containers unloaded from

ships• Allocate yard blocks to newly arriving

containersGates BerthsYard Blocks

Export containers

Import containers

Export containers

Import containers

Land Side Container Yard Water Side

37

Two Competing Objectives of SSAPBalance the workload of blocks– Proper

distribution of arriving containers amongst blocks

Minimize container transportation cost – Minimize the distance traveled by ITs between berth and yard

38

• Reduces relocation of yard cranes• Reduces waiting time of vessels• Reduces average container handling time• Reduces congestion on road network• Improves efficiency of related resources (quay

cranes, yard cranes, storage space, ITs) – Zhang et al., 2003

Workload Balancing is Critical Because…

Workload Imbalance is the variability of number of containers among blocks at some time (we used standard deviation for this measure).

39

We Utilize an Ant-Based Control• ABC Falls under ‘Ant colony optimizations &

algorithms’ • Inspired by collective intelligence of an ant

colony• Can solve complex routing problems in network • For example, used for shortest path, load

balancing• We utilize pheromone laying behavior of ants

Ants have a decision to make

Situation several months later

40

Highlights of Ant Based Control for SSAP• A container terminal is a network of gates, blocks,

berths• Links are bi-directional container transportation

routes• Network also support a traffic of artificial ant

agents• Ants move just like inbound & outbound

containers• Ants lay pheromone as a function of distance &

congestion• Probabilities in pheromone table guide ants &

containers

Fig: Ant choses path according to probability

41

Shortest Path

Workload Balancing

• Ant age Increases with traveled distance• Probability of selection decrease with ant age

• Ants are delayed at congested blocks

• Delay is proportional to degree of congestion

42

Dynamic Interaction between Ants & Containers

43

Simulation

• The snapshot of GUI of our simulation

44

Parameter and Values Used in Model

A demand of 2500 Containers

45

Parameter and Values Used in Model

46

Performance Metric I - Transport Distance

• Converge to shortest path in 500 time-units• Under no congestion converge to lower bound• When congestion occurs distance converge to 2% above

lower bound

47

Performance Metric II - Workload Imbalance

• Standard deviation gradually rises when no congestion• When congestion occurs standard deviation falls• Maintains a distribution to accommodate both objectives

48

Key Findings and Contributions• The approach is effective in balancing workload

in blocks• Reduces transport distances during ship

loading/unloading • Distributed allocation to individual containers• Real-time solution requiring no advanced

container arrival information, more suitable for real-world truck arrival

• Adaptive, scalable.• Synchronous instead of traditional hierarchical

approach

49

Thank You

50

• A class of computational modeling • For simulating the actions and interactions of

autonomous, intelligent agents • To assess the agents’ effects on the system as a

whole.• ABMs are individual-based models• Most agent-based models are often composed of: (1)agents with simple or complex goals(2)distributed decision-making heuristics; (3)learning rules or adaptive processes; (4)an interaction system; and (5)a non-agent environment

What is Multi-agent Systems and Agent-Based Modeling?

51

• ABM is relatively new research field within artificial intelligence.

• ABM is relatively unexplored in the area of terminal operations.

• Suitable for dynamic distributed resource allocation• Traditional approaches use centralized mathematical

programs for optimization• No Polynomial time solution for majority of problems• ABM can outperform in certain domain property• Scalability: Do not require exhaustive search in solution

space, model integration potential• Computational Efficiency: Real time solution, no

advanced computation time is required• Adaptability: No forecast required, no revision for

forecast errors• Fault tolerance: Failure will impact locally, thus more

robust

Why use agent-based modeling in a terminal?

52

• In Recent years integrative modeling for container terminals are being emphasized

• Various processes in a terminal are interconnected

• Improved terminal performance cannot necessarily be achieved by treating the processes separately

• A few studies have attempted such an integrated approach, and they are on the quay side

• More decision tools need to be developed and integrated

• Multiagent systems approach has been proposed as a tool in integrated decisions frameworks in works by Henesey (2006); Franz et al. (2007), Stahlbock and VoB (2008)

• Integrated model using traditional optimization techniques has limited viability due to computational complexity.

Integrative Modeling in CT Research

53

• Drayage is relatively short distance of supply chain

• However, drayage accounts for 25% to 40% of Origin-Destination Expenses

• High cost affect the profitability of an intermodal service

• Reduce the emissions impact on the surrounding communities

• Reducing the idling time of drayage trucks is equivalent to reducing local and regional particulate matter (PM 2.5), nitrogen oxides, and greenhouse gas emissions

• Drayage trucks operate primarily in urban environments, a reduction of these harmful pollutants has a proportionally greater benefit.

Why study drayage?

54

• Terminal operator designates available time windows, truckers choose one of the available time windows

• Experimented in terminals e.g. Los Angeles, Long Beach, Oakland, Port of Vancouver etc. Under consideration at others.

• In theory, the appointment system should reduce truck processing time

• In practice, little time savings for truckers was reported due to lack of structures in appointment system

• Appointment systems are in an early stage of development, with no uniformity between terminals or ports and many implementation issues to be resolved

Appointment System is at early stage of development

55

• The problem addressed in this study is more complex than the original El Farol Bar

• First, the total demand (number of people intending to go to the bar) does not vary with time, truck demands vary throughout the day.

• Second, the bar attendance event occurs at one specific time. Truck dispatching a continuous process that occurs throughout the operational hours of the terminal.

• Third, depots are not homogenous, they are located at different distances; depots closer to the terminal are in a better position to take advantage of the provided real-time information, E(W) will be closer to the actual wait time.

• Fourth, travel time and service time are not applicable to bar problem. Given service time is stochastic, the strategy of estimating truck wait time is more difficult.

Additional difficulty of Decision making by a truck dispatcher VS Individuals in the bar

56

• For each block the initial work volume at the beginning of a planning period is the work volume forecast for that period plus incomplete work volume from the previous period.

• Work volume = Average timeunits required per move X Number of container moves

How to Determine Workload in a Yard Block

57

Proposed Agent-based Model (Contd.)

Pseudo Code of the Program – Part of the Main Loop

58

A Sample Example of Interblock YC Schedule

Initial Assignment Obtained via ‘Reduce Transfer’

(block 0, crane 11); (block 3, crane 19); (block 2, crane 18); (block 7, crane 16); (block 5, crane 14)

Mathematical program gives the same assignments

59

A sample example (Contd.)

60

• Some of the preferences in this example are symmetric

• The crane’s first choice is a block whose first choice is the crane (e.g. Block 0 and Crane 11; Block 7 and Crane 16; Crane 14 and Block 5).

• Notice that Block 3 and Block 0 both wants Crane 11; Block 0 gets it because it is the first item of Crane 11’s list; Block 3 ends up getting 4th choice)

• We can pick different preference strategies; for example, the cranes may use ‘minimum transfer time’, whereas the blocks may use ‘absolute difference’.

Additional Notes on the Sample Example

61

Overview of Storage Space Allocation Problem (SSAP)

62

Block level decision-• Finds the block to store a container• Objectives are ‘workload balancing’ & ‘distance

minimizing’

Stack level decision-• Finds an exact stack in a block• Objective is to ‘minimize future reshuffling’

Two Levels of SSAP

63