125-26.02.2019 CBS-weg 11, Heerlen, The Netherlands

Development of statistics on fishing fleets

ESSnet Big Data – Tracking Ships (2018-2020)

Face to face meeting

Michał Bis

Statistics Poland

2

Agenda

• Fishing fleets (Polish use-cases) – objectives

• Assumptions

• Approach

• Technology

• Conclusions

stat.gov.pl

3

Fishing fleet - objectives

Main objectives:

• Development of a functional prototype including procedures (software, tools), to promote and support processes of collecting, processing and analysing of big data based on AIS for statistics production;

• Experimental statistics in the domain of maritime economy;

• The output of the work available to the ESS and other NSIs.

Specific objectives (Polish use-case):

• Development and implementation of a process of traffic and activity related to the maritime fishing fleet based on AIS data.

stat.gov.pl

4

Fishing fleet

stat.gov.pl

Fishing fleet

AIS data

reference frame of

fishing fleets

fishing areas

fishing ports

Why did we choose this proces?

What existing problems could be solved by using AIS?

What kind of information we need for this proces?

5

Assumptions - AIS data

stat.gov.pl

Fishing fleet

AIS data

According to EU regulations, the system for fisheries controls requires all fishing vessels in Europe above 15 meters to use the AIS transmitters.

https://ec.europa.eu/fisheries/cfp/control/technologies_en

In practice, most fishing vessels (even below 15 meters) use the AIS system for safety reasons.

We plan to investigate whole Polish fishing fleet including deep-sea trawlers, fishing cutters, fishing boats.

6

Assumptions - reference frame of fishing fleets

stat.gov.pl

Fishing fleet

The register of fishing vessels in Poland is maintained by the Minister of Maritime Economy and Inland Navigation.

The register is kept in electronic form on the basis of a regulation of the European Commission.

Each EU country is obliged to maintain (i.e. collect, update and share) on the national fishing fleet register.

Data from the EU fishing fleet register are used to apply the rules of the Common Fisheries Policy.

According to the Commission Implementing Regulation (EU) 2017/218, technical data on individual fishing fleets are published on the website:http://ec.europa.eu/fisheries/fleet/index.cfm?lg=en

Note: Currently the base is unavailable due to modernization works. However, the data can be downloaded in csv files, it is as of June 2018.

reference frame of

fishing fleets

7

Assumptions – reference frame of fishing fleets

stat.gov.pl

Fishing fleet

Based on the register, we are able to prepare a precise database of the fishing fleet. It will help us to generate the current reference frame of fishing fleets(in purpose for filtering AIS data).

What can we find in the register?The list of technical attributes, unfortunately mmsi number is not obligatory.

Selected example attributes:Symbol Description

country_code Country

cfr Code fleet register

licence_ind Fishing license (Y = yes, N = no)

vessel_name Vessel name

port_code Port code

ircs_ind Call sign (Y = yes, N = no)

ircs Call sign

loa Total length

power_main Main engine power

entry_service_year Enter service year

reference frame of

fishing fleets

8

Assumptions - reference frame of fishing fleets

stat.gov.pl

Fishing fleet

Vessel type Number of vessels

Deep-sea trawlers 3

Fishing cutters 124

Fishing boats 707

Total 834

Polish fishing fleet - as of December 31, 2017

reference frame of

fishing fleets

9

Assumptions - fishing ports

stat.gov.pl

Fishing fleet

fishing ports

There are 63 fishing ports in Poland

10

Assumptions - fishing ports

stat.gov.pl

Fishing fleet

fishing ports

Voivodeship Name of the port Fishing boats Fishing cutters

Pomorskie Chałupy x

Pomorskie Chłapowo x

Pomorskie Dębki x

Pomorskie Gdańsk x

Pomorskie Gdańsk Górki Wschodnie x x

Pomorskie Gdańsk Górki Zachodnie x

Pomorskie Hel x x

Pomorskie Jantar x

Pomorskie Jastarnia x x

Pomorskie Jelitkowo x

Pomorskie Kąty Rybackie I x

Pomorskie Kąty Rybackie II x

Pomorskie Krynica Morska I x

Pomorskie Krynica Morska II x

Pomorskie Kuźnica x

Pomorskie Łeba x x

Pomorskie Mechelinki x

Pomorskie Mikoszewo x

Pomorskie Obłuże x

Pomorskie Oksywie x

fishing ports

11

Assumptions - fishing ports

stat.gov.pl

Fishing fleet

fishing ports

Voivodeship Name of the port Fishing boats Fishing cutters

Pomorskie Orłowo x

Pomorskie Piaski I x

Pomorskie Piaski II x

Pomorskie Puck x

Pomorskie Rewa x

Pomorskie Rowy x

Pomorskie Sopot x

Pomorskie Stegna x

Pomorskie Swarzewo x

Pomorskie Sztutowo x

Pomorskie Świbno x

Pomorskie Ustka x x

Pomorskie Władysławowo x x

Warmińsko-mazurskie Frombork x

Warmińsko-mazurskie Kamienica Elbląska x

Warmińsko-mazurskie Nowa Pasłęka x

Warmińsko-mazurskie Suchacz x

Warmińsko-mazurskie Tolkmicko x

Zachodniopomorskie Chłopy x

Zachodniopomorskie Darłowo x x

Zachodniopomorskie Dąbki x

fishing ports

12

Assumptions - fishing ports

stat.gov.pl

Fishing fleet

fishing ports

Voivodeship Name of the port Fishing boats Fishing cutters

Zachodniopomorskie Dziwnów x x

Zachodniopomorskie Dźwirzyno x

Zachodniopomorskie Jarosławiec x

Zachodniopomorskie Kamień Pomorski x

Zachodniopomorskie Kołobrzeg x x

Zachodniopomorskie Lubin x

Zachodniopomorskie Międzywodzie x

Zachodniopomorskie Międzyzdroje x

Zachodniopomorskie Mrzeżyno x

Zachodniopomorskie Niechorze x

Zachodniopomorskie Nowe Warpno x

Zachodniopomorskie Rewal x

Zachodniopomorskie Stepnica x

Zachodniopomorskie Szczecin-Dąbie x

Zachodniopomorskie Szczecin-Stołczyn x

Zachodniopomorskie Świnoujście x x

Zachodniopomorskie Świnoujście-Karsibór x

Zachodniopomorskie Świnoujście-Przytór x

Zachodniopomorskie Trzebież x

Zachodniopomorskie Unieście x

Zachodniopomorskie Ustronie Morskie x

Zachodniopomorskie Wolin x

fishing ports

13

Assumptions – fishing areas

stat.gov.pl

source: http://www.fao.org/fishery/area/Area27/en#FAO-fishing-area-27.1

Polish fishing fleetsoperate in the Baltic Sea, which can be divided into the following fishing sub-areas.

Polish sub-territorial waters include part of fishing sub-areas: 25 and 26

fishing areas

14

Approach

In our process, which we want to implement for the production of statistics, we assume measuring activity and traffic of the fishing fleet.

Activity of fishing fleet - time when a fishing vessel is out of port. That each fishing vessel is associated with a register port. In order to check ship's activity, we refer to its port.

Traffic of fishing fleet - movement of fishing vessels in fishing areas. In this case, we refer to fishing areas.

stat.gov.pl

15

ApproachObjective: Measuring the activity of fishing fleets

• Using a common reference of fishing fleets to filter out only maritime fishingfleets;

• Definition of time and area (fishing ports);

• Definition a rectangular area based on coordinates;

• Using a code developed in Scala or Elasticsearch (geo data).

Variables:

• The average (max, min) time of fishing fleets activity depending on the season and/or weather conditions.

• Fast indicators in near real time :

• The number of fishing vessels in port(s).

• The number of expected entries of fishing vessels to port(s).

stat.gov.pl

16

ApproachObjective: Measuring the traffic of fishing fleets

• Using a common reference of fishing fleets to filter out only maritime fishingvessels;

• Definition of time and area (fishing areas);

• Definition a rectangular area based on coordinates;

• Using a code developed in Scala or Elasticsearch (geo data).

Variables:

• The average (max, min) distances travelled per fishing fleet depending on the season and/or weather conditions.

• The average (max, min) speed per fishing fleet for a specific fishing areas and time period

• The average (max, min) draught of fishing fleet for a specific fishing areas and time period

• We can also analyse traffic intensity of fishing fleet in a particular area

stat.gov.pl

17

Approach

stat.gov.pl

source: https://www.marinetraffic.com/pl/ais/

Fishing vessel's route, depends onmany factors, such as:

• having of a valid fishing license

• compatibility of a fishing gear to the type of fish caught

• weather conditions

Objective: Route of a fishing vessel - visualization for individual observations

Analyzing fishing vessel’s route and movement patterns,we are able to develop methods to detect a potential fishing behavior.The algorithm will be able to analyze the speed, draught of fishing vessel, which can help in the to define fishing activity.

18

Technology - statistical processes

stat.gov.pl

Processing dataCollecting data Analysing data

19stat.gov.pl

Open-source technology used

The aim of this phaseAIS data

collection process The aim of this phase is to obtain, decode and store AIS-data in HDFS system.

As for the technical aspect of collecting AIS-data, there are two possible approaches:- batched data collection- streaming data collection

Technology - collecting data

Spark Streaming – it is an extension of the core Spark API that enables scalable, fault-tolerant processing of live data streams

Scala - programming language combining features of functionaland object-oriented languages. Scala operates on a Java Virtual Machine

20stat.gov.pl

Open-source technology used

The aim of this phaseAIS data

processing process

Spark – fast and general large-scale data processing engine.

The aim of this phase is to validate, integrate and transform datato prepare the data for analysis.

Scala - programming language combining features of function and object languages. Scala operates on a Java Virtual Machine

Technology - processing data

HDFS – abstract file system within a cluster, stores large amounts of data, dividing them and placing them on many machines. Files are stored in the form of blocks arranged in a cluster

Ambari - management platform for Apache Hadoop clusters. It allows planning, installation, configuration and cluster management.

21stat.gov.pl

Open-source technology used

The aim of this phaseAIS data

analysing process

Leaflet – is one of the most popular R Studio libraries (JavaScript) for generating interactive maps.

The aim of this phase is to prepare the data for disseminating.

Various types of visualizations and statistical analysis are made to ensure correct interpretation of results and adapted reports.

Shiny – a framework that allows you to create complex dashboards and analytical applications, using only the knowledge of the R language.

R-interpreted programming language and environment for statistical calculations and visualization of results.

Kibana – a tool that searches Elasticsearch and visualizes information - whether in the form of a table, chart or cartogram.

Technology - analysing data

22

Conclusions

stat.gov.pl

• The process of analyzing traffic and activity of fishing fleets we plan to develop is a universal and easy to implement solution, considering the fact that only one, unified data source will be used - AIS;

• Due to the EU regulations It is possible to prepare a reference frame of fishing fleet (vessels) for all EU countries (public EU fishing fleet register)

• Technology and data availability (national level) are sufficient to prepare a functional prototype (procedures, codes) that will be replicable by other NSIs.

23stat.gov.pl

Thank you for your attention!