Finnish Meteorological Institute opened its data in 2013.
Basically everything that FMI has property rights was opened.
Data is provided in freely in machine readable format.
29.4.2015 INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo 2
FMI Open Data
https://en.ilmatieteenlaitos.fi/open-data
FMI Open Data Portal follows INSPIRE requirements.
29.4.2015 3
FMI Open Data Portal
Meta data
Data Models
Services
The very same data portal works as Open Data and
INSPIRE portal.
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Catalog Service
(CSW)
o Based on GeoNetwork
29.4.2015 4
View Service (WMS)
o Based on GeoServer
o Only the most common layers
published
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Download Service
(WFS 2.0)
o Web Feature Service (WFS) 2.0 Simple Profile
o Based on stored queries
o Predefined data sets with possibility for additional parameters (i.e. time and area)
o In-house production
29.4.2015 5INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Data set Description Time
Interval
Estimated
publish date
Weather
Observations
Temperature, Wind,
Humidity, Ground
Temperature…
10 min Open,
older data to be
added
Sun Radiation UV, Short and Long
Term Radiation…1 min Open
Marine
Observations
Waves, Sea
Temperature, Sea
Level…
1 h Open
Weather Radars Precipitation Rate,
Precipitation Amount…5 min Open,
older data to be
added
Lightning Thunder Strikes in
Finland5 min Open
29.4.2015 INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo 6
Example of Data Sets
29.4.2015 7
Example of Data Sets
Data set Description Time Interval Estimated
publish date
Real Time
Observations
Real Time Observations from
specific location(s)
AWS 2010 –
Soundings 1959 –
Flashes 1998 –
Sea Level 1971 –
Waves 2005 –
Open
older data will
be added
Climatological
Observations
Dayly and monthly
temperature mean and
extreme values from weather
stations
1959 - Open
Climatological
Observations
Monthly temperature and
precipitation rate mean
values interpolated to grid
1961 - Open
Climatological
Reference
Climatological Reference.
Temperature, humidity,
pressure, precipitation
amount and snow depth.
Reference seasons:
1971-2000 1981-
2010
Open
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
29.4.2015 8
Example of Data Sets
Data set Description Time Interval Estimated
publish date
Weather forecast
model HIRLAM RCR
Point forecasts and grid
dataLatest model
run
(4 times a day)
0…54 h
Open
Sea forecast models Sea level point
forecasts, Wave (WAM)
and current (HBM) as
grid data
Latest model
run
(4 times a day)
0...54 h
Open
Environmental
Monitoring Facilities
Weather observation
stations, radars…2015
Aviation
Observations
METAR 30 min open
Ground & mast
observations
Special observations
from ground and masts2015
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
29.4.2015 9
Example of Data Sets
Data set Description Time Interval Estimated
publish date
Air Quality
Observations
Air Quality Observations 1h 2015-2016
Silam Model Dispersion Model for Air
Quality, Forest Fire and
Pollen
Latest model
run (once a day)
0…96h
2015
HELMI Ice Model Ice forecast model Latest model
run
(4 times a day)
0...54 h
open
Soundings Temperature, Humidity,
Pressure, Wind from
ground to 25 km height
2 times a day 2015
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Data Models
o Observations and point forecasts as GML
o The same data is published in:
o MultiPointCoverage
o MeasurementTimeSeries
o SimpleFeature
o Gridded data is provided in
appropriate binary format (Grib,
NetCDF, GeoTiff…)
o WFS members contains the metadata ‘envelope’ with a link to a actual data
29.4.2015 10INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Data Models
gmlcov:MultiPointCoverage
29.4.2015 11
gml:rangeSet
gml:doubleOrNilReasonTupleList
The data is listed for every
point defined in domain set.
gml:domainSet
gmlcov:simpleMultiPoint
The coverage is
defined as a list of
points in 4
dimensional grid (lat,
lon, height, time).
gmlcov:rangeType
The parameters
listed in range set
are defined in
separate element.
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Cons
- Not intuitive
- No natural
structure of XML
XSLT and
Xpath don’t work
Pros
+ Compact
+ Efficient
+ Small file size
+ Works for many
data types
29.4.2015 12
Data Models
gmlcov:MultiPointCoverage
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Data Models
wml2:MeasurementTimeseries
29.4.2015 13
wml2:MeasurementTimeseries
One member contains time
series for one parameter
and one location
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Cons
- Lots of repetition
- Large file size
- Heavy for DOM-
based parsers
- Don’t work i.e. for
thunder strikes
Pros
+ Intuitive
+ Easy to use
+ XSLT & XPath
works
29.4.2015 14
Data Models
wml2:MeasurementTime
series
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Data Models
SimpleFeature
29.4.2015 15
SimpleFeature
One member contains one
time, one parameter and
one location
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Cons
- Lots of repetition
- Very large file size
- Heavy for DOM-
based parsers
Pros
+ Intuitive
+ Easy to use
+ XSLT & XPath
works
+ Ready client
support
29.4.2015 16
Data Models
SimpleFeature
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
29.4.2015 17
Data Type Data Format
Observations wml2:MeasurementTimeseries
gmlcov:MultiPointCoverage
SimpleFeature
Point Forecasts wml2:MeasurementTimeseries
gmlcov:MultiPointCoverage
SimpleFeature
Lighting Observations gmlcov:MultiPointCoverage
SimpleFeature
Grid Forecasts XML Envelope + Grib2/NetCDF
Radar Images GeoTiff / PNG images
METAR IWXXM
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
29.4.2015 18
Data Models File size Comparison
81.7
52.9
1.81.3 1.2 0.20
102030405060708090
Document Size[MB]
CompressedDocumentSize[MB]
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
29.4.2015 19
Data Models Popularity
Comparison
80
19.8
0.20
102030405060708090
Downloads[%]
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Intranet
DMZ
Server 1
GS 1 GS 2 GS 3
Server 2
GS 1 GS 2 GS 3
Server 3
GS 1 GS 2 GS 3
Load Balancer
Configuration
GeoServer
Data
(NFS)
Configuration
(NFS)Database
29.4.2015 20INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Intranet
DMZ
Backend
(WFS)
Load Balancer
Data
(NFS)
Configuration
(NFS)Database
29.4.2015 21INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Frontend Frontend
Backend
(WFS)
Backend
(binary data)Backend
(binary data)
Open Data Service
Cluster
S1 S2 S3
Client Data Service
Cluster
S1 S2 S3
Load Balancer
Configuration
Data
(NFS)
Configuration
(NFS)Database
29.4.2015 22INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Weather Data – Volumes
29.4.2015 23
• In-situ weather measurement 1 TB
• Weather radar data 50 TB
• NWP model gridded data (FMI)
• HIRLAM 120 TB
• AROME 230 TB
• Satellite image data (FMI)
• Globsnow 32 TB
• Other 50 TB
• Climate model data (FMI) 29 TB
• Other models
• SILAM 100 TB
• Tuuliatlas 21 TB
• Other 300 TB
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Producing INSPIRE Data Products
Observations
29.4.2015 24INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Database BS Data ServerBS Data Server
WFS Plugin
Producing INSPIRE Data Products
Point Forecasts
29.4.2015 25INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
File
SystemBS Data Server
BS Data Server
WFS Plugin
Producing INSPIRE Data Products
Grid Forecasts 1/2
File
SystemBS Data Server
BS Data Server
WFS Plugin
29.4.2015 26INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Producing INSPIRE Data Products
Grid Forecasts 2/2
File
SystemBS Data Server
BS Data Server
Download
Plugin
29.4.2015 27INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Producing INSPIRE Data Products
Radar Images 1/2
PostGIS
DBBS Data Server
BS Data Server
WFS Plugin
29.4.2015 28INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Producing INSPIRE Data Products
Radar Images 2/2
PostGIS
DBGeoServer
29.4.2015 29INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
INSPIRE Data Sets
How to define a data set?
o All weather observations from
Finland?
Would cause over 50 000 000 Observations (XML file size ~37 G)
o All observations from one
observation station?
Would cause over 200 data sets
o Even one year’s observations cause
too large data set to handle
29.4.2015 30INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
INSPIRE Data Sets
Meteorological data is a constant
flow of observations
FMI has one data set per data
type, i.e. one for ground weather,
observations, one for Hirlam
weather forecasts, etc…
Every data set have predefined
area and time range.
29.4.2015 31INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
INSPIRE Data Sets
It is notable that data set
response depends on time it’s
requested
Unique identifiers are not
reasonable
29.4.2015 32INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
And a little over300 000 data downloads
per day(3,7 req/s)
At the moment about 7200
registered users
29.4.2015 33
Some Experiences
INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Practicallyno clientsupportscomplexfeatures
Although standards are followed, there’s
a gap between provided data model
and clients’ capabilities
29.4.2015 34INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Some Experiences
GeoServer is modified to support
stored queries in WFS 2.0 (released
in version 2.7)
FMI is going to open the same data as simple features to
support clients
29.4.2015 35INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Some Experiences
Industry ishappy to usestandardized
services
Amateur andfreelancer coders
would prefer simple JSON API
29.4.2015 36INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Some Experiences
So far maybe
even more
professional
interest than
private
Quite many
expected a user
interface to load data
to i.e. to Excel
instead of machine
readable interface
29.4.2015 37INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Some Experiences
…but suites quite
well for exchanging
(subsets of) data.
29.4.2015 38INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Data format is too
verbose for clients to
use directly…
Some Experiences
For now,
very few have
been interested in
forecast models
as a grid data
Point forecasts,
observations and
radar images are the
most interesting data
types
29.4.2015 39INSPIRE Data Models | Finnish Meteorological Insitute | Roope Tervo
Some Experiences
www.fmi.fi
http://www.slideshare.net/tervo/
https://en.ilmatieteenlaitos.fi/open-data
Top Related