Status report on the implementation of GISC at CMA Li Xiang NMIC, CMA 2010.2.
-
Upload
jacob-ortiz -
Category
Documents
-
view
228 -
download
0
Transcript of Status report on the implementation of GISC at CMA Li Xiang NMIC, CMA 2010.2.
Status report on the implementation of GISC at CMA
Li XiangNMIC, CMA
2010.2
To support WIS/GISC’s functions and provide the services of DAR, Data request, Data subscription and GTS data ingest to WIS centers and authorized users.
• Metadata generation
• Metadata Synchronization
• Metadata search
• Data subscription
• Data dissemination
• Web interfaces
Development of GISC service
Metadata Information Generating
VolC1VolC1
......Table B1-B6 from OP for GTS code
list.Vol A
Static Referance Information.
XML File
VolC1 +VolC1 +
Template
GTS Bulletin
CMA Satellite DataCMA TIGGE Data
CMA NWP Data
Metadata Basic Info.
Metadata Generation
Metadata File Generating
• Metadata files: 114,495, including the metadata files for
– GTS data (Vol C1) : 59,175
– CMA NWP + FY satellite data : 1,222
– TIGGE products: 54,098 (10 centers’ products)
Metadata Generation
Auto-generation - DAR metadata for GTS Bulletin
• WMO Volume C1 (Catalogue of Meteorological Bulletins)
- CMA NWP, FY Satellite Data, TIGGE data
Repositories
M e t a d a t a
D a t a B a s e
Metadata Files
Harvest Service
Metadata CheckProviders
Metadata
FTP collect
Provider Service
HarvestersMetadata Files FTP out
Metadata synchronization system
JMA,DWD …
JMA,DWD …
Provider System
Harvest System
Metadata Synchronization (OAI, FTP)
Development environment
• Java jdk1.5
• Oracle 10g
• OAI framework:
- OAICat
- OAIHarvester2
• Web container
- Tomcat
Metadata Synchronization – provider configuration, monitoring
Metadata Synchronization – harvester configuration, monitoring
• Directory navigation
Metadata Search
• Simple Search (keywords, title, time, …)
• Complex Search (geographical location+keywords+…)
• SRU service, being implemented based on OCLC’s SRW/SRU open source
Data subscription
• Web Interface (subscribe data)
• Subscription parser (data order, dissemination metadata)
• Service management (queue, monitoring)
• Data dissemination (ftp, http, email)
BeginOrders
PreparationOrders
Submission Orders Database
Order Metadata
Metadata Database
Delivering
Metadata
OrdersAnalysis
Queue Management
Data Collection
Local Database
Packing, compression and other operational
configuration
Regulation for the file naming
Delivering SystemEnd
Send
Data dissemination
• GTS– protocols: ftp, tcp sockets
– other GTS centers
• Internet– protocols: ftp, http, email
– other centers, end-users
• Implemented the services– Metadata synchronization
• OAI-PMH, FTP
– Remote data request/reply• Access JMA data from CMA WIS portal• Access CMA data from JMA WIS portal
– Web Data Ingest• CMA:bulletin-based interface, TAC->BUFR• JMA: report-based interface• software shared between CMA and JMA
Collaboration with JMA
• Data Synchronization using blog-based technology
• Metadata– DWD Metadata generator– Dissemination metadata
• Contents, XML template• Service backup
– CMA Metadata synchronization components: harvester and provider
Collaboration with DWD
• Data synchronization test (2009.9)– AFD/FTP/SFTP, BLOG/HTTP/HTTPS
– test results: AFD has a better performance on making routine data exchange
Collaboration with DWD
• Metadata synchronization test (2010.1)
• Test beds– CMA: in-house software– DWD: GeoNetwork
• Test results– CMA harvesting metadata from DWD provider : 12387 metada
ta in 14940s ( 1.206s/rec )– DWD harvesting metadata from CMA provider : 47326 metada
ta in 64441s ( 1.361s/rec )
-> more tests planned in March-> analyze performance and make improvement
• Participants: • CMA, DWD, JMA• WISPO
• Discussion on• DAR Metadata • Data Synchronization• Metadata Synchronization • Service Monitoring and Service Backup
• Outcomes• test plans for metadata synchronization
• the document related to DAR metadata, including metadata file identifier, usage of keywords, usage of ISO element <CI_OnlineResource> for accessing online data and products, and submited it to CBS expert team.
Collaboration meeting on WIS development
Collaboration with HKO
• HKO will operate a DCPC in connection with the World Weather Information Service (WWIS) and Severe Weather Information Centre (SWIC) under the WMO Public Weather Services Programme.
• Beijing will be one of its the associated GISC
• NMIC, CMA and HKO made discussion on WIS development in Beijing in November, 2009– DAR metadata
– metadata provision
• Metadata provision tests– planned in March
Progress
Metadata generation √√Metadata Synchronization
√√
Metadata search 80%
(SRU service will be available in March)
Data subscription 80%
Data dissemination √√Web interface 70%
GISC service backup Planned in 2010