Cavity BPM processing and hardware upgrade plans 24.02.2015 A. Lyapin, S. Boogert, E. Yamakawa.
A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya,...
-
date post
21-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya,...
![Page 1: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/1.jpg)
A tool to dynamically merge corresponding weather data to crop data
Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa
National Agriculture Research Center, Tsukuba, Japan
![Page 2: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/2.jpg)
Dead Storage Data Issue A lot of digital data sets are continuously produced in
agricultural experimental stations Using ordinal software such as spread sheet applications But they are likely to be kept in local stations and scientist level
The data sets are isolated and hardly integrated among different locations
How to ease data publication for merging and sharing for end users
![Page 3: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/3.jpg)
Different resource integration issue Separated crop data are hard to be
integrated with different resources, e.g. weather data
Heterogeneity e.g. Models constructed using local
data are only applicable locally How to integrate crop data with
weather data under user’s request
![Page 4: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/4.jpg)
If merging and sharing are possible,
End users can freely combine separated data sets from different locations and perform analysis on them
Datamining over the huge amount of data sets becomes real and we can possibly find out unknown facts
Integration with completely different resources e.g. weather data becomes also possible
Constructing global model also becomes possible
![Page 5: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/5.jpg)
With Web-based data sharing & integration
Once you upload experimental data in spreadsheet file (e.g. MS Excel), data automatically become sharable over the Internet among different locations
Crop data can be integrated with weather data No skill is required Just a part of everyday data management Uniformity of tables is not required Low cost
![Page 6: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/6.jpg)
Internet
DBMS
Application Server
Servlet Container
EJB Container
Application
Concept
Multi-location data sharing and integration Integration with weather data
MetBroker
![Page 7: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/7.jpg)
Basic structure of application
Google Map
Client
Brow
ser
Controller
Crop DBCrop Data Service
Model
ServicesWeb application
MetBroker
AMeDAS
AMeDAS
AMeDAS
View
![Page 8: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/8.jpg)
System components
Java Runtime Environment 1.5.04
PostgreSQL7.4
JRE supported OS
JBoss-4.0.3
EJB3.0(DBMS abstraction)
Struts1.2(Web Interface)
・ IE, Firefox, etc.・ Excel2002, newer
![Page 9: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/9.jpg)
Main menu Very simple menu structure Simple and clean user interface
![Page 10: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/10.jpg)
Data upload
Crop Data ServiceEJB3
SourceXML
CropData
HistoryData transforming
Data validating
XSLT style sheet
Data Schema
a
b
c
Crop Database
a. Transformation of source data file using style XSLT style sheet, storing source data file in XML to SourceXML table.
b. Validating transformed data XML using data schema c. Data service binds data XML to data object (JAXB), maps data object to data records t
o CropData table; stores uploaded information including user name, spreadsheet file name, uploading time, count of data record to History table.
![Page 11: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/11.jpg)
![Page 12: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/12.jpg)
Upload history
![Page 13: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/13.jpg)
Web Application
Crop Data Service EJB3
Crop db
Specifying query conditions then executing data query.
Browsing and/or download queried crop data
Data Query
![Page 14: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/14.jpg)
AMeDAS
Web Application
Crop Data
Location Table
Crop Data Service EJB3
MetBroker
ab
AMeDASAMeDAS
cd
Data integration
![Page 15: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/15.jpg)
Location registration
![Page 16: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/16.jpg)
Weather station selection
![Page 17: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/17.jpg)
Discussion
37 spreadsheet files uploaded 151072 records 6486 varieties 184 properties 256 different experiment locations 1980-2004
Available to open society
![Page 18: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/18.jpg)
Data integration is realized in our brokerage based Data Grid MetBroker is the core of the Data Grid
The application demonstrates the power of Data Grid
This approach can be used to all other crop data sharing and integration
Crop modelers are using this application to develop robust crop models
![Page 19: A tool to dynamically merge corresponding weather data to crop data Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa National Agriculture Research Center,](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6b5503460f94a4a5ea/html5/thumbnails/19.jpg)
Thank you for your attention