GODAN Action WP1: data standards: survey, gap analysis and...
Transcript of GODAN Action WP1: data standards: survey, gap analysis and...
![Page 1: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/1.jpg)
GODANActionWP1:datastandards:survey,gap
analysisandrecommendationsFocusonweatherdatastandards
ValeriaPesceGlobalForumonAgriculturalResearchandInnovation(GFAR)
LeighDodds,JeniTennison,PaulineL'HénaffOpenDataInstitute(ODI)
PanagiotisZervas(AgroKnow)
![Page 2: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/2.jpg)
GlobalOpenDataforAgricultureandNutritioninitiative(GODAN)
TheGODANinitiativewasoneoftheoutputsoftheG8InternationalConferenceonOpenDataforAgricultureandwasannouncedattheOpenGovernmentPartnershipConferenceinOctober 2013.Theinitiativefocusesonbuildinghigh-levelsupportamonggovernments,policymakers,internationalorganizationsandbusiness.
Currentlyaround500partnersworldwidefromnationalgovernments,non-governmental,internationalandprivatesectororganizations.
www.godan.info
![Page 3: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/3.jpg)
GODANAction
Three-and-a-half-yearprogramme launchedbytheUK’sDepartmentforInternationalDevelopment.
GODANActionbringstogetheragricultureandnutritionspecialistsandopendataexpertsandwillsupportGODANinitsmissionbybuildingpeople’scapacitytoengagewithopendata.
http://www.godan.info/godan-action
UndertheDFIDGODANfundingstream
![Page 4: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/4.jpg)
GODANAction:Focalareas1) Standards - Enhancingdatastandardsandpromotingbestpracticein
agricultureandnutritiontoimproveinteroperability.1) Mapofagri-fooddatastandards2) Gapanalysisonuseandusabilityofdatastandards3) Recommendationstoaddressgaps4) à Specificationsà services,pilotimplementations
2) Research - Identifyingandimprovingtoolsandmethodsforevaluationoftheimpactofopendatausageininitiativesandinvestmentsinagricultureandnutrition.
3) Capacities - Buildingthecapacityanddiversityofopendatausers,leadingtomoreeffectiveuseofdataintacklingkeyagricultureandnutritionchallenges.
![Page 5: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/5.jpg)
Purposeoftheglobalmapofdatastandards
• Themainpurposeofaglobalmapofdatastandardsinaspecificfieldistopromotethediscoveryandreuseofvocabularies andtheirproperties,classesandcontrolledvalues.Thereuseofexistingvocabulariespromotesgreaterinteroperabilitybetweenvocabulariesanddatasets.paraphrasingwhattheDublinCoreMetadataInitiativesaysabouttheirDCMIRegistry(http://dcmi.kc.tsukuba.ac.jp/dcregistry/ )
• Helpidentifyoverlaps,duplication,gaps andlimitstoadoption,>>encouragesnottoduplicateeffortsandtocollaboratetobothdevelopandusecommonstandards
![Page 6: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/6.jpg)
Approachforbuildingmapofdatastandards
•Notduplicating,buildingonwhatexists(e.g.syncwithAgroPortal)
•Collaborativeeffort
•Broadcoverage
•Opendataangle
•Designedforgapanalysis (morelater)
Anybodycanaddorclaimastandard
Thesaurus CodelistOntology ISOspecification MessagingstandardTaxonomy
Agronomy Naturalresources Fisheries Valuechains
Allcontributorsacknowledged
Calltoactiontopartnersandexperts
Format APIs Mappings License
![Page 7: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/7.jpg)
CategorizationofdatastandardsbycontentDomain-specific• Bysub-domain:Whichdomainclassification?Attemptedone
basedonFAO+USDAclassificationsHowfartogowithneighboringdisciplines?
• Bydatatype:alignmentwithGODAN“AgriculturalSectorPackage”fortheOpenDataCharter
Sub-domains• AgriculturalResearch,Technologyand
Engineering• Agro- Economics,Businessand
Industry• AnimalScienceandAnimalProducts• EducationandAgriculturalExtension• FarmsandFarmingSystems• FisheriesandAquaculture• FoodandHumanNutrition• ForestScienceandForestProducts• Government,AgriculturalLawand
Regulations• HealthandPathology• NaturalResources,Earthand
Environment• PlantScienceandPlantProducts• RuralandAgriculturalSociology
![Page 8: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/8.jpg)
MapofstandardssofarVEST / AgroPortal MAP OF STANDARDSvest.agrisemantics.org
Numberofdatastandardsbydomain
![Page 9: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/9.jpg)
Criteriaforgapanalysis
Toidentifygaps,needtoidentifyassessmentcriteria.Criteria developedbasedon:
• TheassessmentprocessusedbytheUKGovernment’sOpenStandardsBoard
• TheODIOpenDataCertificates criteria
Categoriesofassessment:Fitnesstopurpose Adoption Usability Openness
![Page 10: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/10.jpg)
ExampleofadoptionassessmentcriteriaUsedinsoftware Isthisstandardusedinsoftwaretools(e.g.as
controlledvaluesforsomefields,orasexportformat,orasdatamodel)?
Yes,inmanytoolsthatareverypopular 5Yes,inafewtoolsthatareverymuchused 4Yes,inafewtools 3Yes,in1-2tools 2No 0Notclear- N/A 1
Usedindatasets Isthisstandardusedindatasets(e.g.asserializationformat,orasdatamodel/elementset,orascontrolledvaluesforsomecolumns/dimensions)?
Yes,inmanybymanyproviders 5Yes,inmany,byafewproviders 4Yes,inafew 3No 0Notclear- N/A 1
Endorsed Doesthestandardhaveastrongsupportfromdifferentinterestgroups?
Yes,verystrongly 3Yes,moderately 2No 0Notclear- N/A 1
Regulatory Isthestandardpublishedbyarecognizedstandardizationbodyorasagovernmentdirective?
Yes 3No 0Notclear- N/A
1
Long-term,sustainable
Isthemaintainingorganizationalong-standingandauthoritativebody?Isthemaintainercommittedtosustainandpreservethestandard?
Yes,highly 3Yes,reasonably 2No 0Notclear- N/A
1
Participatory,collaborative
Isparticipationinthecreationprocessofthestandardopentoallrelevantstakeholders?
Yes 3No 0Notclear- N/A 1
![Page 11: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/11.jpg)
Exampleofusabilityassessmentcriteria
Versatile Isthestandardavailableindifferentformatsfordifferenttechnologies?(e.g.XML,JSON,RDF)?
Yes,manyformats 3Yes,2-3similarformats 2No 0Notclear- N/A 1
ServedbyAPIs ArethereAPIsandwebservicesthatallowapplicationstoworkwiththevocabulary?Chooseasmanyanswersasyouneed.
Yes,togetweb- oruser- friendlyresults3
Yes,tolookupterms/conceptsusingseveralparameters4
Yes,toautomaticallyannotatetextordata5
Yes,toperformcross-walksbetweenvocabularies6
Yes,toextract/lookupsubsetsofvocabularies7
No 0Notclear- N/A 1
Manageable Isthestandardmanagedinacollaborativeenvironment?Isitmanagedonaspecializedvocabularymanagementplatform?
Yes,onspecializedvocabularymanagementplatform3
Yes,onacollaborativeenvironment(e.g.Github)2
No 0Notclear- N/A 1
![Page 12: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/12.jpg)
Exampleofopennessassessmentcriteria
Machine-readable
Isthestandardavailableinmachine-readableformats?(XML,CSV,RDF...)
Yes 3No 0Notclear- N/A 1
Meaningful Ifmachine-readable,isthestandardserializedusingtheappropriatevocabularylanguage(RDFs,OWL,SKOS,OBO)andsemanticallyappropriatevalues?
Yes 3No 0Notclear- N/A 1
Referenceable DoesituseURIsdereferenceableasURLsasidentifiersofclasses,propertiesandinstances?
Yes 3No 0Notclear- N/A 1
Linked IsthestandardavailableasLinkedData?I.e.serializedasRDFandabovealllinkingtoURIsinothervocabularies?
Yes 3No 0Notclear- N/A 1
![Page 13: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/13.jpg)
Mapofstandards– topic1:weatherdata• Specificusecaseofweatherdatausedinfarmmanagementinformationsystems
• Coverage– Total of 65 weather and use-case related data standards
http://vest.agrisemantics.org/by-type-of-data/7623+7550+7626/7623/7550/7626
• Refined classification of weather and use-case data types
Weather / meteorological dataGeospatial data / objectsFarm management data
Weather / meteorological dataWeather observations (live or historical)Weather monitoring infrastructureClimate dataWeather forecasts
Farm management dataeBusiness data (inventory, sales, suppliers...)Crop management data from the farmCrop growth modelsFarm sensors infrastructureObserved field data from the farm
![Page 14: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/14.jpg)
Examplesofdatastandardsrelevantfortopic1
![Page 15: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/15.jpg)
Assessmentandgapanalysis• Weatherdatastandards• Specificusecaseofweatherdatausedinfarmmanagementinformationsystems
Assessmentmetadata+consultationswithexperts
Forgeospatialandweatherdata• BenSchaap(GODANSecretariat)• GiovanniL'Abate(CRAItaly)• SimonCox(CSIROAustralia,RDA)
Forweatherdataforfarmmanagement• SoonhoKim(IFPRI,ICASAstandards)• AndresFerreyra(AgGateway)• HugoBesemer(Wageningen UR)• FrancescoBenincasa(RDAWeatherIG,
BarcelonaSupercomputingCenter)• AllarddeWit(Alterra)• ChristopherBrewster(TNONetherlands)
![Page 16: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/16.jpg)
Weatherdatastandards– Summary• Variety ofdatamodels,dataformatsandvocabulariesthatareusedto
exchangethesedata• Someolderstandardsarestillverymuchused,eitherforlegacy andcompliance
reasons(likeBUFRorGRIB)orbecauseoflong-termpracticeinresearch(NetCDF)
• Standardizationbodieshaveworkedongeospatialandobservationsmodelsandrelatedschemas(ISO/OGC,especiallytheISO19100series),startingtobeusedalsobythemeteorologicalcommunity(CSML, METCE,IWXXMschemas)
• RecentlyAPI-basedweatherdataservicesstartedservingdataondemand andinapplication-friendlyformatslikeJson anduser-friendlyformatslikeCSV
• Workonvariablenamingconventions• FMIS:subsetofweatherdatavariables;agreementwithweatherdata
providers;workonvariablenaming
![Page 17: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/17.jpg)
Gapanalysisonweatherdatastandardsa) Thebiggestchallengeswithweatherdataarerelatedtoissuesofdataavailability,
discoverability,quality,coverageanddocumentation.
b) Datastandardsarenotwelldocumented,differencesbetweenoverlappingstandardsarenotclarified.
c) Bothforweatherdataandforfarmmanagementdata,standardizationofvariablenames acrossthedifferentcommunitiesandevenwithinthesamecommunityisanissue;moreingeneral,therearemanydifferentcodelists usedbydifferentauthorities,withlimitedalignment.
d) Fewdatastandardspublishedasmachine-readableandlinkedvocabularies.
à Intermediariesstillhavetodomostoftheworkconverting,processing,re-purposingthedatabetweenthedifferentstepsinthedatavaluechain.
![Page 18: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/18.jpg)
Keyrecommendationsforbetteruseandusabilityofweatherdatastandards
• Addressdiscoveryissuesrelatingtoweatherdata(improvinguseofdiscoverymetadata tohelpcatalogueanddescribedata)• Improvethedocumentationandself-description ofexistingdatastandards(creatingdeveloperdocumentation;publishingexistingvocabulariesinnewways);offercommunitysupport,Q&Aservices• Identify2-3keycodelists thatshouldbepublishedinmorelinkable,versatileformats;linkkeycodelistsandpublishexistingalignments;providewebservicesforcross-walks.
![Page 19: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/19.jpg)
Nextstepsondatastandards
• Datapublicationonlinehelpdesk• Q&Aservice;Facilitateauthors/expertsonstandardstoregisterandprovidetheirhelp• Facilitatedatapublisherstoregisterandaskforhelpfromtheregistered
authors/expertsonstandards• Sameiterationfornutritiondataandlanddataasforweatherdata
• Usecases>surveyofstandards>gapanalysis>recommendations• Specificationsforstandardinteroperabilityservices
• Basedontherecommendations,forall3typesofdata• Interoperabilityspecificationsthatmakestandards(forall3typesofdata)interoperable
andreusable• Standardinteroperabilityservicesforpilotinterventions
• Basedonrecommendationsandspecificationsofservices• Pilotsforprovidinginteroperabilityservicesforstandardsidentifiedtousecasesfromthe
thematictopics
![Page 20: GODAN Action WP1: data standards: survey, gap analysis and recommendationss3-eu-west-1.amazonaws.com/assets.aims.fao.org/public/... · 2018-06-08 · GODAN Action WP1: data standards:](https://reader033.fdocuments.us/reader033/viewer/2022060410/5f10667d7e708231d448ecd8/html5/thumbnails/20.jpg)
Usefullinks• GODAN:http://godan.info• GODANActionmapofstandards:http://vest.agrisemantics.org• AgroPortal:http://agroportal.lirmm.fr• TheassessmentprocessusedbytheUKGovernment’sOpenStandardsBoard:Corequestions:https://standards.data.gov.uk/core-assessment-questions
• TheODIOpenDataCertificatescriteria:https://certificates.theodi.org/en/• AgSectorPackage:http://agpack.info/• DCKOSTypesvocabulary:http://wiki.dublincore.org/index.php/NKOS_Vocabularies
• Blogpostongapanalysisandrecommendations:http://www.gfar.net/news/gfar-and-odi-lead-work-gap-analysis-and-recommendations-weather-data-standards
Gapanalysisandrecommendationstobepublishedsoon.