NCL and PyNGL Community data analysis and visualization software 2005 User Forum, May 17-19 ...
-
Upload
beverly-francis -
Category
Documents
-
view
217 -
download
0
Transcript of NCL and PyNGL Community data analysis and visualization software 2005 User Forum, May 17-19 ...
NCL and PyNGLCommunity data analysis and
visualization software
2005 User Forum, May 17-19 CISL/IMAGe/SCD Mary Haley, [email protected]
2005 CISL User Forum May 17-19 [email protected]
Topics• NCAR Command Language
– Quick overview– Workshop update– What’s new– Near future plans
• Python interface to NCL’s graphics library– Introduction– Near future plans
• Long range plans for NCL and PyNGL
2005 CISL User Forum May 17-19 [email protected]
NCL - Quick overview
• An interpreted language designed for the analysis and visualization of data
• Powerful in three areas: file I/O, data analysis, visualization
• Runs on most UNIX systems, including MacOSX, LINUX, Windows/Cygwin
• FREE
2005 CISL User Forum May 17-19 [email protected]
NCL overview: file I/O• Data model based on netCDF model• One function to read multiple data formats• Handles netCDF, GRIB-1, HDF4, HDF-
EOS4, CCM History tape• GRIB reader one of the best• ASCII, binary
Answer to question from yesterday: NCL supports NARR data and includes the NARR parameter table (NCEP parameter table 131).
2005 CISL User Forum May 17-19 [email protected]
NCL overview: data analysis• Over 800 functions
- Spherical harmonics- Scalar and vector regridding- Vertical interpolation- EOFs
• Many tailored to geosciences• Most handle missing data• Can call C and Fortran routines
2005 CISL User Forum May 17-19 [email protected]
NCL overview: visualization
• High-quality visualizations
• Contours, XY, vectors, streamlines, maps
• Specialized scripts for skew-T, wind roses, histograms, panels, bar charts
• GSUN interface: NCL scripts for simplifying visualization calls
• Hundreds of plot options available
2005 CISL User Forum May 17-19 [email protected]
NCL strengths• Huge web-based example suite• NCL workshops• Consulting is knowledgeable and quick • Trivial to install, precompiled binaries• Enhancements based on user demand*
* Get ready for the data analysis and visualization survey!
2005 CISL User Forum May 17-19 [email protected]
NCL workshops• 3 full-day classes with lectures and labs• Taught by CGD staff• 22 in the last 4 years: 15 in Boulder, rest off-site:
– University of California at Santa Cruz– University of Alaska at Fairbanks– UCLA (twice)– Purdue– Naval Research Lab - Stennis Space Center– National Ocean Service - NOAA
• Cancelled until further notice
2005 CISL User Forum May 17-19 [email protected]
What’s new
• New NCL website
• Command line options
• Contouring on triangular meshes
• NCL binaries on Earth System Grid
• Enhancements to GRIB reader
• Vis5D writer functions
Online survey later this month
2005 CISL User Forum May 17-19 [email protected]
New NCL websitehttp://www.ncl.ucar.edu/
• Consolidates ngwww.ucar.edu and the www.cgd.ucar.edu graphics website
• Function documentation in one location
• “Applications” includes over 500 examples
• Lots of documentation; printable manuals
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
2005 CISL User Forum May 17-19 [email protected]
Command line option preview
• Predefined options (-V, -x, -P)
• Can assign variables and execute commands:ncl i=1 x=3.14 ‘vars=(/“T”,“U”,“V”/)’ file.ncl
• More options to come: byte-swapping control, printing control; would like to hear from you!
http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/
2005 CISL User Forum May 17-19 [email protected]
Contouring on triangular meshes
Dave Kennison and Dave Brown
2005 CISL User Forum May 17-19 [email protected]
New types of grids you can contour
• Grids already on a triangular mesh
• Grids represented by one-dimensional arrays of x, y, and z values
• Grids with “cell boundary” information
Grid from Tom Gross, NOAA/NOS/CSDL/MMAP
Grid from Tom Gross, NOAA/NOS/CSDL/MMAP
Grid from Brett D. Estrada, Naval Research Laboratory
Grid from Brett D. Estrada, Naval Research Laboratory
HOMME grid courtesy of Steve Thomas, SCD
HOMME grid courtesy of Steve Thomas, SCD
HOMME grid courtesy of Steve Thomas, SCD
From Dave Randall, CSU
From Dave Randall, CSU
http://isccp.giss.nasa.gov/
http://isccp.giss.nasa.gov/
Christophe Cassou (CNRS/CERFACS)
Christiane Jablonowski (ECMWF)
Christiane Jablonowski (ECMWF)
2005 CISL User Forum May 17-19 [email protected]
To see some NCL examples
go to:
http://www.ncl.ucar.edu/Applications/
and click on “Non-uniform grids”
2005 CISL User Forum May 17-19 [email protected]
NCL near future plans• CLOs - initial release• Revisiting our I/O module: HDF5, GRIB2,
netCDF 4 coming up• Streamlines and vectors on a triangular
mesh• Large file support, currently has same
limitations as older versions of netCDF
Did I mention the survey?
2005 CISL User Forum May 17-19 [email protected]
Sites that have downloaded and/or used NCL
in the last few years
Department of Meteorology, Yugoslavia
Nanjing University, China
University of Michigan
UCLA
METRI/KMA, Korea
Frontier Research System for Global Change, Japan
Iowa State
Hong Kong University of Science and Technology
UCSD
Czech Hydrometeorological Institute
Osservatorio Meteorologico Regionale, Italy
Institute for Atmospheric and Climate Science, Switzerland
University of Reading, UK
International Centre for Theoretical Physics, Italy
NOAA-CIRES Climate Diagnostics Center
Cooperative Institute for Mesoscale Meteorological Studies
Woods Hole Oceanographic Institution Joint Program
Department of Geology and Geophysics, Yale
NOAA/NOS/Coast Survey Development Laboratory
Center for Ocean-Land-Atmosphere StudiesDepartment of Physics (Atmospheric Physics), University of TorontoUniversidade Federal da Bahia New Mexico Institute of Mining and TechnologyPacific Northwest National LaboratoryAtmospheric Science Division at LLNLArgonne National LabUniversity of ColoradoLawrence Berkeley National LabUniversity of IllinoisNRL, NavyClimate Change & Impacts Laboratory, UCSCUniversity of WashingtonUniversity of ArizonaArctic Region Supercomputing Center, University of AlaskaCenter for Sustainability and the Global Environment, Univ of Wisconsin-
MadisonFleet Numerical Meteorology and Oceanography CenterAir pollution Section/Research Department, Norwegian Meteorological InstituteFrontier Research System for Global Change, University of Alaska FairbanksUniversity of Virginia
University of NevadaAstronomy, Physics and Geophysics, University of CopenhagenInstitute of Urban Meteorology, Beijing, ChinaScripps Institution of Oceanography, University of CaliforniaCERFACS, FranceEMS Environment Institute, Penn StateCentre Meteo UQAM Montreal, MontrealNational Research Institute for Earth Science and Disaster Prevention, JapanUniversity of L'Aquila, Department of Physics, ItalyInternational Research Institute for Climate Prediction, Columbia UniversityAlaska Experimental Forecast FacilityCentral Research Institute of Electric Power Industry, JapanNational Severe Storms LaboratoryLamont-Doherty Earth Observatory, Columbia University, NYUBC Earth and Ocean Sciences, CanadaEarth Tech, Concord, MassachusettsSchool of the Environment, Leeds University, UKInstitute For Terrestrial and Planetary Atmospheres, State Univ of NYCollege of Oceanic and Atmospheric Sciences, Oregon State UniversityInstitute of Geophysics - Climatology, University of Bergen, NorwayInstitute for Geophysics, UT-Austin
University New South WalesIndonesia National Institute of Aeronautics and SpaceUniversity of NebraskaQueensland Department of Natural ResourcesNC State UniversityUniversity of Costa RicaLouisiana State UniversityMITUniversity College LondonNagasaki UniversityDeutscher WetterdienstInstitute of Atmospheric Physics, Chinese Academy of SciencesUnited Nations University YSA Corporation Chesapeake Research Consortium Naval Postgraduate School University of North Carolina at Chapel HillResearch Center for Environmental Changes, TaiwanIMAQS, U. of Houston Pusan national University, KoreaUniversity of Victoria, British Columdia, Canada
Institute of Oceanology, RussiaRudjer Boskovic Institute, CroatiaMississippi State UniversityAdvanced Environment Monitoring Research Center, KoreaKyushu University, KoreaUniversity of Northern British ColumbiaKorea Meteorological AdministrationInternational Pacific Research Center/School of Ocean and Earth Science, HawaiiGerman Weather ServiceUniversity of WaterlooComalco Aluminium Ltd, New ZealandAltai State University, RussiaUniversity of MaineIQAFF, QuebecKinetica Analysis Corporation/University of Central FloridaCentre for Mathematical Modelling and Computer Simulation, IndiaUniversity of British Columbia Howard UniversityUniversidad del Pais VascoIndian Institute of ScienceUniversity of Delaware
PrincetonCentral Weather Bureau of TaiwanSeoul National University, KoreaCNR-Venice-ItalyWeathernews Americas Inc.Mephise Technology (Taiwan)Purdue UniversityOhio State UniversityUSDA Forest ServiceInternational Arctic Research CenterCalifornia Air Resources BoardU.A.E. Air Force Air DefenseUniversity of NebraskaCWB (Center for Wooden Boats? Central Weather Bureau? Canadian Wheat
Board?)GFDL/NOAAChung-Cheng Institute of TechnologyNational Renewable Energy LabZhejiang Climate Center, BeijingUniversity of Texas at AustinWageningen University, Netherlands
University of Athens Macao Meteorological Bureau, Hong KongDuke University Texas A&M University SD School of Mines Department of Marine, Earth and Atmospheric Sciences, NCSU Universidad Politecnica de Catalunya, SpainNational Taiwan UniversityUniversity of Pecs, HungaryKobe University, JapanUniversity of Tartu, EstoniaUC, Irvine Institute of Meteorology, University of EdinburghSan Francisco State University University of Kentucky College of Resources Science and Technology, Beijing Normal University
Meteo-France AgrometLS Cable Ltd., chiller developmentPotsdam Institute for Climate Impact Research, GermanyARPA Piemonte, Italy
USDA Animal Improvement Programs LaboratoryMarine Meteorological Observation and InvestigationDrexel University Indian Institute of Technology, DelhiCaltechAbu Dhabi International Airport Meteorological OfficeMichigan Tech UniversitySongXiaoya UniversityYonsei University, Seoul, KoreaJackson State UniversityClimate Research, Tokyo University
2005 CISL User Forum May 17-19 [email protected]
Python interface toNCL’s Graphics Library
(PyNGL)
Fred Clare and Mary Haley
http://www.pyngl.ucar.edu/
2005 CISL User Forum May 17-19 [email protected]
What is Python?
An interpreted, interactive, object-oriented programming language often compared
with Perl, Tcl, and Java.
2005 CISL User Forum May 17-19 [email protected]
Python strengths
• Open source and portable• Full object oriented capabilities• Well documented and easy to learn• Modular• Toolkits for building GUIs - Tkinter• 3D visualization - vtk, OpenGL• Extendable with C/C++ and Fortran• Large and rapidly growing user base
2005 CISL User Forum May 17-19 [email protected]
Python strengths: lots of existing code
• PyTables - HDF5 reader• PIL - Python Imaging Library • Libraries of scientific tools
- netCDF module - FFTPACK, SPHEREPACK, LAPACK
• PyGlobus - toolkit for GRID computing
2005 CISL User Forum May 17-19 [email protected]
Why a Python interface?
• Demand for high-quality 2D visualizations
• Mainstream language - wider audience
• Open source
• Collaboration opportunities
• Capitalize on the work of others
• Contribution to Python community
2005 CISL User Forum May 17-19 [email protected]
Introduction to PyNGL
• Modeled after popular GSUN interface• Same 2D publication-quality graphics as NCL• Multiple output formats - same as NCL• Extensive documentation• Line-by-line explanation of examples
import Nglfrom Scientific.IO.NetCDF import *
# Open the NetCDF file.nf = NetCDFFile("meccatemp.cdf","r")
# Get lat/lon/temperature variables.lat = nf.variables["lat"][:]lon = nf.variables["lon"][:]T = nf.variables["t"][0,:,:]
# Open a PS workstation.wks = Ngl.open_wks("ps","mecca")
# Contour & scalar field resources.res = Ngl.Resources()res.sfXArray = lonres.sfYArray = latres.cnFillOn = True
# Draw contour plot.contour = Ngl.contour(wks,T,res)
Ngl.end()
PyNGL
load "gsn_code.ncl"
begin; Open the NetCDF file. nf = addfile("meccatemp.cdf","r")
; Get lat/lon/temperature variables. lat = nf->lat lon = nf->lon T = nf->t(0,:,:)
; Open a PS workstation. wks = gsn_open_wks("ps","mecca")
; Contour & scalar field resources. res = True res@sfXArray = lon res@sfYArray = lat res@cnFillOn = True res@lbPerimOn = False
; Draw contour plot. contour = gsn_contour(wks,T,res)
end
NCL/GSUN
Data from the MECCA Project
2005 CISL User Forum May 17-19 [email protected]
Sample PyNGL visualizations
Dummy data
Courtesy Dennis Shea
Data from the MECCA Project
Data from the MECCA Project
Data from the MECCA Project
Courtesy Sylvia Murphy, NCAR ACD
From John Ertl, FNMOC
Based on a visualization of Joel Norris (Scripps) using dummy data
Climate division data courtesy Mark Stevens, NCAR CGD
Courtesy Mark Stevens, NCAR CGD
2005 CISL User Forum May 17-19 [email protected]
Current PyNGL status• Documentation complete (200+ pages) • Released PyNGL alpha in January 2004• Announced PyNGL beta in September at SciPy• Implementing processing and I/O modules • Researching compatibility with other packages
- especially 3D and file I/O• Responding to user requests and questions
Early reactions have been positive
2005 CISL User Forum May 17-19 [email protected]
Near future PyNGL plans
• Finish beta test
• Release processing and I/O modules for alpha testing
• Add more functionality and examples
• Announce to Python world
• Potential collaboration with other Python developers
2005 CISL User Forum May 17-19 [email protected]
Long range plans forNCL and PyNGL
• Re-architecture the data model
• Performance enhancements - mainly file I/O (aggregation)
• Image processing
• Review of display model
• Open source
2005 CISL User Forum May 17-19 [email protected]
Documentation
http://www.ncl.ucar.edu/
http://www.pyngl.ucar.edu/
My email address: [email protected]
2005 CISL User Forum May 17-19 [email protected]
Questions?