SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from...

29
NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Transcript of SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from...

Page 1: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

NIWA Systems Development TeamAndrew Watkins

SDT Overview

Taking dull repetitive work from scientists and giving it to robots since 2008.

Page 2: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

People Processes & Tools

Page 3: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

The Team

Page 4: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

The Team

• Colocated • Agile Methods• Diverse Skills• Mix of Youth and Experience• Capacity and Capability• 80:20 Development & Research

Page 5: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Work on

• Dataware• Middleware• Web services• Web User interfaces• Thick and thin application clients• Mobile, fax, and email notification services• Other Technical Stuff

Page 6: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Don’t Work on

• Static Web content - Communications• Office Systems – IT Operations• Intranet• GIS• Scientific programming• Model Development• High Performance Computing Facility

Page 7: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Key Relationships

• IT General Manager – Arian de Wit• IT Operations – Anthony Cole• Chief Scientist EI – Jochen Schmidt• GIS – Arne Pallentin• Scientists• Public and Science Users• Peer groups

Page 8: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

People Processes & Tools

Page 9: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Processes

• Agile development methodology

• Full Software Engineering Life Cycle

• Strategic Planning

Page 10: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Agile Software Development

• Scrum• Iterative development• Test driven development• Continuous Integration• Virtual Environments• Evolving process

Page 11: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Software Engineering Baseline Standard

• Revision control.• Auto build process and development project• Test Coverage• Deployment System• Documentation• Issue Tracking and change control process• Run Time monitoring• Technical support plan and SLA.• Users and Access levels are identified• Data is managed

Page 12: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Data Management Process

• Work in progress• A big sell to the science staff• Overcoming hoarding• Making metadata available• Quality control chain• What can software do?

Page 13: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Managing Multiple Projects

• Over 80 online services• Using Wiki to document – basic metadata• Many legacy systems• Ongoing Maintenance• Sprints – burst of focus to delivery• JIRA Issue Tracking – custom workflow• Prioritisation

Page 14: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

People Processes & Tools

Page 15: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Atlassian Confluence – One.NIWA

Page 16: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Atlassian Confluence – One.NIWA

Page 17: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Atlassian Confluence – One.NIWA

Page 18: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Atlassian JIRA

Page 19: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Atlassian JIRA

Page 20: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Eclipse

Page 21: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Subversion

Page 22: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Cruise Control

Page 23: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Selenium Web App Test

Page 24: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Programming Languages and Frameworks

• Java• Frameworks – Struts, Seam• N-Tier systems, Components• Web services, Web UI

• PHP• Frameworks – Symfony, Zend• Simpler systems, Front ends

Page 25: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Relational Database

• PostgreSQL/ PostGIS– Free, Open Source, Full Featured– Strong Geospatial Support

• Oracle– Perceived Quality, Performance– License costs – Distribution issues.

• Others - Legacy– Informix, Empress, Paradox, MySQL– Actively retiring

Page 26: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

Non Relational Database

• eXist– XML datastore– Good for reference information– Lookups, Code lists– XQuery– XSLT

Page 27: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

NIWA Systems

Page 28: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

NIWA Systems Goals

• Web Friendly• Restful• Documented APIS• Open Data by default• Open Source tools first choice• Mostly Free• Standards Compliant

Page 29: SDT NIWA Systems Development Team Andrew Watkins SDT Overview Taking dull repetitive work from scientists and giving it to robots since 2008.

NIWA Systems Goals

• Avoid Vendor Lock in

• Avoid CMS Embedding

• Avoid Brand Embedding

• Don’t reinvent wheel

• Have Fun