Developing an Integrated Business Solution with Telemetry and GIS

Presented at the 7th Annual ISA Water & Wastewater and Automatic Controls Symposium

Holiday Inn Castle Resort, Orlando, Florida, USA – Aug 7-9, 2012 – www.isa.org

Developing an Integrated Business Solution

with Telemetry and GIS

Michael Waddell1* and Isabel Szendrey2

1CDM Smith Inc., Cambridge, MA, USA

(*correspondence: [email protected]) 2Puerto Rico Aqueduct and Sewer Authority, San Juan, PR

KEYWORDS

Telemetry, GIS, Integration, Decision Support, Information Management, Project Management, Data,

Analysis, Web, iPad

ABSTRACT

SCADA and telemetry allow operators to maintain systems and manage processes. Instrumentation can

require significant initial investments as well as on-going costs for what typically comprises a single,

departmental purpose. Operational data often become “silos” that cannot be easily accessed or reused,

missing a strategic opportunity to benefit the whole organization.

Between 2008 and 2010, CDM Smith and its Puerto Rico affiliate, CDM Caribbean, assisted the Puerto Rico

Aqueduct and Sewer Authority (PRASA) in implementing an island-wide telemetry system, consisting of

over 1,500 sites and 100,000 I/O points. It was one of the largest water/wastewater telemetry projects in

the United States, part of a multi-phase technology upgrade to deliver reliable, real-time monitoring and

control distributed across five regional control centers and a central headquarters. The telemetry project

objectives were to improve efficiency, reduce operating costs and comply with regulatory requirements.

While the project ensured compliance and control, PRASA and CDM Smith recognized an immediate

opportunity for extending the business benefit by leveraging the telemetry data and integrating it with its

existing web-based GIS. From concept through initial design and prototyping, CDM Smith coordinated with

the telemetry team and instrumentation vendor to build a rich, easy to use visualization interface that

enabled faster decision making, strategic insights and broader information distribution.

This paper explains the benefits and techniques for integrating data from disparate systems. Using PRASA

as a case study, it demonstrates the enhanced business value that supports longer-term monitoring

insights, managerial analysis and decision making. Likely audience members are program managers,

project managers, planners and owners.

Introduction

PRASA profile

The Commonwealth of Puerto Rico measures 100 by 35 miles for a service are of approximately 3,500

square miles having elevation ranging from 0 to 4,400 feet. PRASA’s service population includes 3.8 million

residents and 1 million tourists. The varied topography entails 180 water service areas, over 1,900 water

Waddell and Szendrey 2


Holiday Inn Castle Resort Hotel, Orlando, Florida, USA – August 7-9, 2012 – www.isa.org

tanks, 130 water treatment plants (541 MGD), 1,600 pump stations, 7,700 miles of distribution pipelines

and 3,900 miles of sewer pipe lines (NFMA-2006).

Regional operations

PRASA is a decentralized organization composed of five operating regions with an Executive President and

five Executive Directors to manage each region. Figure 1 shows a simplified organization structure (it omits

centralized headquarters functions such as human resources, security, and information systems).

Figure 1 - PRASA organizational structure, focused on regional operations

Figure 2 - PRASA Regions – Metro, East, South, West and North (Metro, Este, Sur, Oeste, Norte)

Facilities are broadly distributed and generate a wide array of information. Regional information is

gathered in various data formats because systems were built to meet local needs.

Technology Modernization

In 2004, faced with rising costs, customer needs and aging facilities, PRASA’s management elected to

transform the utility both internally and externally by establishing a long-term goal to become a world-

class utility. Meeting this challenge required the planning and implementation of an unprecedented $2.3

billion capital improvements program with island-wide coverage. CDM Smith and CDM Caribbean became




Program Manager for the West (Oeste) region in 2005 to assist in PRASA’s capital improvement program

and restructuring initiative. This initiative has evolved and is now deployed island-wide with CDM Smith’s

presence in the West, South and East regions.

As part of the technology upgrade program, PRASA focused on multiple infrastructure and operational

areas, including instrumentation and geographic information systems (GIS) to improve management,

maintenance and planning. With updated and new installations for Supervisory Control and Data

Acquisition (SCADA) and telemetry, operators were able to maintain systems and manage processes. New

infrastructure surveys and updated GIS systems aided both operations and planning by improving the

accuracy and coverage of location and attribute information and associating engineering documents with

collection and distribution system networks in a navigable, queryable GIS interface.

The GIS and instrumentation projects required significant investments and incur on-going costs for what

comprise vertical purposes: GIS for planning and engineering, and instrumentation for operations. While a

single purpose system provides the expected benefits to its primary users, it may become an information

“silo.” While completing the GIS upgrade, CDM Smith and PRASA identified the potential to inject

telemetry data into the GIS in an information integration solution which would extend the business value

of both systems. PRASA’s West Region agreed to support a demonstration project.

The topic of this paper is how PRASA leveraged these two significant, yet disparate projects, to benefit

multiple stakeholders and realize new organizational value by developing a business solution which

integrates GIS and telemetry.

Island-wide Telemetry Implementation – A Local Affair

PRASA has been implementing SCADA systems to help centralize monitoring and control of their water

treatment plants. This project is also referred to as the “automation” project and involves control of

filtration plants by monitoring turbidity, chemical addition, etc. The planning and design for this system is

a lengthy process, focused on major facilities, especially the plants. The automation project has not yet

resulted in tools that support local operational staff who are responsible for regional water distribution.

For addressing local needs, PRASA’s regional units have acquired their own systems to monitor and report

on operational parameters such as water tank levels, in response to customer needs. For example, when a

water tank level drops too low, customers experience low pressures and the local utility receives

complaints.

In the West region, the telemetry vendor installed monitoring equipment that uses a GSM radio network

to communicate with the remote facilities: tanks, pump stations and water filtration plants. Telemetry

allows measurements at a distance to be sent via radio waves to a receiving location. As shown in Figure 3

- Regional Water Distribution Telemetry. The remote monitoring systems use solar power to run the

equipment as well as charge the backup battery. The receiving unit gets the sensor data as delimited text

files from the remote sites and delivers it to a Windows-based, HMI software system via the historian

database. The HMI software displays tabular and graphical presentation of the data. The master polling

and data mapping is done through the information acquired via the GSM network. Any data that needs to

be passed between sites is also handled via the same network.




Figure 3 - Regional Water Distribution Telemetry

By design, telemetry is limited to specific monitoring functions; the data are not viewable with other

facility information and are not queryable or customizable. Readings are displayed on a simple Google

map for spatial reference. Additionally, because each region proceeded independently, multiple telemetry

systems are in place.

The information integration prototype was conceived as a method to display data from multiple telemetry

systems on an island-wide GIS to allow multiple staff to better visualize and respond to alarm conditions

and allow a broader understanding of water distribution factors like insufficient supply, maintenance

schedules, trends and customer demand. The next section illustrates the telemetry human machine

interface (HMI) and the extensive benefits of integration with GIS.

Telemetry Provides Water Tank Level Readings

The telemetry system from provides telemetry data for water tank levels and pump station pressures from

facilities in the PRASA West (Oeste) region; integrating this data into the island-wide GIS presented an

opportunity to deliver added value from both systems.

The HMI user interface (UI) is a web-based, client-server application that built as a custom Microsoft

ActiveX component which runs within Microsoft Internet Explorer. Figure 4 shows a portion of the HMI.

Figure 4 - Tank (asset) list filtered for one item in DataOnline UI




This UI is purpose-built for monitoring readings from tank water level instruments within the telemetry

network; it is suitable for technical operators, but is not user-friendly and is not meant to be accessed by

non-technical users. Readings can be viewed against a simple Google map, as shown in Figure 5 and Figure

6, but otherwise lacks operational context. While it provides essential monitoring information, the data is

trapped in its information silo.

Figure 5 - Telemetry water tank level information overlaid on a Google map

Figure 6 – Regional view of water distribution system monitoring information overlaid on a Google map




GIS

Under a separate technology modernization project, PRASA worked with CDM Smith and other consulting

firms to update their GIS, which focuses on the utility’s asset management needs: locating, identifying,

digitizing and characterizing water and waste water assets. The upgraded GIS supports planning activities,

such as water system modeling, capacity planning, and cost-effective maintenance as well as operational

responses to acute events. The island-wide effort required new GIS data collection, data management

improvements, hardware/software upgrades and training on GIS best practices. Work continues as new

facilities are added, assets are identified and field inspections verify attributes. While the GIS

improvements are out of scope for this discussion, they provided the crucial backbone for telemetry

integration.

Piloting Information Integration

To demonstrate the potential benefits, CDM Smith undertook a pilot project supported by PRASA’s West

region where DataOnline had established water tank and pump station monitors. CDM Smith’s

Information Management group, consisting of both GIS and custom application development staff, led by

its water resources expert, designed a multi-phase approach which converted their original GIS Viewer

website into an integration hub. CDM Smith used complementary mapping and application development

technologies to construct two integrated solutions: a web browser-based site and an Apple iPad “app;”

these are known as AquaWeb and AquaPad, respectively. The underlying technologies are ESRI ArcGIS

Server, Microsoft ASP.NET, Adobe Flex and Apple iOS. The iPad app further extended the value of the

integration project by leveraging ArcGIS map services that were already built for the web-based

application, further providing mobility, ease of use and, literally, information at your fingertips.

AquaWeb users log into the web application and select from several information options: basic GIS,

advanced GIS, Applications, and related documents. The Telemetry Application is selected from the

Applications section. The initial Telemetry Viewer screen displays tank levels on a high-quality, National

Geographic basemap overlaid with PRASA regional boundaries and tank locations (Figure 7). AquaPad

draws on the same, centralized ESRI map services and presents a similar initial view. Note, the majority of

telemetry information is located in the West region where the pilot integration project was established.

Figure 7 - Initial view (map extent) of Telemetry Viewer map in AquaWeb (primarily West region data).




Tank colors indicate alarm conditions – red (critical low water level), yellow (alert high level), and green

(normal range). Clicking on (or tapping) the tank symbol provides current readings in a graph, as shown

below (Figure 8).

Figure 8 – Views of Celedonia water level graphs in AquaWeb (on left) and AquaPad (on right)

By activating additional GIS layers, the viewer displays added context – showing tanks with connected

assets such as pump stations, pipes and valves as illustrated in the AquaWeb screenshots Figure 9 and

Figure 10. Other available layers include hydrants, schools, neighborhoods, and natural features.

Figure 9 - Selected GIS feature layers provide additional water distribution context




Figure 10 - Water distribution assets on orthophoto background showing Buena Vista tank details in popup

Typical Operational Uses

By looking at the telemetry data combined with the comprehensive GIS map, operators and staff can more

easily determine where alarm conditions are present and look for impacted facilities, affected population,

and topographical characteristics while responding to a problem.

Display capabilities include:

• Graphing recent tank trends (Figure 14). The operator can use this to see how quickly and

frequently the tank level is falling. This might give an idea of how severe the issue is. When the

trend graph display is expanded over multiple days or weeks, the operator can detect whether this

is typical or aberrant tank behavior.

• Linking operational “sectorization” documents. Sectorization documents provide planimetric and

schematic views of subsystems. By integrating sectorization documents into the GIS, operators

can easily access operational information and quickly diagnose water supply issues. Figure 11 and

Figure 12 show excerpts of the sectorization document for “Sistema de Distribucion: Pozos Cabo

Rojo, Region Oeste - Fuentes de Abasto: Planta de Filtración Miradero y Pozos Cabo Rojo”




(Distribution System: Cabo Rojo Wells, East Region – Sources of Supply: Miradero Filtration Plant

and Cabo Rojo Wells).

Figure 11 - Linked sectorization document, page 1

The combination of telemetry data, dynamic map features and distribution details supports effective

troubleshooting and dispatching to restore normal operations as quickly as possible.

For example, if a tank level drop is atypical (reference Figure 14) and the tank is supplied by a pump

station, the pump station may be out of service. If the pump station is operational, there may instead be a

water main break, or a valve to a lower pressure zone may have been opened, thus draining the tank.




Figure 12 - Linked sectorization document, page 2

Analyzing Trends with the Telemetry Details Viewer

Besides supporting operational decision making by providing context for alarm response, the integrated

GIS presents an overall perspective that supports quality control and managerial oversight, as illustrated

by the Telemetry Details Viewer, which is modeled on an interactive stock ticker widget (Figure 14). The

user accesses the Details Viewer from the Detalles (Details) link in the popup window (Figure 13), or the

Detalles link in the attributes pane.




Figure 13 - Accessing the Telemetry Details Viewer from the Detalles (Details) link

Figure 14 - Telemetry Details Viewer provides interactive timeline and analytics for selected facility (Espino tank)

This interface does not replace the HMI-based interface but extends the single-purpose limits of the

operational UI by combining the historical telemetry data with geographic context and allowing both




technical and managerial users to easily interact with the information in a more useful way. It also

reinforces the context of the telemetry information within the fuller infrastructure.

Figure 15 - HMI tank level time series (Buena Vista tank)

Key problems and Solutions

Like any “system of systems” the usefulness of the integrated view is wholly dependent on the quality of

the underlying component systems, following the age-old computer adage: Garbage In, Garbage Out

(GIGO).

The core challenges of information re-use and integration are data quality and a correct understanding of

the data to be integrated. By data quality we mean that both the original data sets and the references that

link them are valid. If a reference link does not reliably identify a unique data item there can be an

appearance of integration without any benefit. In fact, unsuccessful data integration can create subtle

information errors which are hard to detect and actually undermine the integration value.

Therefore, in the process of planning data integration, the system architect must combine technical

expertise with knowledge from domain experts to confirm and document data references, uniqueness

assumptions, reliability and testing protocols, thus avoiding misinformation. This task is further

complicated when systems are “owned” by different groups supported by different vendors. In this case,

PRASA facilitated the interaction between the telemetry vendor and GIS teams to ensure the exchange of

information.

Despite careful planning, after the initial proof of concept was demonstrated, the integration prototype

required a significant data architecture change due to three factors:

1. Issues with obsolete links to GIS identifiers (“FacilityID”) which contained non-unique references;




2. Dependence on telemetry identifiers (“DeviceID” / “Channel ID”) which changed during telemetry

device maintenance; and

3. Occasionally missing telemetry readings which crashed the iPad application and hid tanks with

missing data.

Each of these issues required re-examination of the integration assumptions and subsequent design

revisions and data corrections. The next section describes how the integration works at a high-level,

followed by a discussion of how the blocking issues were resolved.

How It Works

The telemetry vendor provided CDM Smith with access to their telemetry readings via a File Transfer

Protocol (FTP) server accessible via the Internet. They also provided a technical memo that identifies the

data elements in each record, enabling CDM Smith to create a data mapping table (“tblLocation”) that

links identifier values from the telemetry reading device and related features in the GIS. CDM Smith built a

core component that polls the FTP directory hourly and retrieves the latest readings. This starts the

dynamic data integration processing cycle.

To process a batch of telemetry readings, the PRASA Telemetry Integration Service (PTIS) downloads files

from the FTP server and reads through the records. A record consists of these data elements:

sequence number, tank identification number, device id, date/time, offset from GMT, reading,

unit of reading, alarm type, alarm text, and record status

Figure 16 illustrates the processing loop in which PTIS parses a record, matching the unique feature

identifier (tank identification number) with the mapped feature identifier in table tblLocation and selects

its linked GIS FacilityID and pre-configured alarm levels. It calculates an alarm condition (normal, high, low)

for color-coding the tank symbols and stores the feature identifiers, alarm condition, reading (e.g., water

level), and timestamp in two data integration tables: one for current readings (one row per device) and

one for historical readings (multiple, time-series rows per device). Storing the current readings in one

table (tblTeleCurrent) enhances system performance when presenting the current status of many facilities

in the initial overview. Storing the cumulative readings in a separate table (tblTeleHistory) behaves like a

SCADA historian table allowing drill-down, longitudinal visualization.

On the application side, a service in the GIS application associates GIS features (water tanks, pump

stations, etc.) with readings from tblTeleCurrent to symbolize the feature map (see Figure 7) and populate

the popup graph (see Figure 8 and Figure 9). A similar application service links the selected GIS feature to

the long-term readings in tblTeleHistory to generate graphs in the Telemetry Details Viewer (see Figure

13). Like the Telemetry Integration Database, the services are parameterized to handle multiple telemetry

types and features. The PTIS supports both AquaWeb and AquaPad simultaneously.




Figure 16 - Telemetry data integration workflow for processing imported telemetry data

There is no mystery in this process – once it is working. But consider these factors:

• Each telemetry vendor has potentially different data schema

• Each type of telemetry source (water tank level, pump pressure, temperature, turbidity,

chlorination, etc.) has different metrics

• Each telemetry source may have different units of measure, time scales and operating ranges

• Each data source may have a different owner with whom to coordinate.

Integration can be a complex task; the linking architecture must accommodate a variety of telemetry

devices and configurations; the visualization components must be flexible and effective at presenting

multiple data types. The prototype integration database was extended to meet these multiple data

requirements. The revised design is illustrated in the entity relationship diagram (ERD) below (Figure 17).




Figure 17 – Telemetry integration database links GIS assets with multiple telemetry types and vendors

Development Approach

Beyond the initial prototype that demonstrated the proof of concept, the application development team

adopted a rapid delivery methodology known as Agile Development which succeeded in addressing key

issues and introducing newly requested features, such as the Telemetry Details Viewer (Figure 14) and

security subsystem (not shown). In the process, the team evolved the application from a prototype to a

highly performant, stable, secure and flexible application. Agile Development includes frequent product

deliveries to solicit feedback and adjust to emerging priorities; it fosters cross-functional communication

and engagement with stakeholders and addresses issues sooner rather than later. With the cooperation

and support of users who are invested in the process, problems and feedback that arise can actually

improve the product during its construction, while users benefit from continual system improvements.

Extending the Information Systems Investment

As stated above, both the GIS and the Telemetry systems focus individually on their own technical,

operational and engineering goals that must be met before integration can be considered. However, after




these goals are reached, the integration step provides a way to extend the investments in both systems

beyond their original scope.

One way in which benefits are extended is meeting the information needs of a broader constituency

without burdening the primary users, the core systems or specialized licenses. Information re-use, like

recycling, does not manufacture new information but extends the life of existing resources by

transforming them from single purpose to multi-use, from information silos to shared knowledge.

Next Steps for Long-term Integration Benefits

Besides the pilot integration project in the West region, the wider integration benefit is to consolidate the

island-wide telemetry systems into the shared view. This will allow PRASA to leverage the island-wide GIS

while sustaining their independent regional telemetry systems. PRASA’s integrated telemetry and GIS

supports this future benefit today. Other anticipated functionality includes richer document management

and a dashboard presentation of key performance indicators. Also in development is a GIS-based water

capacity planning subsystem that tracks new construction water supply needs.

Summary

While technically challenging to develop, the integrated system laid the foundation for extending the

value of PRASA’s technology investment by combining existing data sources into a single, flexible

interface. The foundation is both a technical and managerial achievement that required integration of

disparate technologies and communication across multiple business units. Technically, the application

development team solved a host of issues around GIS and telemetry identifiers, real time data,

synchronization, map projections, UI design, performance and data maintenance. The technical success

required managerial commitment, active communication and effective partnering with multiple

stakeholders. Using the Agile Development methodology aided communication, issue resolution and more

rapid delivery of benefits to PRASA staff. The resulting system is more than a sum of its parts; it provides

insight and access to better understand what is going on in the water distribution network. It is designed

for the future to expand with additional asset types by combining location-based infrastructure,

engineering information and performance parameters in a dynamic visualization context.

List of Acronyms

ERD .................. Entity Relationship Diagram

FTP ................... File Transfer Protocol

GIGO ................ Garbage In, Garbage Out

GIS .................... Geographical Information System

GMT ................. Greenwich Mean Time

GSM ................ Global System for Mobile Communications, originally Groupe Spécial Mobile

HMI .................. Human Machine Interface

PRASA .............. Puerto Rico Aqueduct and Sewer Authority

PTIS .................. PRASA Telemetry Integration Service

RTU .................. Remote Terminal Unit

SCADA .............. Supervisory Control and Data Acquisition

UI ..................... User Interface




Authors

Michael Waddell: Mr. Waddell leads application development projects specializing in data integration to

realize organization-wide benefits. He is Applications Development Practice Leader at CDM Smith and has

been in the industry over 22 years. Contact: [email protected]

Isabel Szendrey: Ms. Szendrey is the Auxiliary Director of Planning at PRASA. She is responsible for

developing fully integrated solutions that support the Authority’s daily operations. Ms. Szendrey has been

working in the water/wastewater sector for over 10 years.

Resources

http://cdmsmith.com/en-US/Solutions/Water/Landmark-Capital-Improvements-in-Puerto-Rico.aspx

http://en.wikipedia.org/wiki/Telemetry

http://eponline.com/articles/2010/05/07/puerto-rico-authority-to-spend-195-m-on-126-drinking-water-

plants.aspx

http://www.dataonline.com/

http://www.gdb-pur.com/investors_resources/presentations/PRASA-MAGNYPResentation.pdf

http://www.wpi.edu/Pubs/E-project/Available/E-project-042908-

154559/unrestricted/ExpanRioPrietoWaterDistSys.pdf

Developing an Integrated Business Solution with Telemetry and GIS

Documents

Transcript of Developing an Integrated Business Solution with Telemetry and GIS