Overview of TSDS and TSDS High Level End User … Tech...Overview of TSDS and TSDS High Level End...

Overview of TSDS and

TSDS High Level End

User Process Map

Simple Solution. Brighter Futures.

Copyright © 2013 Texas Education Agency. All rights reserved. TEA confidential and proprietary.

A data collection and reporting system that will:

Modernize the PEIMS data collection process to reduce technology risk and system downtime, allowing for more system availability and ease of use, and;

Put real-time student performance data in the hands of educators to improve student achievement, and;

Become the one common data collection platform for TEA to reduce the data collection burden on districts and charter schools.

Simple Solution. Brighter Futures.

Introducing the Texas Student Data System

Click to edit Master title style


10

How will TSDS help LEAs?

Achievement

studentGPS™ Dashboards will help educators monitor and assess student performance

Vital information is in one easy-to-use dashboard, saving teachers time and effort

Cost

TSDS offers a streamlined process to reduce LEA time and effort, and an optional SSIS to reduce data management costs

LEA Compliance

The new TSDS Client-Side Validation Tool and longer loading windows in TSDS PEIMS will make it easier and faster to load clean data

TEAL for TSDS and the TSDS Portal Roles

2

TEAL2

TEAL – Texas Education Agency Login

A single sign-on that provides users access to many TEA applications

Secure authentication ensures only authorized users can access the applications

Users must first create a TEAL identity and then request account access to the specific applications needed

TEAL Account Access and Approval Process3

Once a TEAL ID is created, the user must request access to the specific TEA applications to which they need access

If a request for account access is not made within 60 days, the TEAL account is inactivated (the user will get email notification each day for 7 days before the inactivation occurs)

Requests for account access must first be approved by the Organization Approver (Superintendent, ESC Director, or designated alternate; this also may be a Limited Approver) if no action is taken within 5 days the request is rerouted to the alternate approver (if the organization has one). If no action by the alternate within 5 days, the user gets an email notification & must resubmit the request.

Once requests are approved at the organization level, they are routed to the TEA Service Approver for approval

TEAL TSDS Portal TSDS Portal Roles

4

TSDS Portal Roles5

Role DescriptionstudentGPS™ Dashboards

An LEA authorized TEAL role for access to the Dashboards. For each user, it is mapped to one of the 7 local Dashboards roles. Access to TSDS with this role must be approved by the LEA Superintendent or designee. May be requested through traditional TEAL process or the LEA may opt for automated provisioning.

studentGPS™ Dashboards Configurator

A temporary LEA authorized role that allows the user to configure LEA goals and provision accounts for other LEA users. Must be selected with the studentGPS™ Dashboards User role. Corresponds to the local System Administrator role in the Dashboards – usually for the LEA Dashboards or Data Steward. Access to TSDS with this role must be approved by the LEA Superintendent or designee and Approver enters the expiration date.

ODS Campus Data Loader

An LEA authorized role for the scheduling and loading of campus data into the eScholar data warehouse. Targeted users: Campus Data Steward or Campus PEIMS Coordinator.

ODS Data Loader An ESC/LEA authorized role for the scheduling and loading of LEA data into the eScholar data warehouse. Targeted users LEA: LEA PEIMS Coordinator and/or Data Steward; Targeted users ESC: Technical and/or PEIMS Champion.

TSDS Portal Roles6

Role DescriptionUniq‐ID Campus An LEA authorized role providing search and update access to Unique ID at the

campus level.Uniq‐ID LEA An LEA authorized role providing search and update access to Unique ID at the LEA

level.Uniq‐ID Search An authorized role providing search‐only access to Unique ID at the statewide level.

TIMS LEA Support Role designed for LEA staff providing TSDS support for their own LEA. Access to TSDS with this role must be approved by the LEA Superintendent or designee.

TIMS ESC Support Role designed for ESC staff providing TSDS support for one or more LEAs. Access to TSDS with this role must be approved by the LEA Superintendent or designee.

PEIMS Campus Submitter

An LEA authorized role providing the ability to load, validate and view PEIMS data and reports for campus level data to the PEIMS Data Mart. Targeted users: Campus Data Administrator.

PEIMS Data Submitter

An ESC or LEA authorized role that will allow the user the ability to load, validate and view PEIMS data and reports. Targeted user LEA: PEIMS Steward; Targeted users ESC: Technical and/or PEIMS Champion.

PEIMS Data Schedule/Promoter (LEA/ESC)

An ESC or LEA authorized role for scheduling and promoting of ESC, LEA and/or campus level data to the PEIMS Data Mart. Targeted users LEA: PEIMS Steward; Targeted users ESC: Technical and/or PEIMS Champion.

TSDS Portal Roles7

Role DescriptionPEIMS Data Completer

An ESC or LEA authorized role of a user that can load, view, report, set as complete, and lock the data. Targeted user LEA: PEIMS Steward; Targeted user ESC: PEIMS Champion

PEIMS Data Approver (LEA/ESC)

An ESC or LEA authorized role that will allow the user the ability to view reports and approve or reject a PEIMS data collection. Targeted users: Superintendent or ESC Executive Director

PEIMS Data Accepter (ESC)

An ESC authorized role that will allow the user the ability to view, run reports, and approve or reject a PEIMS data collection. Targeted users: ESC Data Accepter

FERPA Policy Admin

An LEA authorized role providing the ability to configure FERPA policy through the TSDS solution. Targeted users: LEA/ESC FERPA administrator and Data Steward.

Overview of Data Loading Process

3



Overview of Data Loading Process 14

• Interchange extracts are created from source systems in XML format

• Must align with TEDS

• Downloaded from TSDS Portal

• Used by LEAs & ESCs to validate XML interchanges prior to loading

• Errors in interchanges must be corrected in source systems & revalidated

LEA Source Systems

Client Side

Validation Tool

• A secure FTP file transfer client utility

• Downloaded from TSDS Portal & installed at LEA/ESC

• Facilitates On Demand & Scheduled file transfers

• Sends XML files to eDM

• Uses Service Account credentials for login & password

• User selects files to be transferred

• File name is validated • User can view

transfer status

Data Transfer

Utility (DTU)

• Automatically picks up files that have been transferred by the DTU

• Runs files though data validations & updates the ODS

• Users can also manually upload files

• Error files are generated at initial file validation & again during ETL process

• Errors in interchanges must be corrected in source systems & revalidated

• User can verify that the ODS has been updated by viewing status in the eDM

eDM (Manage Data

Loads)

• User schedules and can monitor the promotion of data from the ODS to the PDM

• User specifies collection and submission (e.g. Fall, First)

• User selects Category (e.g. Staff) and Sub Category(ies) to be loaded (e.g. Employment Payroll Summary)

• User can monitor the status

PEIMS

Application

Dashboards Data

Mart

• Data is loaded from ODS to a staging DDM

• If load successful, then it is moved to the production DDM for studentGPS™ display

• Metric calculations occur during the load of data from the ODS to the DDM

• Data is automatically refreshed nightly, no user interaction needed

• System will prevent incomplete data from being loaded

Data Interfaces

Other TEA Systemsthat do not use PID Web Services

ODSFile/WS Load

ValidationLoad ReportsAudit ReportsError Reports

SchedulingWorkload

Web Service

PDMPEIMS Data

Mart

DDMDashboard Data

MartDashboard Load

Operational Reports

160160 Data Mart

TSDS PEIMS App.

Promote Loaded DataValidate Submission

Search DataFinalize Submission

studentGPS™ Dashboards

Educators View Performance Data

PID Web Service

Other TEA Systemsthat need PID checks

State-sponsored

SIS

SIS/HR/Finance/

Other

LEA

Staging A

reaTE

DS

Interchange Files

TEAL S

ecurity System

TEAL Security System

Other agency/state/federal

systems

160 App.Embedded in PortalPromote Loaded DataValidate SubmissionFinalize Submission

Main_PRD

UniqueID& PID

TSDS PortalHosted by TEA

Education Data Warehouse

Campus/LEA/ESC Systems

DTU

ComponentsDiagram

LEA/ESC Systems

TSDS: Accessible only by campus/LEA/ESC

TSDS: Authorized TEA access

Non-TSDS TEA systems

Client-side validations

eScholar validations

UniqueID validations

Server-side validations

Accessible only by campus/LEA/ESC

Ver 3.1 | 26 Nov 2013

TSDS High Level End User Process Map

4

Manage Data Loads(eDM)

XML Interchange Files from Source Data Systems

*1

If necessary, Download latest

version of the TSDS Client-Side

Validation Tool via the TSDS Portal *3

Validate XML Interchange Files (TSDS Client-Side

Validation Tool) *2

Run the TSDS Client-Side Validation Tool

and Review Error Reports

Data Errors?

On Demand (manual) or

Schedule DTU Process

ETL Data Errors?

Promote Loaded Data(TSDS PEIMS Application)

*4

Data reflected in studentGPS™ Dashboards

PDMODS

Review Error Reports

eDM picks up the XML Interchange

Files

ETL

No

Correct the Data in the Source Data Systems and Re-

extract XML Interchange Files

Pass XML Interchange File

Validation in eDM?

Yes No

Yes

No

Schedule Data Load by

Category and Sub-Category

TSDS High Level End User Process Map

TSDS PEIMS ETL

Promote Loaded Data

Data Errors?

Yes

Yes

Prepare/Finalize TSDS PEIMS Submission

(TSDS PEIMS Application) *5

Review Generated

TSDS PEIMS Validation

Reports

Run and Review TSDS

PEIMS Validation

Reports

Fatal Data Errors?

Fatal Data Errors?

NoYesYes

LEA Marks Data Complete (LEA Data) or

ESC Marks Data Complete

(ESC Data)

Superintendent Approves LEA Data or ESC

Director Approves ESC

Data

TSDS PEIMS Submission has been Finalized

XML XML

XML

DDM

XML

Run and Review QA

Reports

No

Data Correction Required?

ESC Accepts LEA or ESC Data

NoYes

XML

Roles and Responsibilities Key

ESC TSDS PEIMS Champion/LEA TSDS PEIMS Coordinator/

Steward

ESC Technical Champion/LEA Data

Steward

Joint Responsibility –TSDS PEIMS & Technical/Data

Document Number: TSDS-Tech-R001-D001

studentGPS™ ETL

No

Review TSDS studentGPS™

Reports

BatchManager

FileManager

Prepare data for Core Collection Data Mart CCDM

Run and Review TSDS

Reports

Data Correction Required?

Yes

XML

No

TSDS Submission has been Finalized

TSDS Data Approver approves

submission

Responsibility – TSDS Data Approver



TSDS High Level End User Process Map Key: Roles and Responsibilities Key

26

Roles and Responsibilities are color-coded.



XML Interchange Files from Source Data 31

XML Interchange files are extracted from the source data systems

The extract process will vary by LEA and by Source Data Vendor

This is outside the scope of responsibilities for either role defined in this map

The TSDS Client-Side Validation Tool is a critical component



33

TSDS Client-Side Validation Tool, DTU and eDM

1- TSDS Client-Side Validation Tool

2- TSDS Data Transfer Utility: Transfer files to eDM

3- TSDS eData Manager: Load data to the ODS



35

eDM Data Loading

Errors: Correct Data in Source System No Errors:

Advance

Errors: Correct Data in Source System

No Errors: Advance



Data Promotion from ODS 37

After the data is loaded in the ODS:

Data promotion to the PEIMS Data Mart (PDM) OR

Data promotion to the Dashboard Data Mart (DDM)



Promoting Data to the PDM 38

Schedule Data Promotion

Category

Subcategory

ETL Plan is executed

PDM is updated



40

TSDS PEIMS Validation

Review Validation Reports generated by TSDS PEIMS

If there are fatal errors: correct in source data system

No fatal errors: data advances



42

If the data was resubmitted, the user can run the

Validation Reports again

If the data is clean, the user runs & reviews the QA reports. Data corrections may be required.

TSDS PEIMS Validation (continued)



TSDS PEIMS Prepare/Finalize Submission 44

The ESC accepts either the LEA or

ESC data

The Superintendent approves the

LEA data or the ESC Director Approves the

ESC data



46

DDM Data Loading

Automatic data promotion

As soon as the data is updated, it is reflected in

the studentGPS™ Dashboards

Validation Errors: Correct data in source system

TSDS Roles and Responsibilities

5



TSDS Roles and Responsibilities 49

ESC Technical Champions/LEA Data Stewards

ESC TSDS PEIMS Champions/LEA TSDS PEIMS Stewards/Coordinators

Joint Responsibility

Manage File Transfer in DTU Run TSDS PEIMS Validation Reports

Run the TSDS Client-Side Validation Tool

Manage Data Load in eDM Run TSDS PEIMS QA Reports Download the latest version of the TSDS Client-Side Validation Tool if necessary

Manage Data Promotion to PDM

Mark TSDS PEIMS Data as Ready

Manage Data Promotion to DDM

Manage ESC Acceptance and Superintendent Approval of Data

Run studentGPS™ Dashboards Validation Reports

Questions?

6

Overview of TSDS and TSDS High Level End User … Tech...Overview of TSDS and TSDS High Level End...

Documents

Transcript of Overview of TSDS and TSDS High Level End User … Tech...Overview of TSDS and TSDS High Level End...