1

Federal Transparency.gov As Data For the Digital Government Strategy

Dr. Brand NiemannDirector and Senior Data Scientist

Semantic Communityhttp://semanticommunity.info/

AOL Government Bloggerhttp://gov.aol.com/bloggers/brand-niemann/

April 9, 2013http://semanticommunity.info/FederalTransparency.gov

2

Background• Third Annual Government Big Data Forum:

– Big Data in the Age of Analytics: From Infrastructure to Sense Making• Kirit Amin (Department of Commerce), Shawn Kingsberry (Recovery Accountability and

Transparency Board), and Donna Roy (DHS NIEM/Information Sharing Environment)

• People, Processes, and Technology– The Kitchen Model of Big Data: Chefs, Ingredients, Utensils, and Recipes

• Chef's are Data Scientist, Ingredients are Data Specified or Needed, Utensils and Recipes are the Tools and Experience used by the Data Scientists, respectively.

• New Digital Government Strategy Paradigm:– All content as data = big data

• Done this recently for Census Data Visualization Gallery and NIEM/ISE.

• Now for the Four FederalTransparency.gov Web Sites:– FederalTransparency.gov– EducationJobsFund.gov– FederalReporting.gov– Recovery.gov

3

The 7 Habits of Highly Effective People Habit 2: Begin With The End In Mind

• Habit 2 is based on imagination--the ability to envision in your mind what you cannot at present see with your eyes. It is based on the principle that all things are created twice. There is a mental (first) creation, and a physical (second) creation. The physical creation follows the mental, just as a building follows a blueprint. If you don't make a conscious effort to visualize who you are and what you want in life, then you empower other people and circumstances to shape you and your life by default. It's about connecting again with your own uniqueness and then defining the personal, moral, and ethical guidelines within which you can most happily express and fulfill yourself. Begin with the End in Mind means to begin each day, task, or project with a clear vision of your desired direction and destination, and then continue by flexing your proactive muscles to make things happen.

• My Note: My End In Mind Is Federal Transparency.gov As Data For the Digital Government Strategy.

Source: https://www.stephencovey.com/7habits/7habits-habit2.php

4

FederalTransparency.gov

http://www.federaltransparency.gov/Pages/default.aspx

5

EducationJobsFund.gov

http://www.educationjobsfund.gov/Pages/home.aspx

6

FederalReporting.gov

https://www.federalreporting.gov/federalreporting/home.do

7

FederalReporting.gov Templates

http://semanticommunity.info/@api/deki/files/23789/FederalReportingTemplate_-_Contracts.xlshttp://semanticommunity.info/@api/deki/files/23790/FederalReportingTemplate_-_Grants_and_Loans.xls

8

Recovery.gov

http://www.recovery.gov/Pages/default.aspx

9

Tufte’s Lights-On-Map

http://www.recovery.gov/Transparency/MapGallery/Pages/TufteLightsOn.aspx

My Note: Need to produce this in a toolthat links to the underlying data.

10

FederalTransparency.gov Knowledge Base

http://semanticommunity.info/FederalTransparency.gov

My Note: Well-Defined URLTo Each Piece and Mashupof Related Pieces.

11

FederalTransparency.govSpreadsheet

http://semanticommunity.info/@api/deki/files/23778/FederalTransparency.xlsx

Seven Spreadsheets and TheirData Elements and DefinitionsThat Make the Data Ecosystem.See Slide 14.

12

FederalTransparency.gov Spotfire: Recovery.gov

Filters

Details-on-Demand

History of Spending Spending Versus Jobs

Tufte-like Lights-On-Map

Interactivity of Data in Visualizations

Big Data Set

Story and Metadata

599,743 Rows and 98 Columns

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FederalTransparency-Spotfire

13

FederalTransparency.gov Spotfire: Education Jobs Fund

Top Ten States

Award Amount Versus Number of Jobs

Rank Order

Instructions

Filters

Details-On-Demand2012 4th Quarter

Data Set

TotalsData Set

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FederalTransparency-Spotfire

14

FederalTransparency.gov Spotfire: Data Ecosystem

Data Completeness Data Completeness

Data Completeness Data Dictionary

Data Inventory

Data ElementRepository

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FederalTransparency-Spotfire

15

FederalTransparency.gov Spotfire: State Boundaries

My Note: Additional BoundariesFor Counties, CongressionalDistricts, MSAs, etc. Can Be Added.

Data Set

Filters

Metadata andInstructions Visualization

Filter Result

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FederalTransparency-Spotfire

16

Conclusions and Recommendations

• The Third Annual Government Big Data Forum – “Big Data in the Age of Analytics: From Infrastructure to Sense Making” Provided the End in Mind for this Work:– Federal Transparency.gov As Data For the Digital Government Strategy

• The People, Processes, and Technology of Big Data Suggests The Kitchen Model of Big Data: Chefs, Ingredients, Utensils, and Recipes:– The Chef's are Data Scientist, Ingredients are Data Specified or Needed, Utensils and Recipes are the

Tools and Experience used by the Data Scientists, respectively.• The New Digital Government Strategy Paradigm of All Content as Data = Big Data:

– Did this recently for Census Data Visualization Gallery and NIEM/ISE and Now for the Four FederalTransparency.gov Web Sites:• FederalTransparency.gov• EducationJobsFund.gov• FederalReporting.gov• Recovery.gov

• Created a MindTouch Knowledge Base, an Excel Spreadsheet, and a Spotfire Dashboard and Data Ecosystem to Audit, Assess, and Improve on Work of Government Accountability and Transparency Board and to Audition for the Recovery Independent Advisory Panel.

• Do Comparison Maps for a Map Gallery Like the Census Data Visualization Gallery.– http://semanticommunity.info/Census_Data_Visualization

17

Spotfire 5.5

http://spotfire.tibco.com/en/spotfire-5-5.aspx

18

TIBCO Acquires Maporama

http://maps.tibco.com