Opportunities and Challenges in

Harnessing Social Big Data in

Government

Adegboyega Ojo, Insight Centre for Data Analytics

National University of Ireland, Galway

adegboyega.ojo@insight-centre.org

Big Data in Public Administration Panel, dg.o 2014, Auguscalientes, Mexico

19 June 2014

Agenda

The talk will touch on:

o Social Big Data (SBD) in Government

o Social Big Data Infrastructure by Insight @ NUI Galway

o Challenges to SBD adoption in Government

o Conclusion

18 June 2014

Social Big Data (1)

Big data is any data:

• too big to process with current tools – Volume

• arriving too fast for optimal storage and indexing - Velocity

• too heterogeneous to fit into a rigid schema - Variety

o Volume usually in magnitude of PetaByte (1015) or ExaByte (1018)

o Velocity implies the need for tools to monitor, analyse and

manage real-time streaming data

o Variety connotes the mixture of structured and unstructured

information of different formats including text, video, audio,

images, databases, etc.

Big data is also exhaustive as it often captures data representing a

whole population rather than samples and fine grained in resolution.

The Challenge is how to generate valuable insights from these data.

18 June 2014

Social Big Data (2)

Social Big Data is Big Data generated from user contributed data

such as posts, comments, and user interactions across social media

websites such as Twitter, Facebook or YouTube.

They are generated as real-time data streams from diverse sources

on the social web.

18 June 2014

www.xposedmediaworld.com www.athgo.org

Social Big Data - Monitoring & Analytics

There is more than 400* social media monitoring tools in the

market. Main features are as follow:

• Summarizing, reporting and data visualization

• Multilingual search and text analytics for entity extraction,

name matching, name translation, government intelligence, e-

discovery and financial compliance

• Sentiment detection and analysis

• Event detection and analysis

• Citizen and customer engagement and management

Examples - HootSuite, BlitzMetrics, Brandwatch, CARMA, IBM’s

jStart, Social Mention etc.

18 June 2014

* http://www.pamorama.net/2013/05/12/50-top-tools-for-social-media-monitoring-

social-media-analytics-social-media-management-2013/

Social Big Data - Monitoring & Analytics:

Example - Social Mention

18 June 2014

Social Big Data in Government – Example 1

The federal government is seeking a firm that "continuously

monitors social media content on a daily basis in near real time and

(can) provide web-based, online media metrics and reporting

capabilities”.

Part of the job will be to gauge the sentiment and tone of posts and

to determine their reach.

Project will run until January 2019.

http://www.thestar.com/news/canada/2013/11/29/social_media_to

_be_monitored_by_federal_government.html

18 June 2014

Social Big Data in Government – Example 2

The Food and Drug Administration has ordered the social media

monitoring tool to help FDA’s Office of External Affairs (OEA) better

understand its audience and gauge the effectiveness of its activities

and outreach campaigns.

The tool will generate metrics to help better monitor the success

and failure of tweets and status updates in various social media

platforms such as Facebook and Twitter.

The metric should be, but is not limited to: engagement, sentiment,

likes, follows, and mentions.

18 June 2014

http://www.futuregov.asia/articles/2013/sep/25/us-agency-implement-social-media-monitoring-tool/

Social Big Data in Government – Example 3

Livehoods is a research project from the

School of Computer Science at Carnegie

Mellon University.

Presents a new approach for studying the

dynamics, structure, and character of a

city on a large scale using social media and

machine learning.

Using data such as tweets and check-ins, it

is able to discover the hidden structures of

the city with machine learning.

18 June 2014

http://livehoods.org/

Social Big Data in Government – Example 4

o Egyptian Government plans to develop a new Social Media Monitoring

System at a scale similar to the NSA PRISM program.

o The government will build services to scan social media site like

Facebook, WhatsApp, and Twitter in real-time for keywords that are

“contrary to law and public morality,” or contain calls to protest or

strike.

o The system will also search for and flag any dissent against the

government and identify individuals who help the public form

opinions that are contrary to law.

18 June 2014

http://www.theguardian.com/world/2014/jun/02/egypt-police-monitor-social-media-dissent-facebook-twitter-protest

Social Big Data Infrastructure (1)

Open sourced, flexible and available architecture for large scale

computation and data processing on a network of commodity

hardware.

18 June 2014

Social Big Data Infrastructure –

Super Stream Collider by Insight@NUIG

18 June 2014

superstreamcollider.org for demo

Social Big Data Infrastructure –

Social Media Linked Dataspace by Insight 1

Leveraging Linked Data Platform for Semantically-Interconnecting

Online, Social Media for Corporate Brand and Market Sector

Reputation Analysis

18 June 2014

http://linked2media.eu/

Social Big Data Infrastructure –

Social Media Linked Dataspace by Insight 2

18 June 2014

Social Big Data Infrastructure –

Language Resources for Government Domain

18 June 2014

Islam Hassan and Adegboyega Ojo,

Enabling GOV3.0 through Semantic

Web, Natural Language Processing

and Text Analytics, Transforming

Government (t-Gov) 2014

Workshop, June 12-13, 2014

Requires

collaborating with

different groups:

NLP, Information

Mining, Sensor

Middleware, Social

Software units and

Reasoning and

querying units.

Social Big Data Adoption in Government

Issues and Challenges

18 June 2014

o Developing Social Big Data adoption roadmap

o Evolving Enterprise Data Management? Infrastructure to

accommodate unstructured and semi-structured data, from data

sources such as social media?

o Identifying criteria for selecting social big data solutions that

meets business and IT needs?

o Identifying data governance aspects (e.g. quality, usability,

security, etc.) to be considered when implementing or sourcing

Social Big Data solutions?

o Ensuring that decisions informed by insights generated are

implemented

o How can Social Media Monitoring and Big Data service provider

support agencies in implementing their Social Big Data programs?

http://www.infosys.com/bigdataedge/resources/Documents/7-key-considerations.pdf

Conclusion

Social Big Data is a relatively new phenomenon in government as

shown in examples. Established experiences are few.

Experiences and solutions from private sector may be useful in

designing solutions in government, noting difference in contexts

Social Big Data capabilities (technical, organizational and financial)

has to be developed in government while simultaneously engaging

the services of SBD, Big Data and Cloud service providers.

All the above requires (Social) Big Data and Cloud Computing

adoption policies in government.

18 June 2014