CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 1 | 12

Lecture No. 13

Big Data, Internet of Things and Smart Cities

What is Big Data?

Big Data is also data but with a huge size. Big Data is a term used to describe a collection of

data that is huge in size and yet growing exponentially with time. In short such data is so large

and complex that none of the traditional data management tools are able to store it or process it

efficiently.

Examples of Big Data

Following are some the examples of Big Data-

The New York Stock Exchange generates about one terabyte of new trade data per day.

The statistic shows that 500+terabytes of new data get ingested into the databases of social

media site Facebook, every day. This data is mainly generated in terms of photo and video

uploads, message exchanges, putting comments etc.

A single Jet engine can generate 10+terabytes of data in 30 minutes of flight time. With

many thousand flights per day, generation of data reaches up to many Petabytes.

Types of Big Data

BigData' could be found in three forms:

1. Structured

2. Unstructured

3. Semi-structured

Structured

Any data that can be stored, accessed and processed in the form of fixed format is termed as a

'structured' data.

Examples of Structured Data

An 'Employee' table in a database is an example of Structured Data

Employee_ID Employee_Name Gender Department Salary_In_lacs

2365 Rajesh Kulkarni Male Finance 650000

3398 Pratibha Joshi Female Admin 650000

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 2 | 12

7465 Shushil Roy Male Admin 500000

7500 Shubhojit Das Male Finance 500000

7699 Priya Sane Female Finance 550000

Unstructured

Any data with unknown form or the structure is classified as unstructured data.

Examples Of Un-structured Data

The output returned by 'Google Search'

Semi-structured

Semi-structured data can contain both the forms of data. We can see semi-structured data as a

structured in form but it is actually not defined with e.g. a table definition in relational DBMS.

Example of semi-structured data is a data represented in an XML file.

Examples Of Semi-structured Data

Personal data stored in an XML file-

<rec><name>Prashant Rao</name><sex>Male</sex><age>35</age></rec>

<rec><name>Seema R.</name><sex>Female</sex><age>41</age></rec>

<rec><name>Satish Mane</name><sex>Male</sex><age>29</age></rec>

<rec><name>Subrato Roy</name><sex>Male</sex><age>26</age></rec>

<rec><name>Jeremiah J.</name><sex>Male</sex><age>35</age></rec>

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 3 | 12

Data Growth over the years

Please note that web application data, which is unstructured, consists of log files, transaction

history files etc. OLTP systems are built to work with structured data wherein data is stored in

relations (tables).

Characteristics Of Big Data

(i) Volume – The name Big Data itself is related to a size which is enormous. Size of data plays a

very crucial role in determining value out of data. Also, whether a particular data can actually be

considered as a Big Data or not, is dependent upon the volume of data. Hence, 'Volume' is one

characteristic which needs to be considered while dealing with Big Data.

(ii) Variety – The next aspect of Big Data is its variety.

Variety refers to heterogeneous sources and the nature of data, both structured and unstructured.

During earlier days, spreadsheets and databases were the only sources of data considered by

most of the applications. Nowadays, data in the form of emails, photos, videos, monitoring

devices, PDFs, audio, etc. are also being considered in the analysis applications. This variety of

unstructured data poses certain issues for storage, mining and analyzing data.

(iii) Velocity – The term 'velocity' refers to the speed of generation of data. How fast the data is

generated and processed to meet the demands, determines real potential in the data.

Big Data Velocity deals with the speed at which data flows in from sources like business

processes, application logs, networks, and social media sites, sensors, Mobile devices, etc. The

flow of data is massive and continuous.

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 4 | 12

(iv) Variability – This refers to the inconsistency which can be shown by the data at times, thus

hampering the process of being able to handle and manage the data effectively.

Benefits of Big Data Processing

Ability to process Big Data brings in multiple benefits, such as-

o Businesses can utilize outside intelligence while taking decisions

Access to social data from search engines and sites like facebook, twitter are enabling

organizations to fine tune their business strategies.

o Improved customer service

Traditional customer feedback systems are getting replaced by new systems designed with Big

Data technologies. In these new systems, Big Data and natural language processing technologies

are being used to read and evaluate consumer responses.

o Early identification of risk to the product/services, if any

o Better operational efficiency

Big Data technologies can be used for creating a staging area or landing zone for new data before

identifying what data should be moved to the data warehouse. In addition, such integration of

Big Data technologies and data warehouse helps an organization to offload infrequently accessed

data.

What is Internet of things (IoT)

The internet of things, or IoT, is a system of interrelated computing devices, mechanical and

digital machines, objects, animals or people that are provided with unique identifiers (UIDs) and

the ability to transfer data over a network without requiring human-to-human or human-to-

computer interaction.

A thing in the internet of things can be a person with a heart monitor implant, a farm animal with

a biochip transponder, an automobile that has built-in sensors to alert the driver when tire

pressure is low or any other natural or man-made object that can be assigned an Internet Protocol

(IP) address and is able to transfer data over a network. Increasingly, organizations in a variety of

industries are using IoT to operate more efficiently, better understand customers to deliver

enhanced customer service, improve decision-making and increase the value of the business.

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 5 | 12

How IoT works

An IoT ecosystem consists of web-enabled smart devices that use embedded systems,

such as processors, sensors and communication hardware, to collect, send and act on data

they acquire from their environments. IoT devices share the sensor data they collect by

connecting to an IoT gateway or other edge device where data is either sent to the cloud

to be analyzed or analyzed locally. Sometimes, these devices communicate with other

related devices and act on the information they get from one another. The devices do

most of the work without human intervention, although people can interact with the

devices -- for instance, to set them up, give them instructions or access the data.

The connectivity, networking and communication protocols used with these web-enabled

devices largely depend on the specific IoT applications deployed.

IoT can also make use of artificial intelligence (AI) and machine learning to aid in

making data collecting processes easier and more dynamic.

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 6 | 12

Why IoT is important

The internet of things helps people live and work smarter, as well as gain complete control over

their lives. In addition to offering smart devices to automate homes, IoT is essential to business.

IoT provides businesses with a real-time look into how their systems really work, delivering

insights into everything from the performance of machines to supply chain and logistics

operations.

IoT enables companies to automate processes and reduce labor costs. It also cuts down on waste

and improves service delivery, making it less expensive to manufacture and deliver goods, as

well as offering transparency into customer transactions.

As such, IoT is one of the most important technologies of everyday life, and it will continue to

pick up steam as more businesses realize the potential of connected devices to keep them

competitive.

IoT benefits to organizations

The internet of things offers several benefits to organizations. Some benefits are industry-

specific, and some are applicable across multiple industries. Some of the common benefits of IoT

enable businesses to:

monitor their overall business processes;

improve the customer experience (CX);

save time and money;

enhance employee productivity;

integrate and adapt business models;

make better business decisions; and generate more revenue.

IoT encourages companies to rethink the ways they approach their businesses and gives them the

tools to improve their business strategies.

Generally, IoT is most abundant in manufacturing, transportation and utility organizations,

making use of sensors and other IoT devices; however, it has also found use cases for

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 7 | 12

organizations within the agriculture, infrastructure and home automation industries, leading some

organizations toward digital transformation.

IoT can benefit farmers in agriculture by making their job easier. Sensors can collect data on

rainfall, humidity, temperature and soil content, as well as other factors, that would help

automate farming techniques.

The ability to monitor operations surrounding infrastructure is also a factor that IoT can help

with. Sensors, for example, could be used to monitor events or changes within structural

buildings, bridges and other infrastructure. This brings benefits with it, such as cost saving, saved

time, quality-of-life workflow changes and paperless workflow.

A home automation business can utilize IoT to monitor and manipulate mechanical and electrical

systems in a building. On a broader scale, smart cities can help citizens reduce waste and energy

consumption.

IoT touches every industry, including businesses within healthcare, finance, retail and

manufacturing.

Pros and cons of IoT

Some of the advantages of IoT include the following:

ability to access information from anywhere at any time on any device;

improved communication between connected electronic devices;

transferring data packets over a connected network saving time and money; and

automating tasks helping to improve the quality of a business's services and reducing the

need for human intervention.

IoT standards and frameworks

There are several emerging IoT standards, including the following:

IPv6 over Low-Power Wireless Personal Area Networks (6LoWPAN) is an open standard

defined by the Internet Engineering Task Force (IETF). The 6LoWPAN standard enables any

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 8 | 12

low-power radio to communicate to the internet, including 804.15.4, Bluetooth Low

Energy (BLE) and Z-Wave (for home automation).

ZigBee is a low-power, low-data rate wireless network used mainly in industrial settings.

ZigBee is based on the Institute of Electrical and Electronics Engineers (IEEE) 802.15.4

standard. The ZigBee Alliance created Dotdot, the universal language for IoT that enables

smart objects to work securely on any network and understand each other.

LiteOS is a Unix-like operating system (OS) for wireless sensor networks. LiteOS supports

smartphones, wearables, intelligent manufacturing applications, smart homes and the internet

of vehicles (IoV). The OS also serves as a smart device development platform.

OneM2M is a machine-to-machine service layer that can be embedded in software and

hardware to connect devices. The global standardization body, OneM2M, was created to

develop reusable standards to enable IoT applications across different verticals to

communicate.

Data Distribution Service (DDS) was developed by the Object Management Group (OMG)

and is an IoT standard for real-time, scalable and high-performance M2M communication.

Advanced Message Queuing Protocol (AMQP) is an open source published standard for

asynchronous messaging by wire. AMQP enables encrypted and interoperable messaging

between organizations and applications. The protocol is used in client-server messaging and

in IoT device management.

Constrained Application Protocol (CoAP) is a protocol designed by the IETF that specifies

how low-power, compute-constrained devices can operate in the internet of things.

Long Range Wide Area Network (LoRaWAN) is a protocol for WANs designed to

support huge networks, such as smart cities, with millions of low-power devices.

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 9 | 12

IoT Famous Platform:

1. Amazon Web Services (AWS) IoT is a cloud computing platform for IoT released by

Amazon. This framework is designed to enable smart devices to easily connect and

securely interact with the AWS cloud and other connected devices.

2. Arm Mbed IoT is a platform to develop apps for IoT based on Arm microcontrollers.

The goal of the Arm Mbed IoT platform is to provide a scalable, connected and secure

environment for IoT devices by integrating Mbed tools and services.

3. Microsoft's Azure IoT Suite is a platform that consists of a set of services that enables

users to interact with and receive data from their IoT devices, as well as perform various

operations over data, such as multidimensional analysis, transformation and aggregation,

and visualize those operations in a way that's suitable for business.

4. Google's Brillo/Weave is a platform for the rapid implementation of IoT applications.

The platform consists of two main backbones: Brillo, an Android-based OS for the

development of embedded low-power devices, and Weave, an IoT-oriented

communication protocol that serves as the communication language between the device

and the cloud.

5. Calvin is an open source IoT platform released by Ericsson designed for building and

managing distributed applications that enable devices to talk to each other. Calvin

includes a development framework for application developers, as well as a runtime

environment for handling the running application.

Consumer and enterprise IoT applications

There are numerous real-world applications of the internet of things, ranging from consumer IoT

and enterprise IoT to manufacturing and industrial IoT (IIoT).

In the consumer segment, for example, smart homes that are equipped with smart thermostats,

smart appliances and connected heating, lighting and electronic devices can be controlled

remotely via computers and smartphones.

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 10 | 12

Wearable devices with sensors and software can collect and analyze user data, sending messages

to other technologies about the users with the aim of making users' lives easier and more

comfortable. Wearable devices are also used for public safety -- for example, improving first

responders' response times during emergencies by providing optimized routes to a location or by

tracking construction workers' or firefighters' vital signs at life-threatening sites.

In healthcare, IoT offers many benefits, including the ability to monitor patients more closely

using an analysis of the data that's generated. Hospitals often use IoT systems to complete tasks

such as inventory management for both pharmaceuticals and medical instruments.

Smart buildings can, for instance, reduce energy costs using sensors that detect how many

occupants are in a room. The temperature can adjust automatically -- for example, turning the air

conditioner on if sensors detect a conference room is full or turning the heat down if everyone in

the office has gone home.

In agriculture, IoT-based smart farming systems can help monitor, for instance, light,

temperature, humidity and soil moisture of crop fields using connected sensors. IoT is also

instrumental in automating irrigation systems.

What is a smart city?

A smart city is a framework, predominantly composed of Information and Communication

Technologies (ICT), to develop, deploy, and promote sustainable development practices to

address growing urbanization challenges.

A big part of this ICT framework is essentially an intelligent network of connected objects and

machines that transmit data using wireless technology and the cloud.

Cloud-based IoT applications receive, analyze, and manage data in real-time to help

municipalities, enterprises, and citizens make better decisions that improve quality of life.

Citizens engage with smart city ecosystems in a variety of ways using smartphones and mobile

devices, as well as connected cars and homes. Pairing devices and data with a city’s physical

infrastructure and services can cut costs and improve sustainability.

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 11 | 12

Communities can improve energy distribution, streamline trash collection, decrease traffic

congestion, and even improve air quality with help from the IoT.

For instance,

Connected traffic lights receive data from sensors and cars adjusting light cadence and timing to

respond to real-time traffic, thereby reducing road congestion.

Connected cars can communicate with parking meters and electric vehicle (EV) charging docks

and direct drivers to the nearest available spot.

Smart garbage cans automatically send data to waste management companies and schedule pick-

up as needed versus on a pre-planned schedule.

And citizens’ smartphone becomes their mobile driver’s license and ID card with digital

credentials, which speeds and simplifies access to the city and local government services.

Together, these smart city technologies are optimizing infrastructure, mobility, public services,

and utilities.

How is IoT technology making cities smarter and better?

Secure wireless connectivity and IoT technology are transforming traditional elements of city life

- like streetlights - into next-generation intelligent lighting platforms with expanded capabilities.

The scope includes integrating solar power and connecting to a cloud-based central control

system that connects to other assets in the ecosystem.

These solutions shine far beyond simple lighting needs.

High-power embedded LEDs alert commuters about traffic issues, provide severe

weather warnings, and provide heads up when environmental like fires arise.

Streetlights can also detect free parking spaces, EV charging docks, and alert drivers

where to find an open spot via a mobile app. charging might even be able from the

lamppost itself in some locations!

What makes smart cities successful?

In addition to people, dwellings, commerce, and traditional urban infrastructure, there are four

essential elements necessary for thriving smart cities:

Pervasive wireless connectivity

Open data

CS-111 Intro to Information and Communication Technologies BS Computer Science - SSUET

Prepared By Engr. M.Nadeem P a g e 12 | 12

Security you can trust in

Flexible monetization schemes

What’s the best wireless technology for smart cities?

The first building block of any smart city application is reliable, pervasive wireless connectivity.

While there’s no one-size-fits-all, evolving Low Power Wide Area Network (LPWAN)

technologies are well suited to most smart city applications for their cost efficiency and

ubiquity.

These technologies include LTE Cat M, NB-IoT, LoRa, Bluetooth, and a few others that all

contribute to the fabric of connected cities.

The advent of 5G technology is expected to be a watershed event that propels smart city

technology into the mainstream and accelerates new deployments.

Examples of smart cities

London, New York, Amsterdam, Paris, Reykjavik, Tokyo, Singapore, Copenhagen, Berlin,

Vienna