1

Arpan PalHead, Innovation Lab, Kolkata

R&D in Intelligent InfrastructureInnovation Labs, Kolkata and Bangalore

Theme Outline

3

Intelligent Infrastructure Theme

“Intelligent”• Context-aware

• Sense the environment• Extract meaningful information• Analyze to make inferences• Respond through value-added information

“Infrastructure”• Home, Building, Store, Vehicle, Roads,

Utility Generation / Distribution Network

IntelligentInfra

@Home

@Building

@Vehicle@Utility

@Mobile

@Store

@Road

Target Verticals • Transportation / Govt. Authorities / Healthcare / Education /

Energy / Water

Goals• Build common framework and platforms • Example use cases for initial platform requirements• Platform Differentiators

• Multi-system Integration and Fusion / Analytics and Privacy Control Toolboxes • Long term Research Outcomes in

• Sensing, Communication, Informatics, Robustness, Large Data Handling, HMI, Privacy/Trust and Services

• Service level interfaces on top of the Platform for Business Teams to write Intelligent Infrastructure Applications

4

Integrated Platform for Intelligent Infrastructure

People Feedback & Emotions

Social Media

Integrated Services

Sensors & IoTPlatform

Traditional Monitoring & Control Systems Citizen Data

Smart Integration Platform

Transportation Healthcare Electricity

WaterPublic Safety Tourism

Smart Integrated Services

Sense

Analyze

Extract

Respond

Intelligence

Smart Domain Services

Community

etc.

Sense: People Activity, Appliances, Vehicles , Road, Home/Bldg, Utility Infrastructure

Detect gas leakage/water contamination : mobilize rescue team, suggest optimum route

Divert Road Traffic in case of Water Pipeline Burst

Correlate Electricity/Water /Gas consumption patterns

Intelligent Integration Platform

Integrated Intelligent Services

Real-time Integrated Platform for Services and Analytics (RIPSAC)

5

Overall Solution Landscape

Example Application Use CasesIntelligent Transportation Pothole Detection / Share-a-ride / Unmanned Level Crossing Alert Car Diagnosis / Prognosis Driving Behavior Analysis / Automated Insurance ClaimIntelligent Retail Phone-based localization in Shopping MallsIntelligent Utility Home Energy GatewayIntelligent City Services Surveillance for Safe City Remote Healthcare Distance EducationIntelligent Agriculture WSN for mKrishi

Internet

Research Focus

7

RIPSAC Architecture

Internet

End Users & Renderers

Administrators

Data Aggregation

Device Monitoring & Management

Device Integration & Management Services

Visualization

Components

Statistical Analysis

Machine Learning

Rule Engines

Event Processing

Reasoning &

OntlologyOptimization

Analytics Services

Application Components

Application Services

Data Storage

Data Access / Query

Continuous Queries

Storage

Data & Event Distribution Messaging

Messaging & Event Distribution Services

Core Services

Ap

plic

ati

on

Serv

ices

Presentation Services

Portal User Interfaces

Application Support Services

Integration & Orchestration

PlannersPlatform API

& SDKs

Internet / WAN

Sensors & Gateway Devices

OS & Device Drivers

Edge Middleware

AnalyticsPersistenc

e

Application Services

APPLICATION SERVICES

BACK-END PLATFORM

INTERNET

GATEWAY

Privacy

BACK-END PLATFORM

GATEWAY

8

RIPSAC Platform Research

Challenges• Reducing the cost of Communication• Preserving the Battery power• Extracting information from ambient-noise-corrupted sensor data• Multi-sensor fusion and analytics, sometimes in real-time• Security and Privacy• Common Framework for Sensor Data Acquisition, Storage and Management

Approach•Edge processing algorithm toolbox (sensor, audio, image, video) for context information extraction

addresses the information extraction, cost of communication, privacy and battery power preservation issue

•Lightweight communication protocols over Internet addresses the cost of communication and battery power preservation issue

•Algorithm toolbox for real-time streams / complex event processing, data sketching, reasoning, fusion and non-parametric statistical processing

addresses the real-time multi-sensor fusion and analytics issue•Privacy preserving sensor data masking, encryption and tokenizing toolbox with access control and authentication

addresses the security and privacy issue•OGC based sensor data acquisition, storage and management framework

Collaboration with ISI, Kolkata on Privacy

9

Accelerometer Analytics in Mobiles

Accelerometer Feature extraction and FusionUse Case - Pothole detection, Driving behavior analysis

• Learning-based adaptive design• Feature extraction and anomaly detection done

on mobile device to conserve battery power, reduce communication cost and preserve privacy

• Voting based Fusion at Back-end• Low computational complexity algorithm using

low rate sampling

Car-road Interaction ModelingUse Case - Pothole simulation, Car Prognosis

• Model based on physical interaction between cars and roads

• Inverse modeling for Car Prognosis

10

Use Case - Help people find a Ride-share

• Social sub-graph formation between ride-givers and ride-takers

• Real-time decision system based on instantaneous location and route matching

• Streams Reasoning for real-time traffic condition alert and suggestions about alternate ride-givers

• Trust model based on “Friend-of-Friends” kind of relationship

• Reduction of search complexity

Intent based Social Network Formation

11

Mobile Location based Alerting

Use Case - Automated SMS Alert System for Unmanned Level Crossings

• Instantaneous prediction train approach based on location and speed sensing

• Instantaneous identification of mobile phones in the vicinity of level crossing through LBS and Intent discovery

• Real-time alert system using SMS• Streams Reasoning for sophisticated

alert logic

12

Upper Audio band Communication in Mobiles

Use Case - In-store loyalty programs and Target promotions for Retail

• Complete baseband communication chain design using upper audio band (16 to 22 KHz) carrier

• Low computational complexity to preserve battery power

• Novel waveform design for fast detection time (less than 1 second)

• Programmable range (1 m to 8 m) through power control

• Error Control Coding for Robustness

13

Multimedia Processing for Context Extraction

Use Case – Indoor People Counting for TV Viewership

• Multi-algorithm fusion to improve detection probability

• Face Detection, Body Posture Detection and Background Modeling are combined to improve the detection above 92 % from 75 %

Use Case – Audio and Gesture-based Interaction

• Gesture sensing through Kinect• Resolution of Multi-party commands

through directional audio augmented with gesture

14

Image Processing in Mobiles

Use Case – Automated Insurance Claim

• Noise cleaning and pre-processing on VIN Images

• Low computational complexity OCR on VIN images

• Vehicle Contour based Damage Estimation based on Template matching

Use Case – Participatory sensing for Surveillance

• Reduction in Video Compression computation Complexity

• Innovative sampling strategy

Collaboration with ISI, Kolkata

15

Analog Video multiplexing and metadata insertion

internet

Use Case – Distance Education System

• Designed for Bandwidth / spectrum constrained environments

• Crowd-sourced lecture content generation

• Multimedia over Broadcast and Interactivity over Internet with innovative synchronization scheme

• Multiplexing of multiple video lectures and audio in a single video stream

• Insertion of Metadata within analog video

Collaboration with IIT, Kharagpur

16

Electrical Load Analytics

Use Case – Home / Building Energy Gateway for Appliance level energy consumption detection

• Multi-protocol Gateway compliant to multiple standards

• Appliance Identification from aggregated meter data using source separation

• Demand-response and load-scheduling

• Smart-grid monitoring• Carbon-footprint Analytics

Collaboration with IIT, Mumbai

17

Power-efficient wireless Mesh Network

Use Case – Mobile Agro-advisory system

• Real-time sending of Soil Temperature, Soil Moisture, Atmospheric Temperature and Atmospheric Humidity

• Cellular Backhaul• Zigbee based wireless mesh

network• Power-efficient sleep mode design

18

Multimodal Event Filtering and Response Planning

Filter with city soundscape info Heat-map based on participatory sensing Categorize traffic noise, gunshot or explosion

Filter with social network analysisNLP and Semantic analysis of geo-coded social n/w tweets

Filter with local participatory surveillance- Event triggered participatory sensing

reduced false alarm

Events from video surveillance system

Fusion of information

Use Case – City Surveillance SystemIssue with existing audio / video surveillance systems• Too many false alarms on events• Audio sensors are static and hence not

pervasive

Emergency ResponsePlanner

Scalable, Adaptable, with plan execution monitoring and real-time plan revision

19

Large Scale Storage

Use Case – Large Scale Surveillance Video Storage for safe city • Scalable, peta-byte

scale storage for video and audio tracks and event meta data

• Web services based standardized interfaces

• Rich query interface

TCS NVR

Internet /WAN

Event in XML, Video

Camera Controls, Camera Receiving

Live View

Network Video Storage

NVTRTSP URI

Video/Audio Track & Metadata –

Camera ID, Loc, Tilt/Pan

Camera Controls, Camera Receiving

Live View

NVARTSP URI

Video/Audio Track & Metadata – Event XML

NVT Receivin

g

NVS Receivin

g

TCS NVR Interface ONVIF 1.2 Interface

NVT/NVAONVIF 2.0 Interface

NVT/NVAOther standard

Interface like PISA

Device Management, Device Discovery

Recording, Receiving

Query

SOSGeospatial DB, Key value

Store

BLOB StoreObject Store

Server & Storage Farm

SES

NVD

NVR – Network Video RecorderNVD – Network Video DisplayNVT – Network Video TransmitterNVA- Network Video AnalyzerNVS – Network Video StorageSOS- Sensor Observation ServicesSES – Sensor Event ServicesONVIF – Open Network Video Interface Forum

Live Event SubscriberStored Event Viewer

20

Non-parametric Statistical Processing on Big data

Local end patient display

ECG Ultrasound Blood Pressure Viscope

Instrument Control

Remote End(Home / Medical Camps)

Patient

ECG

USG

ViScope

Camera

Modem Console

MicrophoneDisplay

MedicalRepository

Internet

Internet

Internet

Live Session

Recorded Session

Expert End(Hospital/Doctor’s clinic)

Use Case – Remote Medical Consultation, Continuous Patient Monitoring, Push-based Patient Alerts

• Multi-protocol Gateway (device / mobile)

• Relation Pattern / Clustering from Patient medical sensor data

• Rule Engine to send pro-active alerts

• Augmentation of secondary feeds (e.g. weather, pollution level etc.0

• Integration with Clinical Decision-support System

21

Frugally-engineered ubiquitous devices

Home Infotainment Platform• Over-the-top STB• Mash-up of Internet content with

Broadcast content• Infotainment, Banking and Hotel

Applications• Also used for Home Energy

Gateway, Medical Gateway and Distance Education

ECG Monitors• 3-lead wearable and 12-lead

portable ECG monitors• Continuous monitoring for a

prolonged period for preventive as well as post-hospitalization care

Thank Youarpan.pal@tcs.com