Cps innovation lab kolkata iiest
-
Upload
arpan-pal -
Category
Technology
-
view
18 -
download
0
Transcript of Cps innovation lab kolkata iiest
1 Copyright © 2014 Tata Consultancy Services Limited
Internet-of-Things and Cyber-physical Systems- Exploratory Research in Signal Processing, Communication and Computing
Dr. Arpan PalPrincipal Scientist and Research HeadInnovation Lab, KolkataTCS
Apr 15, 2023
2
Cyber-physical Systems – Internet-of-Everything
Humans
Physical Objects
and Infrastruct
ure
Computing Infrastruct
ure
Peo
ple
Con
text
Dis
cove
ry
PhysicalContext Discovery
INTERNET OF EVERYTHING
Physical Context
Discovery
What is happening, where and when
People Context
DiscoveryWho is doing what, where and when,
who is thinking what
Internet of
Digital
Internet of
Things
Internet of
Humans
ABI Research. May 7, 2014
"In the next century, planet earth will don an electronic skin. It will use the Internet as a scaffold to support and transmit its sensations.“ - Neil Gross 1999
3
It’s a Connected Universe
NEVER FORGET YOUR PILLS MONITOR YOUR ACTIVITYMONITOR THE AGED
source: http://postscapes.com/internet-of-things-examples/
HEAT YOUR HOME EFFICIENTLY
MAKE SURE THE OVEN IS OFF
TRACK DOWN THOSE LOST KEYS
KEEP THE CITY CLEAN RECEIVE POLLUTION WARNINGSUSE ELECTRICITY MORE EFFICIENTLY
ConnectedIndividual
ConnectedHome
ConnectedCity
4
Research Programs Outline
Mobile Phone Sensing
Camera Sensing Other SensorsBio-Sensing
Signal and Image ProcessingProtocols and Networking
Parallel and Distributed ComputingData Analytics (Computational and Semantic) and Modeling
EDGE
CLOUD
Personal Context Discovery (Location, Activity, Psychology)• Mobiles• Cameras • Bio-sensing
• Manage Scale •Reduce Network Load•Increase Compute Capacity•Reduce Storage Requirement
• Handle Interoperability
• Easy-to-use Analytics
Physiological Sensing using Mobile Phones
Mobile phone and Robot based sensing
Human Activity Detection and
Behavior Modeling
IoT Platform Solutions
Affordable Wellness
&Healthcare
Mobile Interactive Remote Sensing
PROGRAMS
6
Integrated Platform for Intelligent Enterprise
People Feedback & Emotions
Social Media
Integrated Services
Sensors & IoTPlatform Legacy Monitoring & Control Systems Enterprise Data
Smart Integration Platform
Transportation Human Resources Energy
OperationsSafety Asset Tracking
Smart Integrated Services
Sense
Analyze
Extract
Respond
Intelligence
Smart Domain Services
Supply Chain
Security and Surveillance
Sense: People Context, Appliances, Building, Plant, Utility Infrastructure
Sync Transportation with Remote Operations Link Asset Tracking and Safety with Surveillance
Employee Wellness and Energy Preservation as Community Initiatives
Intelligent Integration Platform
Integrated Intelligent Services
7
Requirements and Challenges for IoT – Need for a Platform
Applications need support for
Visibility Capture & store data from sensors
InsightsPatterns, relationships and models
Control Optimize and actuate
TCUP – TCS Connected Universe PlatformA horizontal platform for addressing the IoT Software and Services market
Model-driven DevelopmentModel the Domain Knowledge
Model the Infrastructure – Network, Storage, Compute
Model the Analytics – map to Domain Requirements
Model the Architecture – Device and Cloud
TCUP Platform
Model the Sensor – Semantics, Phenomenon
8
TCUP Design and Architectural Highlights
18 patents filed, Standard Body Contribution - IETF and Singapore ITSC
• Fog Computing – Utilize unused compute power of edge devices
Distributed Computing on Edge Devices
• To reduce network congestion• Adequate Security and Reliability
Adaptive, Lightweight yet Secure Communication Protocols
• For economical scaling of sensor data store
Efficient Compression
• Statistical and Information-theoretic measure to find out potential privacy-breaching content
Sensitivity Measurement and Privacy Preservation
• Semantic annotation of sensors• Sensor Search Engine
Semantic Enabled Sensor Explorer
• Algopedia – Algorithm Repository, Search and Recommendation• Semantic Sensor Web
Model-driven Development
ManageScale,
Reduce Cost
Handle Privacy
Manage Diversity and Inter-
opEase of
Development
9
Horizontal operators(semantic integration) operates on data from heterogeneous sources to created integrated data streams.
Semantic Sensor Web - From Data to Wisdom
temperature
humidity
odor
image
high temperature
gaseous odor
light
concentrated light
high temperature indicates fire
gaseous odor indicates gas discharge
Fire from Gas Leak, evacuate
immediately, send fire fighting team
equipped with gas leakage
data
information
knowledge
wisdom
Vertical operators(semantic abstraction) operates on artifacts at each level and transcends them to the next level
F PCS(Data, KB*) → Information
F PCS(Knowledge, KB) → Wisdom
F PCS(Information, KB) → Knowledge
KB: Knowledge base
Adopted from: Physical-Cyber-Social Computing: An early 21st Century Approach, Amit Sheth et. al.
10
Research Outcomes – Some of the Results
Publications in ACM Sensys, Ubicomp, Infocomm, Middleware
7 times less network load
10 times less latency
Lightweight Security – 5
times less overhead
Improved Privacy via Sensitivity Detection
Improved Compute through Edge Devices
12
Human Data Collection and Behaviour Modelling - Program Overview
Research Goals - Given a context, predict behaviour. Given behaviour, find out context.Focus Domains – Organization Behavior, Consumer BehaviorCurrent Available Models – Statistical, Need of the Day – Models based on physical data
Meetings – Group or One-on-one
Digital Communication
Individual @Work or Leisure
Individual – Day in the Life Of
• Strength and Polarity of Relationships
• Meeting Flow• Emotive State• Outcome
• Formal / Informal, Business / Social
• Strength and Polarity of Relationships
• Tenor of Communication
• Nature of Communication
• Location and Time spent
• Mood and Physical State
• Engagement Level
• Cognitive Load
• Location and Time spent
• Activity• Behavioral
Routine• Social
Interaction• Moment of
decision makingMicrophone
Email / Knome
Mobile Phone, Desktop Mounted Cameras, EEG/GSR
Mobile Phone, Kinect, EEG/GSR,Smart MeterSurveillance Camera
13
Human Identification and Activity Detection using Kinect
Human Identification– Skeleton Model Based / Depth
based– 20 joints of skeleton data
• 2D Camera with IR depth sensor
• Excitation by IR light pattern
Human Identification • Gait cycle detection• Feature extraction from
skeleton joints• Training• Recognition
Papers in IEEE Fuzz, CEC, IEEE SMC, UbiComp and ECCV
Activity • Sitting• Standing• Walking
Human activity recognition using RGB-DAccuracy is above 90%
14
Activity Detection using Mobile Phone Inertial Sensors
Activity Detection– Uses Accelerometer Data– Gyroscope and Magnetometer for orientation
correction– Step Count, Stride Length Estimation– Walking, Brisk Walking, Running, Sitting, Falling
Classification
Peak Detection and Step Validation using IPA;Calculating Step cycle lengths for all valid steps in the window
Classification of window activity using step frequencies derived from step cycle lengths
Noise Cancellation and pre-processingCalorie Count from Step Count and Type of Activity
Papers in UbiComp
~90% Accuracy~80% Accuracy
15
Geo-fencing– Using Magentometer
Proximity Detection– Using Bluetooth RSSI
Inertial Navigation– Step Count + Stride Length (personalized
model)– Gyroscope and Magnetometer-corrected
Inertial Navigation Wi-Fi based Zoning and Triangulation
– Based on RSSI of known location of 3 or more access points
– Attenuation modeling of the building– Unsupervised Learning through physical
modeling Fusion, Tracking and Correction
– Kalman Filter based Tracking– Particle Filter based Correction
Mobile Phone based Indoor Localization – Inertial and Wi-Fi
• Colleague Finder in Large Offices
• Shopper Localization in Retail Stores
• Emergency Evacuation in Large Buildings
Papers in Mobiquitous, UbiCompSelected for Indoor Localization
Competition in IPSN 2014
16
Cognitive Load on Human Brain
Cognitive Load 23+45=? 1846890129 + 2374609823=?
Use Cases Personalized education User interface design
EEG GSR
Papers in IEEE SMC, IEEE Fuzz, ACM BIBE
17
Emotion and Engagement level using Camera
Why Camera− Unobtrusive Sensing @Work
Purpose− Identifying the mood a person at work− How a person is engaged at work
Scope− Facial emotion analysis on unconstrained environment using
camera on desktop/laptop − Mood and human engagement at work place using desktop camera
Partial occlusion for facial expression recognition Engagement analysis Micro emotions / expressions
http://www.ecse.rpi.edu/~cvrl/tongy/aurecognition.html
Irfan Essa (1994), “Analysis, interpretation and synthesis of facial expressions“, PhD Thesis, MIT, Cambridge, MA, USA. (Advisor: Alex (Sandy) Pentland)
19
Affordable Healthcare using Mobile Phones
Sensing various physiological parameters using smartphone sensors with minimal attachments
Low cost solution for initial screening in preventive healthcare / wellness
Solutions need to be stable, repeatable and robust
New disease diagnostics and treatment protocols
– Requires long observation over large set of patients
Geriatric care, monitoring chronic patients Go into Wearable in future Heart condition
– Heart rate, Heart rate variability, Blood Pressure, ECG
– Foetal Heart rate Lung condition
– Spirometry, Respiratory Rate Pupil condition
– Pupillary dilation response Pulse Diagnostics
Purpose
Scope
Robust Solutions using camera, microphone and accelerometer
20
PoC Approach and Expected Novelty
HR, HRV, RRPPG extractionRealtimeVideo
Audio from Mic
Accelerometer
Cardiovascular Model
SpO2
BP, ECG
Spirometry
Breathing Rate
Image of eyePupillary Reflex
21
Photo-plethysmography (PPG) using Mobile Phone Camera
Subject1 Subject2 Subject3
Actual Detected Actual Detected Actual Detected
68 66 66 63 85 84
2.9% 4.5% 1.1%
Papers at Mobihoc, IEEE BIBE, SenSys, ICASSP
Data set Pd Ps PP-diff < 15
Standard dataset (14 features) 92.9% 74.7% 77.9%
TCS dataset - add height, weight, age
99.3% 82.7% 85.5%
23
Sensing the Physical World
Mobile phone based crowd sensing
Robot assisted sensing
www.popularmechanics.com
www.engadget.com
www.allthingssd.com
apollo2.cs.illinois.edu
Camera based sensing
24
Intelligent Transportation – Vehicle Model Driven Sensor Data Analysis
KNOWN PARAMETERS EFFECTS TARGET INDUSTRYVehicle Type & Driving Behavior Road Condition Monitoring City MunicipalityRoad Condition & Driving Behavior Car Prognosis Automotive
Road Condition & Vehicle Type Driving Behavior Analysis Insurance
Acceleration a(t) = f (H(t), v(t), R(t), D(t))
H(t)
Papers in ICST, Percom
25
Phone Microphone based Sound Scaping
Solution OverviewEvent driven with participatory sensing aided audio surveillance system • Classification of Traffic Noise (Honk Detection) and Crowd Noise
Papers in CODIS, ISSNIP, ISDA
26
Phone Camera based 3D Reconstruction from 2D images
Input Images
Dense Reconstruction without using mobile inertial sensors
- 20 images, compute time (4 core, 1GPU) ~ 20 min
- 120 images, compute time (16 core, 1GPU) ~ 30 min
- Bandwidth saving ~ 8 times, if done on mobile
Sparse Reconstruction using Mobile Inertial Sensors for Camera Position Estimation
• 20 images, compute time (4 core, 1GPU) ~ 3 min (without using inertial sensors)
• 20 images – compute time (4 core, 1GPU) ~10 sec. (with inertial sensors)
• Bandwidth saving ~ 200 times, if done on mobile
• Sparse good enough for many applications• Mobile Sensing and ACCV (submitted)• Dense Reconstruction with mobile
inertial sensors under progress with more number of images target < 1min
Dense Reconstruction -120 images
Dense Reconstruction - 20 images
• Low cost solution for 3D reconstruction from multiple 2D images captured from mobile device.
• Motion information from the inbuilt inertial sensors – for camera position estimation
• Applications in Agro-advisory service, Remote Diagnostics, Remote Healthcare
27
Multi-sensor Fusion for Robot-assisted Sensing
Application in remote sensing in hazard-prone areas• Robot carries 2D camera and heat / chemical sensors on a rotating arm• 3D reconstruction from the 2D vision• Estimation of Heat / gas leak / sound Source (direction and range) through passive
directional signal processing• Fusion of heat / gas / sound map on reconstructed 3D vision map
www.ese.wustl.edu
Ongoing Work Possible reuse from 2D-3D reconstruction and sound
classification
Cloud point from 3D vision
Possible gas / heat
source (ROI)
Source direction
and intensity
29
TCS - Pioneering IT Innovation in India
1970s
Offshore Model Alliances with Major IT
Players
University Alliances, Systems Engineering
1970s -1980s
New Labs, Bio SuiteDhruvam
Silicon Valley Ecosystem
Mastercraft, Revine,Quartz Program,
Tools Foundry, Rice Husk Ash
2000 - 2005
2007-2011
IPR Focus and Policy Talent Management
TCS COIN TM
Software Tools -Casepac Migration
Re-engineering
Industry & Services Practices Set Up
Foundation of TRDDC-India’s first Industrial R&D
Center in IT, S/W Engg
CTO/R&D structureInnovation Framework
More Domain Labs
2006
1981 1987
1991
1998
2012
4E ModelScaled
Invention, Co-Innovation
The journey continues…
30
Innovation@TCS - Innovation Labs
Bangalore, India1
TCS Innovation Labs - Bangalore
Chennai, India2
TCS Innovation Labs - ChennaiTCS Innovation Labs - RetailTCS Innovation Labs - Travel & HospitalityTCS Innovation Labs - InsuranceTCS Innovation Labs - Web 2.0TCS Innovation Labs - Telecom
Cincinnati, USA3
TCS Innovation Labs - Cincinnati
Delhi, India4
TCS Innovation Labs - Delhi
Hyderabad, India5
TCS Innovation Labs - HyderabadTCS Innovation Labs - CMC
Kolkata, India6
TCS Innovation Labs - Kolkata
Mumbai, India7
TCS Innovation Labs - MumbaiTCS Innovation Labs - Performance Engineering
Peterborough, UK8
TCS Innovation Labs - Peterborough
Pune, India9
TCS Innovation Labs - TRDDC - Process EngineeringTCS Innovation Labs - TRDDC - Software EngineeringTCS Innovation Labs - TRDDC - Systems ResearchTCS Innovation Labs - Engineering & Industrial Services
1 2
3
4
597
6
8
• Associates in R&D2000+
• Innovation Labs19• Papers Published in last two years1000+
• Patents filed in last two years350+
• Patents granted till date100+• TCS RSP funded Research Scholars150+• Internships in Labs in last two years200+