Edge-to-Cloud Compute with MxNet -...
Transcript of Edge-to-Cloud Compute with MxNet -...
![Page 1: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/1.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Aran Khanna, Software Developer, AWS
Miro Enev, Solutions Architect, NVIDIA
Edge-to-Cloud Compute with MxNet
AWS & NVIDIA
Presented @ GTC 2017 &
![Page 2: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/2.jpg)
Overview
Edge-To-Cloud Compute Models
Advantages of Hub & Spoke
Case Study / Demo
References & Tools
![Page 3: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/3.jpg)
IIoT & AI
AI/DL is primed to disrupt the Industrial Sector.
![Page 4: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/4.jpg)
NVIDIA IoT
![Page 5: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/5.jpg)
Build
Operate
Maintain & Repair
Design
Robots & Smart Manufacturing
Predictive Maintenance
Generative Design
Inspection & Efficiency
NVIDIA IoT
![Page 6: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/6.jpg)
Amazon IoT & GreenGrass
![Page 7: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/7.jpg)
Amazon IoT Software & NVIDIA Hardware
Key Functions• Data Ingest
• Compressed Inference
• Full Inference / Trained Model Query
• Model Training
Deployment Models
Cloud <-> Edge
Cloud <-> Hub <-> Edge
![Page 8: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/8.jpg)
Emerging Deployment Model
Cloud
Elastic GPU [ Tesla ]
Hub/Gateway
Jetson [ Tegra ]
Edge
Sensors [ RasPi ]
Trends: Latency, Cost, Reliability, Privacy/Security Policy
![Page 9: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/9.jpg)
![Page 10: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/10.jpg)
![Page 11: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/11.jpg)
Handoff
![Page 12: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/12.jpg)
Use Cases Of ML at The Edge
Transportation AR & VR Industrials
Smart Home Robotics Body Cameras
![Page 13: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/13.jpg)
In This Demo I will:
-Show you why you should care about Machine Learning at
the Edge
-Walk through the value of running deep learning
applications with MXNet on a set of edge devices.
-Demonstrate how AWS services make managing deep
learning at the edge easier than ever.
![Page 14: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/14.jpg)
MXNet Everywhere
Low End:
Raspberry Pi
- 32 Bit ARMv7
- ARM NEON
- Raspbian
High End:
NVIDIA Jetson
- 64 Bit ARM Arch64
- CUDA
- Ubuntu
![Page 15: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/15.jpg)
Easy as 1…2...3...
1.
git clone
https://github.com/
dmlc/mxnet.git
--recursive
3.
cd python &&
sudo python
setup.py install
2.
cd mxnet &&
make
![Page 16: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/16.jpg)
ImageNet Models
- Well studied
- Form the base models for
most image tasks
- Widely available pre-trained
models
Try it Yourself:
http://data.mxnet.io/models
![Page 17: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/17.jpg)
Running the Model
# Load the network parameters from the cloud
def load_model(model_url=“http://data.mxnet.io/models/imagenet/squeezenet”):
urllib.urlretrieve(model_url, ‘model.zip’)
unzip(model.zip)
sym, arg_params, aux_params = mx.model.load_checkpoint(’Image_Model', 0)
mod = mx.mod.Module(symbol=sym, context= mx.cpu())
mod.bind(for_training=False,data_shapes=[('data', (1,3,224,224))])
mod.set_params(arg_params, aux_params)
return mod
# Predict on an Image
def predict(image_path, mod):
img = cv2.cvtColor(cv2.imread(filename), cv2.COLOR_BGR2RGB)
img = cv2.resize(img, (224, 224))
img = np.swapaxes(img, 0, 2)
img = np.swapaxes(img, 1, 2)
img = img[np.newaxis, :]
mod.forward(Batch([mx.nd.array(img)]))
return mod.get_outputs()[0].asnumpy()
![Page 18: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/18.jpg)
Making This Production Ready With AWS
Management
- Rollout changes
- Coordinate local
devices
- Handle network
and hardware
failure
Aggregation
- Gather and
centralize data
from the Edge
- Connect to
existing systems
in the cloud
Logging
- Monitor
performance
- Analyze uptime
- Diagnose
failures
![Page 19: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/19.jpg)
Enter AWS IoT and AWS Greengrass
![Page 20: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/20.jpg)
On The Pi: SqueezeNet Object Classifier
Iandola et. Al
![Page 21: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/21.jpg)
Linking the Pi to AWS
# Load the network parameters
model = load_model.ImagenetModel('synset.txt', 'squeezenet_v1.1')
# Run Predictions on Images Captured With the Local Camera
def Callback(client, userdata, message):
args = json.load(message.payload)
predictions, img = model.predict_from_cam(args.model)
myAWSIoTMQTTClient.publish(”sdk/test/predict”, predictions,1)
if(predictions[0][0] < 0.8):
myAWSIoTMQTTClient.publish(”sdk/test/escalate”, img, 1)
myMQTTClient.connect()
myMQTTClient.subscribe(”sdk/test/control", 1, Callback)
![Page 22: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/22.jpg)
On The TX1: ResNet-152 Object Detector
Szegedy et. Al
![Page 23: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/23.jpg)
Linking the TX1 to AWS
# Load the network parameters
model = load_model.ImagenetModel('synset.txt', ’resnet-v2')
pi_model = load_model.ImagenetModel('synset.txt', 'squeezenet_v1.1')
# Run Predictions on Images Captured With the Local Camera
def Callback(client, userdata, message):
args = json.load(message.payload)
predictions = model.predict_from_img(args.img. args.model)
myAWSIoTMQTTClient.publish(”sdk/test/predict”, predictions, 1)
# Fine tune the squeezenet model to for redeployment to Pi
if(predictions[0][0] > 0.8):
pi_model.update_model(args.img, predictions)
myMQTTClient.connect()
myMQTTClient.subscribe(”sdk/test/escalate", 1, Callback)
![Page 24: Edge-to-Cloud Compute with MxNet - NVIDIAon-demand.gputechconf.com/gtc/2017/presentation/s7571...Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA Edge-to-Cloud](https://reader031.fdocuments.us/reader031/viewer/2022041016/5ec7f1cc6ac82244b105bc51/html5/thumbnails/24.jpg)
Local Learning Loop
Poorly
Classified
Picture
Updated
Model
Fine Tune Model With
Accurate Classification