Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth...
Transcript of Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth...
![Page 1: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/1.jpg)
Building an IP Network Camera
Frank HunlethTwitter: @fhunleth
Erlang Factory 2014
![Page 2: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/2.jpg)
Agenda
● Introducing Erlang to a project● The camera● Demo● Embedded plumbing and development● Performance● Conclusion
![Page 3: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/3.jpg)
Embedded Calibration Slide
Microcontrollers
32-bit Microprocessors
RTOS
Generalpurpose
Embedded Linux
Android, iOS,Ubuntu, Windows
Limited resource/Specific purpose
This talk is focused in this area of embedded
![Page 4: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/4.jpg)
How to use Language X in an Embedded Project
● Low risk - can always implement in C first● Worked in the past for TCL, Lua, and Javascript● My first attempt to introduce Erlang to an organization
so that it could be used for real on the next project
Operating System
C/C++ framework
Config scripts, diagnostic, DSL
C/C++ application
![Page 5: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/5.jpg)
Fail
What makes Erlang/OTP interesting is its focus on robust and scalable systems.
![Page 6: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/6.jpg)
Try #3,4, or 5 - IP Camera
● Build a non-trivial embedded device– Didn't need to be a camera, but wanted a device with a hard
real-time component and decent network usage
– Make sure that SW infrastructure supports production use cases
– Should benefit from in robustness or simplicity by using Erlang/OTP
● Questions– Will Erlang be too slow?
– Will Erlang require too much DRAM and Flash memory?
– Will developing Erlang in a cross-compiled environment be a pain?
– Will I miss all of the libraries and frameworks available in C?
– Will I have confidence in the Erlang/OTP platform?
![Page 7: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/7.jpg)
Constraint
● If a feature exists in both Erlang and an embedded Linux environment, use the Erlang version
● Examples– No shell scripts
– No SysV init, systemd, upstart (these provide process initialization and supervision)
![Page 8: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/8.jpg)
Agenda
● Introducing Erlang to a project● The camera● Demo● Embedded plumbing and development● Performance● Conclusion
![Page 9: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/9.jpg)
Hardware Setup
ImagerMT9V034
GPS
Cape EEPROM
TI AM3359
MicroSD
10/100 Ethernet
Debug Console
Beaglebone Black
Lens Holder
![Page 10: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/10.jpg)
AM3359
Hardware Components
ARM Cortex-A8Programmable Real-time Unit (PRU)
I2C Master
Data/10
Pixel Clock,HSync,VSync
Control - I2C
MT9V034 B&W Imager
GPIO Reset
![Page 11: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/11.jpg)
Cam
era Interfa ce
High-level Software Components
Frame capture
Hard real-timePRU assembler
PRU
Frame 1Frame capture
C PortsErlang
JPEG encodelibjpeg-turbo
Erlang/ALEGPIO
Erlang/ALEI2C
Frame 2
Ready interrupt
Imager reset
Imager control
Erlang/A
LE
Streaming Video Handler
Static Handler(index.html, JS)
Cow
boy
WebSockets Handler(Control)
![Page 12: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/12.jpg)
Motion JPEG Streaming
HTTP/1.1 200 OKcontent-type: multipart/x-mixed-replace; boundary=<boundary>
GET /video HTTP/1.1
Content-Type: image/jpeg
Content-Type: image/jpeg
--<boundary>
Content-Type: image/jpeg
--<boundary>
--<boundary>
![Page 13: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/13.jpg)
Streaming Code
handle(Req, _State) -> Boundary = boundary(), Headers = [{<<"MIME-Version">>, <<"1.0">>}, {<<"content-type">>, <<"multipart/x-mixed-replace; ", "boundary=", Boundary/binary>>}], {ok, Req2} = cowboy_req:chunked_reply(200, Headers, Req), send_first_picture(Req2), send_pictures(Req2).
send_pictures(Req) -> Pic = troodon_cam:get_next_picture(), Msg = [multipart_header(Pic), Pic, delimiter()], ok = cowboy_req:chunk(Msg, Req), send_pictures(Req).
![Page 14: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/14.jpg)
Demo
![Page 15: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/15.jpg)
Agenda
● Introducing Erlang to a project● The camera● Demo● Embedded plumbing and development● Performance● Conclusion
![Page 16: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/16.jpg)
Nerves-Project Work Flow
Erlang/OTP applications
BEAMs and cross-compiled
binaries
Erlang/OTP release
Base Firmware Image
SDCard imageand
firmware update package
rebar or erlang.mk
relx
fwtool + scripts
nerves-sdk customizations to
Buildroot
Updated filesystem on target
(Development)
relsync
![Page 17: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/17.jpg)
Raw SDCard Image
Master Boot Record
Bootloaders
Root Filesystem A
Read-only
Root Filesystem B
Read-only
Application Data Partition
Read-write
Linux kernelErlangLibrariesMain application
Ping-pong locationused when upgradingthe firmware
Majority of nonvolatilememory for use by theapplication
Raw images are only needed for initial code load and bulk device programming
![Page 18: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/18.jpg)
relsync
● Reprogramming SDCards gets old quickly!!!● relsync synchronizes the files in the generated Erlang/OTP
release directory with corresponding ones on the target● Like rsync except
– Communicates via the Erlang distribution protocol
– Reloads modules that changed
– Runs scripts pre and post sync to stop and start Erlang/OTP applications (needed to update ports)
● Limitations– Target must have writable FS (currently using a union FS)
– NIFs and linked-in port drivers can't be updated
![Page 19: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/19.jpg)
Initialization - Booting to Erlang
● erlinit– Replacement for /sbin/init that starts an Erlang/OTP
release
– Similar to a release start script, but in C
– Supports remounting root fs with unionfs for development
– Configurable via Linux kernel command line
![Page 20: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/20.jpg)
Agenda
● Introducing Erlang to a project● The camera● Demo● Embedded plumbing and development● Performance● Conclusion
![Page 21: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/21.jpg)
Performance - Throughput
● Imager set to capture at 45 fps (22 ms/frame)● Browser stats
– Average FPS ~43.5 fps (23 ms/frame)
– About 1 frame dropped per second
● Profiling revealed JPEG encode time taking 22-23.5 ms– AM3358 doesn't support HW encoding
– Did not investigate tuning JPEG Turbo library
● Take away: Erlang is not a bottleneck
![Page 22: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/22.jpg)
End-to-end Latency Measurements
![Page 23: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/23.jpg)
Software Setup
Latency Measurement App(Erlang, of course)
More info: https://github.com/fhunleth/cam_latency
TimestampedFrames
Received: 0.481 (~71 ms)
UARTvia erlang-serial HTTP
via httpc
![Page 24: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/24.jpg)
Latency Results
● Average frame latency 78 ms (best 66 ms, worst 94 ms)
Exposure time
Transfer from
imager
JPEG encode
Erlang / Cowboy/HTTP TX
Receive HTTP stream
Display time
updates
~10ms 20 ms20 ms 23 ms1-11 ms
● Easily accounted for latencies - 54-66 ms● Remaining latencies - 12-30 ms
– Not unexpected; requires more instrumentation
– Doubt that Erlang overhead is significant
![Page 25: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/25.jpg)
Boot Time (4.38 seconds)
![Page 26: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/26.jpg)
DRAM Usage (/proc/meminfo)
● Connect, stream for 5 seconds, disconnect, repeat
● erlang:memory/0 reports 7.7 MB at steady state
![Page 27: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/27.jpg)
Footprint - 18 MB RootFS
3.5 MB 3.38 MB 1.9 MB 1.9 MB 7.39 MB
2.77 MB 1.56 MB 1.30 MB .34 .29 .22 .2 .16 .16 .1 .1
● Erlang/OTP release tools significantly trim footprint● Usage on par with C++ frameworks like Qt
![Page 28: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/28.jpg)
Conclusions
● Will Erlang be too slow?– No. Lack of HW compression is the bottleneck as it should have been.
● Will Erlang require too much DRAM and Flash memory?– No. Flash footprint was on par with C/C++ frameworks. DRAM usage stable.
● Will developing Erlang in a cross-compiled environment be a pain?– Erlang shell + relsync can be pretty nice
● Will I miss all of the libraries and frameworks available in C?– Cowboy is far superior to anything I had used in C/C++
– Still hit or miss when searching for Erlang libraries
● Will I have confidence in the Erlang/OTP platform?– Yes - platform feels very robust (no unexplained crashes)
– Interface to C very easy
– Rough edges look easily fixed with time and effort
![Page 29: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/29.jpg)
Nerves-Project
● All source code is open source– Mostly MIT licensed
– Buildroot and build scripts are GPLv2
● Upcoming– Hobby → Real products
– Documentation
– Network configuration improvements
– Elixir!!
● http://nerves-project.org
![Page 30: Building an IP Network Camera - erlang-factory.com · Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014. Agenda Introducing Erlang to a project The](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c148d6dc475737806268d/html5/thumbnails/30.jpg)
Building an IP Network Camera
Frank HunlethTwitter: @fhunleth
Erlang Factory 2014