ROS-COMPLIANT FPGA COMPONENT TECHNOLOGY … 2017 ROS Compliant... · PEAR-LAB Utsunomiya Univ. /...
-
Upload
truongkhanh -
Category
Documents
-
view
215 -
download
0
Transcript of ROS-COMPLIANT FPGA COMPONENT TECHNOLOGY … 2017 ROS Compliant... · PEAR-LAB Utsunomiya Univ. /...
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
ROS-COMPLIANT FPGA COMPONENT TECHNOLOGY
ndash INSTALLATION OF FPGA INTO ROS
Takeshi Ohkawa Yutaro Ishida
Yuhei Sugata Hakaru Tamukoh Utsunomiya University
Kyushu Institute of Technology
2017922 ROSCon2017Vancouver 1
This research and development work (done by Utsunomiya Univ)
was supported by the MICSCOPE152103014
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 2
Problem of computer vision on robots
times 5 Speed
Over 1 min to recognize just six objects
RoboCup 2016
Domestic Pre-Competition
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
FPGA accelerates DNN
bull ldquo1 min is too long for the taskrdquo
bull GoogLeNet[3] (22 layers)
bull 4 seconds for an inference
bull CPU Core i5-5200U 22GHz
bull Need of accelerator
bull GPU ndash Power consumption Heat
bull FPGA ndash optimized circuit for the app = high performancepower efficiency
2017922 ROSCon2017Vancouver 3
[3] ldquoModel Zoordquo httpdlcaffeberkeleyvisionorgbvlc_googlenetcaffemodel
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 4
Evaluation of CPU GPU FPGA
Model of Deep Neural Networks VGG16
Dataset Cifar 10
Device
NVIDIA Jetson TX1Xilinx ZCU102
(FPGA)
Quad-core
ARM Cortex-A57
256-core
Maxwell GPU
Zynq UltraScale+
MPSoC
Clock Freq 19 GHz 998MHz 100MHz
Efficiency [FPSW] 0032 0376 143
Accuracy [] 9235 9030
H Yonekawa and H Nakahara ldquoOn-Chip Memory Based Binarized Convolutional Deep Neural Network Applying Batch Normalization Free Technique on an
FPGArdquo Parallel and Distributed Processing Symposium Workshops (IPDPSW) 2017 IEEE International 2017
38 times efficient
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Recent trends on FPGA
bull Application bull Image Recognition (local feature DNN)
bull Security (encryption)
bull Big Data (compression mining)
bull Devicebull Single Chip - ARM + FPGAbull Mar 2011 Xilinx Zynq
bull Oct 2011 ALTERA SoC FPGA
bull 2016- (announced) Intel Broadwell Xeon + FPGA (ALTERA Arria 10)
bull Toolbull High-Level Synthesis (C to HDL)
bull Servicebull Apr 2017 Amazon EC2 F1 instancebull FPGA Xilinx Virtex UltraScale+ VU9P(2586150 Logic Cells)
2017922 ROSCon2017Vancouver 5
Intels $167B Altera Buyout
(June 2 2015)
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Problem for introducing FPGA
bull ldquoDifficult design of FPGArdquo
bull Why FPGA is difficult to develop
bull Language HDL (Hardware Description Language) or HLS (High-level Synthesis)
bull Need of performance tuning techniques (processing and memory access FPGA expertise)
bull Long compile time
bull Component technology is necessary
bull In order to usereuse high-performance FPGA design easily from software
bull Letrsquos make ROS-node as FPGA-component
2017922 ROSCon2017Vancouver 6
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Proposed by Utsunomiya Univ
Mar 2015
Proposed by Utsunomiya Univ
Mar 2017Proposed by Kyutech
Mar 2015
Our proposal FPGA component technologies for ROS system (3 types)
bull (2) ROS-compliant
FPGA component on
Programmable SoC
bull ARM ROS on Linux
bull FPGA Application logic
2017922 ROSCon2017Vancouver
Application ROS node
input output
Publisher Subscriber SubscriberPublisherPS(ARM)
PL(FPGA)
ROS-Compliant FPGA Component(SoC)
interfacefor FPGA
interfacefor FPGA
bull (3) Fully-hardwired ROS-compliant FPGA component
bull Talks ROS protocol by using TCPIP stack
bull Direct connection to FPGA Application logic
Application
ROS node
input output
Publisher SubscriberFPGA
SubscriberHW
PublisherHW
Hardware ROS-Compliant FPGA Component
bull (1) COMTA (PC and
Programmable SoC)
bull PC ROS on Linux
bull ARM bridge (TCPIP)
bull FPGA Application logic
7
Programmable SoC System-on-chip ARM+FPGA ie Xilinx Zynq-7000 Intel (ALTERA) SoC FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 8
(1)COMTA Experiment Following human by using image processing
Processing results are given by custom ROS topic
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 2
Problem of computer vision on robots
times 5 Speed
Over 1 min to recognize just six objects
RoboCup 2016
Domestic Pre-Competition
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
FPGA accelerates DNN
bull ldquo1 min is too long for the taskrdquo
bull GoogLeNet[3] (22 layers)
bull 4 seconds for an inference
bull CPU Core i5-5200U 22GHz
bull Need of accelerator
bull GPU ndash Power consumption Heat
bull FPGA ndash optimized circuit for the app = high performancepower efficiency
2017922 ROSCon2017Vancouver 3
[3] ldquoModel Zoordquo httpdlcaffeberkeleyvisionorgbvlc_googlenetcaffemodel
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 4
Evaluation of CPU GPU FPGA
Model of Deep Neural Networks VGG16
Dataset Cifar 10
Device
NVIDIA Jetson TX1Xilinx ZCU102
(FPGA)
Quad-core
ARM Cortex-A57
256-core
Maxwell GPU
Zynq UltraScale+
MPSoC
Clock Freq 19 GHz 998MHz 100MHz
Efficiency [FPSW] 0032 0376 143
Accuracy [] 9235 9030
H Yonekawa and H Nakahara ldquoOn-Chip Memory Based Binarized Convolutional Deep Neural Network Applying Batch Normalization Free Technique on an
FPGArdquo Parallel and Distributed Processing Symposium Workshops (IPDPSW) 2017 IEEE International 2017
38 times efficient
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Recent trends on FPGA
bull Application bull Image Recognition (local feature DNN)
bull Security (encryption)
bull Big Data (compression mining)
bull Devicebull Single Chip - ARM + FPGAbull Mar 2011 Xilinx Zynq
bull Oct 2011 ALTERA SoC FPGA
bull 2016- (announced) Intel Broadwell Xeon + FPGA (ALTERA Arria 10)
bull Toolbull High-Level Synthesis (C to HDL)
bull Servicebull Apr 2017 Amazon EC2 F1 instancebull FPGA Xilinx Virtex UltraScale+ VU9P(2586150 Logic Cells)
2017922 ROSCon2017Vancouver 5
Intels $167B Altera Buyout
(June 2 2015)
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Problem for introducing FPGA
bull ldquoDifficult design of FPGArdquo
bull Why FPGA is difficult to develop
bull Language HDL (Hardware Description Language) or HLS (High-level Synthesis)
bull Need of performance tuning techniques (processing and memory access FPGA expertise)
bull Long compile time
bull Component technology is necessary
bull In order to usereuse high-performance FPGA design easily from software
bull Letrsquos make ROS-node as FPGA-component
2017922 ROSCon2017Vancouver 6
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Proposed by Utsunomiya Univ
Mar 2015
Proposed by Utsunomiya Univ
Mar 2017Proposed by Kyutech
Mar 2015
Our proposal FPGA component technologies for ROS system (3 types)
bull (2) ROS-compliant
FPGA component on
Programmable SoC
bull ARM ROS on Linux
bull FPGA Application logic
2017922 ROSCon2017Vancouver
Application ROS node
input output
Publisher Subscriber SubscriberPublisherPS(ARM)
PL(FPGA)
ROS-Compliant FPGA Component(SoC)
interfacefor FPGA
interfacefor FPGA
bull (3) Fully-hardwired ROS-compliant FPGA component
bull Talks ROS protocol by using TCPIP stack
bull Direct connection to FPGA Application logic
Application
ROS node
input output
Publisher SubscriberFPGA
SubscriberHW
PublisherHW
Hardware ROS-Compliant FPGA Component
bull (1) COMTA (PC and
Programmable SoC)
bull PC ROS on Linux
bull ARM bridge (TCPIP)
bull FPGA Application logic
7
Programmable SoC System-on-chip ARM+FPGA ie Xilinx Zynq-7000 Intel (ALTERA) SoC FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 8
(1)COMTA Experiment Following human by using image processing
Processing results are given by custom ROS topic
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
FPGA accelerates DNN
bull ldquo1 min is too long for the taskrdquo
bull GoogLeNet[3] (22 layers)
bull 4 seconds for an inference
bull CPU Core i5-5200U 22GHz
bull Need of accelerator
bull GPU ndash Power consumption Heat
bull FPGA ndash optimized circuit for the app = high performancepower efficiency
2017922 ROSCon2017Vancouver 3
[3] ldquoModel Zoordquo httpdlcaffeberkeleyvisionorgbvlc_googlenetcaffemodel
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 4
Evaluation of CPU GPU FPGA
Model of Deep Neural Networks VGG16
Dataset Cifar 10
Device
NVIDIA Jetson TX1Xilinx ZCU102
(FPGA)
Quad-core
ARM Cortex-A57
256-core
Maxwell GPU
Zynq UltraScale+
MPSoC
Clock Freq 19 GHz 998MHz 100MHz
Efficiency [FPSW] 0032 0376 143
Accuracy [] 9235 9030
H Yonekawa and H Nakahara ldquoOn-Chip Memory Based Binarized Convolutional Deep Neural Network Applying Batch Normalization Free Technique on an
FPGArdquo Parallel and Distributed Processing Symposium Workshops (IPDPSW) 2017 IEEE International 2017
38 times efficient
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Recent trends on FPGA
bull Application bull Image Recognition (local feature DNN)
bull Security (encryption)
bull Big Data (compression mining)
bull Devicebull Single Chip - ARM + FPGAbull Mar 2011 Xilinx Zynq
bull Oct 2011 ALTERA SoC FPGA
bull 2016- (announced) Intel Broadwell Xeon + FPGA (ALTERA Arria 10)
bull Toolbull High-Level Synthesis (C to HDL)
bull Servicebull Apr 2017 Amazon EC2 F1 instancebull FPGA Xilinx Virtex UltraScale+ VU9P(2586150 Logic Cells)
2017922 ROSCon2017Vancouver 5
Intels $167B Altera Buyout
(June 2 2015)
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Problem for introducing FPGA
bull ldquoDifficult design of FPGArdquo
bull Why FPGA is difficult to develop
bull Language HDL (Hardware Description Language) or HLS (High-level Synthesis)
bull Need of performance tuning techniques (processing and memory access FPGA expertise)
bull Long compile time
bull Component technology is necessary
bull In order to usereuse high-performance FPGA design easily from software
bull Letrsquos make ROS-node as FPGA-component
2017922 ROSCon2017Vancouver 6
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Proposed by Utsunomiya Univ
Mar 2015
Proposed by Utsunomiya Univ
Mar 2017Proposed by Kyutech
Mar 2015
Our proposal FPGA component technologies for ROS system (3 types)
bull (2) ROS-compliant
FPGA component on
Programmable SoC
bull ARM ROS on Linux
bull FPGA Application logic
2017922 ROSCon2017Vancouver
Application ROS node
input output
Publisher Subscriber SubscriberPublisherPS(ARM)
PL(FPGA)
ROS-Compliant FPGA Component(SoC)
interfacefor FPGA
interfacefor FPGA
bull (3) Fully-hardwired ROS-compliant FPGA component
bull Talks ROS protocol by using TCPIP stack
bull Direct connection to FPGA Application logic
Application
ROS node
input output
Publisher SubscriberFPGA
SubscriberHW
PublisherHW
Hardware ROS-Compliant FPGA Component
bull (1) COMTA (PC and
Programmable SoC)
bull PC ROS on Linux
bull ARM bridge (TCPIP)
bull FPGA Application logic
7
Programmable SoC System-on-chip ARM+FPGA ie Xilinx Zynq-7000 Intel (ALTERA) SoC FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 8
(1)COMTA Experiment Following human by using image processing
Processing results are given by custom ROS topic
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 4
Evaluation of CPU GPU FPGA
Model of Deep Neural Networks VGG16
Dataset Cifar 10
Device
NVIDIA Jetson TX1Xilinx ZCU102
(FPGA)
Quad-core
ARM Cortex-A57
256-core
Maxwell GPU
Zynq UltraScale+
MPSoC
Clock Freq 19 GHz 998MHz 100MHz
Efficiency [FPSW] 0032 0376 143
Accuracy [] 9235 9030
H Yonekawa and H Nakahara ldquoOn-Chip Memory Based Binarized Convolutional Deep Neural Network Applying Batch Normalization Free Technique on an
FPGArdquo Parallel and Distributed Processing Symposium Workshops (IPDPSW) 2017 IEEE International 2017
38 times efficient
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Recent trends on FPGA
bull Application bull Image Recognition (local feature DNN)
bull Security (encryption)
bull Big Data (compression mining)
bull Devicebull Single Chip - ARM + FPGAbull Mar 2011 Xilinx Zynq
bull Oct 2011 ALTERA SoC FPGA
bull 2016- (announced) Intel Broadwell Xeon + FPGA (ALTERA Arria 10)
bull Toolbull High-Level Synthesis (C to HDL)
bull Servicebull Apr 2017 Amazon EC2 F1 instancebull FPGA Xilinx Virtex UltraScale+ VU9P(2586150 Logic Cells)
2017922 ROSCon2017Vancouver 5
Intels $167B Altera Buyout
(June 2 2015)
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Problem for introducing FPGA
bull ldquoDifficult design of FPGArdquo
bull Why FPGA is difficult to develop
bull Language HDL (Hardware Description Language) or HLS (High-level Synthesis)
bull Need of performance tuning techniques (processing and memory access FPGA expertise)
bull Long compile time
bull Component technology is necessary
bull In order to usereuse high-performance FPGA design easily from software
bull Letrsquos make ROS-node as FPGA-component
2017922 ROSCon2017Vancouver 6
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Proposed by Utsunomiya Univ
Mar 2015
Proposed by Utsunomiya Univ
Mar 2017Proposed by Kyutech
Mar 2015
Our proposal FPGA component technologies for ROS system (3 types)
bull (2) ROS-compliant
FPGA component on
Programmable SoC
bull ARM ROS on Linux
bull FPGA Application logic
2017922 ROSCon2017Vancouver
Application ROS node
input output
Publisher Subscriber SubscriberPublisherPS(ARM)
PL(FPGA)
ROS-Compliant FPGA Component(SoC)
interfacefor FPGA
interfacefor FPGA
bull (3) Fully-hardwired ROS-compliant FPGA component
bull Talks ROS protocol by using TCPIP stack
bull Direct connection to FPGA Application logic
Application
ROS node
input output
Publisher SubscriberFPGA
SubscriberHW
PublisherHW
Hardware ROS-Compliant FPGA Component
bull (1) COMTA (PC and
Programmable SoC)
bull PC ROS on Linux
bull ARM bridge (TCPIP)
bull FPGA Application logic
7
Programmable SoC System-on-chip ARM+FPGA ie Xilinx Zynq-7000 Intel (ALTERA) SoC FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 8
(1)COMTA Experiment Following human by using image processing
Processing results are given by custom ROS topic
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Recent trends on FPGA
bull Application bull Image Recognition (local feature DNN)
bull Security (encryption)
bull Big Data (compression mining)
bull Devicebull Single Chip - ARM + FPGAbull Mar 2011 Xilinx Zynq
bull Oct 2011 ALTERA SoC FPGA
bull 2016- (announced) Intel Broadwell Xeon + FPGA (ALTERA Arria 10)
bull Toolbull High-Level Synthesis (C to HDL)
bull Servicebull Apr 2017 Amazon EC2 F1 instancebull FPGA Xilinx Virtex UltraScale+ VU9P(2586150 Logic Cells)
2017922 ROSCon2017Vancouver 5
Intels $167B Altera Buyout
(June 2 2015)
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Problem for introducing FPGA
bull ldquoDifficult design of FPGArdquo
bull Why FPGA is difficult to develop
bull Language HDL (Hardware Description Language) or HLS (High-level Synthesis)
bull Need of performance tuning techniques (processing and memory access FPGA expertise)
bull Long compile time
bull Component technology is necessary
bull In order to usereuse high-performance FPGA design easily from software
bull Letrsquos make ROS-node as FPGA-component
2017922 ROSCon2017Vancouver 6
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Proposed by Utsunomiya Univ
Mar 2015
Proposed by Utsunomiya Univ
Mar 2017Proposed by Kyutech
Mar 2015
Our proposal FPGA component technologies for ROS system (3 types)
bull (2) ROS-compliant
FPGA component on
Programmable SoC
bull ARM ROS on Linux
bull FPGA Application logic
2017922 ROSCon2017Vancouver
Application ROS node
input output
Publisher Subscriber SubscriberPublisherPS(ARM)
PL(FPGA)
ROS-Compliant FPGA Component(SoC)
interfacefor FPGA
interfacefor FPGA
bull (3) Fully-hardwired ROS-compliant FPGA component
bull Talks ROS protocol by using TCPIP stack
bull Direct connection to FPGA Application logic
Application
ROS node
input output
Publisher SubscriberFPGA
SubscriberHW
PublisherHW
Hardware ROS-Compliant FPGA Component
bull (1) COMTA (PC and
Programmable SoC)
bull PC ROS on Linux
bull ARM bridge (TCPIP)
bull FPGA Application logic
7
Programmable SoC System-on-chip ARM+FPGA ie Xilinx Zynq-7000 Intel (ALTERA) SoC FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 8
(1)COMTA Experiment Following human by using image processing
Processing results are given by custom ROS topic
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Problem for introducing FPGA
bull ldquoDifficult design of FPGArdquo
bull Why FPGA is difficult to develop
bull Language HDL (Hardware Description Language) or HLS (High-level Synthesis)
bull Need of performance tuning techniques (processing and memory access FPGA expertise)
bull Long compile time
bull Component technology is necessary
bull In order to usereuse high-performance FPGA design easily from software
bull Letrsquos make ROS-node as FPGA-component
2017922 ROSCon2017Vancouver 6
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Proposed by Utsunomiya Univ
Mar 2015
Proposed by Utsunomiya Univ
Mar 2017Proposed by Kyutech
Mar 2015
Our proposal FPGA component technologies for ROS system (3 types)
bull (2) ROS-compliant
FPGA component on
Programmable SoC
bull ARM ROS on Linux
bull FPGA Application logic
2017922 ROSCon2017Vancouver
Application ROS node
input output
Publisher Subscriber SubscriberPublisherPS(ARM)
PL(FPGA)
ROS-Compliant FPGA Component(SoC)
interfacefor FPGA
interfacefor FPGA
bull (3) Fully-hardwired ROS-compliant FPGA component
bull Talks ROS protocol by using TCPIP stack
bull Direct connection to FPGA Application logic
Application
ROS node
input output
Publisher SubscriberFPGA
SubscriberHW
PublisherHW
Hardware ROS-Compliant FPGA Component
bull (1) COMTA (PC and
Programmable SoC)
bull PC ROS on Linux
bull ARM bridge (TCPIP)
bull FPGA Application logic
7
Programmable SoC System-on-chip ARM+FPGA ie Xilinx Zynq-7000 Intel (ALTERA) SoC FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 8
(1)COMTA Experiment Following human by using image processing
Processing results are given by custom ROS topic
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Proposed by Utsunomiya Univ
Mar 2015
Proposed by Utsunomiya Univ
Mar 2017Proposed by Kyutech
Mar 2015
Our proposal FPGA component technologies for ROS system (3 types)
bull (2) ROS-compliant
FPGA component on
Programmable SoC
bull ARM ROS on Linux
bull FPGA Application logic
2017922 ROSCon2017Vancouver
Application ROS node
input output
Publisher Subscriber SubscriberPublisherPS(ARM)
PL(FPGA)
ROS-Compliant FPGA Component(SoC)
interfacefor FPGA
interfacefor FPGA
bull (3) Fully-hardwired ROS-compliant FPGA component
bull Talks ROS protocol by using TCPIP stack
bull Direct connection to FPGA Application logic
Application
ROS node
input output
Publisher SubscriberFPGA
SubscriberHW
PublisherHW
Hardware ROS-Compliant FPGA Component
bull (1) COMTA (PC and
Programmable SoC)
bull PC ROS on Linux
bull ARM bridge (TCPIP)
bull FPGA Application logic
7
Programmable SoC System-on-chip ARM+FPGA ie Xilinx Zynq-7000 Intel (ALTERA) SoC FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 8
(1)COMTA Experiment Following human by using image processing
Processing results are given by custom ROS topic
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 8
(1)COMTA Experiment Following human by using image processing
Processing results are given by custom ROS topic
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
2017922 ROSCon2017Vancouver 9
(1)COMTA Experiment Results
Device
Conv Prop
Core i5-
5200U
22GHz
ARM Dual Core
667MHz + FPGA
Frame Rate
[FPS]2906 1725
Power
[W]26 47
Efficiency
[FPSW]112 369
33 times efficient
Status Idling
Computing load of Conv method
50 points increase
Status Following human
30
80Heavy task can be
offloaded onto FPGA
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
cReComp automatic generation of ROS component form HDL
2017922 ROSCon2017Vancouver
bull Target
bull Programmable SoC (Xilinx Zynq)
bull Input
bull Application Logic HDL
bull Configuration Fie
bull scrp (Text)
Python (using PyVerilog)
bull Output
bull HDL FIFO buffer control
bull C++ ROS node
HW IFSW IF
ROS node
v
ROS-Compliant FPGA component
cpp
Data comm
Hardware
config
User
logic
generate
Software
cReComp
FIFO
FIFO
scrp or py
v
Experimental (6 undergraduategraduates)result
finished componentization within an hour (42 minutes maximum)
10
Zedboard
ZYBO
Shared at httpsgithubcomKumikomicReComp
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(2) ROS-compliant FPGA component on Programmable SoC
Open-Source Packages Available in ROSorg
2017922 ROSCon2017Vancouver 11
Shared at
httpsgithubcomKumikomiopenreroc_motion_sensor
httpsgithubcomKumikomiopenreroc_pwm
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
DEMO MOVIE FPGA-ROS node of FAST key-point detection
2017922 ROSCon2017Vancouver 12
Input
Image
FAST key-points
by PC
FAST key-points
by FPGA
ROS node
on FPGA board
USB camera
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- structure of the evaluation system
bull FPGA is accessed via Gigabit Ethernet
bull Image processing circuitbull HLS (Xilinx VivadoHLS) from C++ (OpenCV)
bull SubscriberPublisher HW (HDL)bull Implemented using TCPIP stack (SiTCP)
2017922 ROSCon2017Vancouver 13
Sub HWIP1921681017
Pub HWIP1921681018
output_data
Image
processing
circuit
(AXI-Stream)
image
_view
Fast_PCimage
_viewoutput_pc
uvc_
camera_
node
image_
view
pre_pr
ocess_
node
imageraw input_data
FPGA
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
Source code of Image Processing Circuit ndash for Xilinx Vivado HLS
2017922 ROSCon2017Vancouver 14
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
How does the FPGA ROS-node (Hardware PubSub) work
bull ROS wire protocol (TCPIP packets) ie PublishSubscribe messagesbull Anyone can be a ROS node who talks TCPIP in ROS manner
bull It is not the matter if it is Software Hardware (FPGA) or something
2017922 ROSCon2017Vancouver 15
Letrsquos hack the ROS wire protocol
Software Hardware (FPGA) something
Node
Publication SubscriptionTopic
msg
Node Node
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP① and STEP② is communication with master
bull master is a name server in ROS system
bull XMLRPC remote procedure calling using HTTP as the
transport and XML as the encoding
Procedure of PublishSubscribe messaging in ROS system (13)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master XMLRPC
16
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull STEP③ Subscriber sends a connection request
to Publisher by using requestTopic (XMLRPC)
bull STEP④ Publisher returns host address and
port number of TCPROS
Procedure of PublishSubscribe messaging in ROS system (23)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234) ①subscribe
(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
XMLRPC
17
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data transmission starts in TCPROS protocol
bull STEP⑤ Subscriber establishes a TCP connection
bull STEP⑥ Data transmission repeats
Procedure of PublishSubscribe messaging in ROS system (33)
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345
⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
18
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Binary data efficient data transferbull Easy to handle in hardware
bull Example bull ROS message = int32 x2
TCPROS protocol
2017922 ROSCon2017Vancouver
52 54 00 60 81 a8 52 54 00 26 a6 ae 08 00 45 00
00 40 fb 14 40 00 40 06 c9 57 c0 a8 7a 73 c0 a8
7a 87 8a 81 df 36 8f de ee 08 61 a8 82 70 80 18
00 eb 76 7e 00 00 01 01 08 0a 00 03 19 ba 00 02
de 55 08 00 00 00 04 00 00 00 05 00 00 00
int32 a
int32 b
ROS Message
Data message size(int32) +int32+int32
TCPIP packet (captured by WireShark)
19
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Publisher HW can be implemented with 1 TCPIP port
(3) Fully-hardwired ROS-compliant FPGA component
- Separation for Hardwired Publisher Node
2017922 ROSCon2017Vancouver
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
PublisherSW on PC
PublisherHW on FPGA
Subscriber
master
Publisher SW
topic name ldquobarrdquo
hostname foo
XMLRPC port number 1234
Publisher HW
hostname FPGA
TCPROS port number 3456
Subscriber
subscribe to ldquobarrdquo ③requestTopic(ldquobarrdquoTCP)
④ FPGA3456
XMLRPC
TCPROS
Publisher
①advertise(ldquobarrdquo foo1234)
①subscribe(ldquobarrdquo)
②foo1234
Publisher Subscriber
master
③requestTopic(ldquobarrdquoTCP)
④ foo2345⑤ connect(foo2345)
⑥ data transmission
XMLRPC
TCPROS
20
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull Measurement communication latency
bull FPGA component does not process data (relay only)
bull Used ROS message type ldquosensor_msgsimagerdquo
bull Comparison PCARMFPGA
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation system
2017922 ROSCon2017Vancouver
input
input_data output_data
outputPCARM
FPGAComp
Publish Subscribe Publish Subscribe
msg
latency
21
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
bull FPGA Xilinx Spartan-6 XC6SLX110bull Board exTri-CSI (e-treesJapan Inc)
bull Two SiTCP instances for Gigabit Ethernet ports
(3) Fully-hardwired ROS-compliant FPGA component
- Amount of hardware resource
2017922 ROSCon2017Vancouver
RESOURCE UTILIZATION
Slice Registers 9253126576 ( 7)
Slice LUTs 742163288 (11)
RAMB16BWER
s54268 (20)
httpe-treesjpproductsextri-csi
22
FPGA does not process data (relay only)
TUchida ldquoHardware-Based TCP Processor for Gigabit Ethernet IEEE Transactions on Nuclear
ScienceVol55 NoSIG 3 pp1631-1637 2008
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
(3) Fully-hardwired ROS-compliant FPGA component
- Evaluation results Image data transfer (sensor_msgimage)
2017922 ROSCon2017Vancouver
2548
129
55
101
290
3463
175
0
50
100
150
200
250
300
350
1M 2M 6M
tim
e(m
s)
data size(byte)
PC-PC PC-ARM PC-FPGA
Our implementation is faster
than software on ARM processor
bull Throughput 550Mbpsbull 58 of maximum performance (SiTCPrsquos MAX 949Mbps)
23
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
Summary amp Future work
bull Proposal Three types of ROS-compliant FPGA components
bull Robot applications of the components
bull Object recognition using DNN architecture
bull SLAM Distributed Visual-SLAM processing
bull Robot Image processing (ORB feature) on FPGA
bull Cloud SLAM processing at highly-parallel environment
bull Challenges at RoboCupHome2018
bull Movie RoboCupHome2017
1st place Winner at domestic standard
platform league (TOYOTA HSR)
2017922 ROSCon2017Vancouver 24
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
THANK YOU
2017922 ROSCon2017Vancouver 25
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References
bull POINTERS to Packagesbull OpenReroc (Open source Reconfigurable Robot Component) project which aims to
promote the concept of ROS-compliant FPGA componentbull httpkumikomigithubioOpenReroc
bull Example implementation of Xilinx Zynq (ARM+FPGA) for Indigo packages at ros wikibull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_pwm
bull httpwwwrosorgbrowsedetailsphpdistro=indigoampname=openreroc_motion_sensor
bull URLsbull Home service robot project team ldquoHibikino-Musashirdquo for RoboCupHome
httpwwwbrainkyutechacjp~hmawordpressabout-us
bull RoboCupHome Movie httpswwwyoutubecomchannelUCJEeZZiDXijz6PidLiOtvwQ
bull RoboCupHome Website httpwwwrobocupathomeorg
bull Zynq-7000 All Programmable SoC Xilinx httpwwwxilinxcomproductssilicon-devicessoczynq-7000html
2017922 ROSCon2017Vancouver 26
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27
PEAR-LAB Utsunomiya Univ Tamukoh Lab Kyutech
References (22)
bull [1] Takeshi Ohkawa Kazushi Yamashina Takuya Matsumoto Kanemitsu Ootsu Takashi YokotaArchitecture Exploration of
Robot System using ROS-Compliant FPGA ComponentProc 27th IEEE International Symposium on Rapid System Prototyping
(RSP) pp72-78 Oct 2016 DOI httpdxdoiorg10114529902992990312
bull [2] Yuhei Sugata Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota Acceleration of publishsubscribe messaging in
ROS-Compliant FPGA Component The 8th International Symposium on Highly Efficient Accelerators and Reconfigurable
Technologies (HEART2017) 7-9 June 2017 (PDF is not available yet)
httpwwwruhr-uni-bochumdeheart2017programhtml
bull [3] Kazushi Yamashina Takeshi Ohkawa Kanemitsu Ootsu and Takashi Yokota ldquoProposal of ROS-compliant FPGA Component
for Low-Power Robotic Systems - case study on image processing applicationrdquo Proceedings of 2nd International Workshop on
FPGAs for Software Programmers FSP2015 pp 62-67 2015 httpsarxivorgftparxivpapers1508150807123pdf
bull [4] Kazushi Yamashina Hitomi Kimura Takeshi Ohkawa Kanemitsu Ootsu Takashi Yokota ldquocReComp Automated Design Tool
for ROS-Compliant FPGA Componentrdquo In proc IEEE 10th International Symposium on Embedded MulticoreMany-core
Systems-on-Chip (MCSoC-16) 2016
bull [5] A Suzuki T Morie H Tamukoh ldquoFPGA Implementation of Autoencoders Having Shared Synapse Architecturerdquo Proc of the
23rd Int Conf on Neural Information Processing (ICONIP2016) Lecture Notes in Computer Science Vol 9947 pp231-239 2016
bull [6] S Yokota J Li Y Ogishima H Kubo H Tamukoh and M Sekine ldquoSelf-Learning of Feature Regions for Image Recognitionrdquo
Journal of Computer Sciences and Applications Vol3 No1 pp1-10 2015
bull [7] Yutaro Ishida et al ldquoApproach to accelerate the development of practical home service robots - RoboCup Home DSPLrdquo
The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) 2017
2017922 ROSCon2017Vancouver 27