Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment...
-
Upload
cory-creek -
Category
Documents
-
view
215 -
download
1
Transcript of Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment...
![Page 1: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/1.jpg)
Progress report
2012/05/01 匡建慈
![Page 2: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/2.jpg)
Reminding
פ Goals : to build a multi-core platform with Hadoop environment
Base board & FPGA
& fiber ports
Tegra2 Application layer
Transaction layer
Physical Layer
Linux Device Driver
DMA in Tegra2Router circuit in FPGAFiber link
User programUser API
![Page 3: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/3.jpg)
![Page 4: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/4.jpg)
What to do and what we have
פ What we have פ Tegra2 software program co-work with FPGA to do simple work .פ A circuit example with fiber communication
פ What to do פ a example by combining examples we have to achieve a
framework from software , hardware , and fibers . ( O )פ Example interface unified פ Achieve API package פ Interrupt mechanism
פ Porting Hadoop environment on proposed framework . ( X )
![Page 5: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/5.jpg)
פ Combined example block diagram
DMA
TX_Buf
RX_Buf
TX_IF
RX_IF
GTPBASE
Fiber link
Linux Devic
e Driver
User Prog.
Circuit layer SoftwareDriver
DMAIF
Clk_wiz
clk_120mBASE_CLK
![Page 6: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/6.jpg)
פ API packageפ Global variable : device number , buffer pointerפ init : open device , memory mappingפ Finalize : close device , memory um-mapping .פ FPGAWrite : write buf_w into circuit layer by DMAפ FPGARead : Read data from circuit by DMA and store into buf_r
פ header packet mechanism פ In future , it wiil be applied on
APIs
![Page 7: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/7.jpg)
פ Interrupt mechanismפ Circuit layer :
פ pull up MEM_R/W_Ready to setup an interrupt to CPUפ A interrupt means it’s ready for the next user read/write call
פ Driver layer פ Driver code in original release might be got stuck due to mutex
usage .פ According to previous work on MultiARM , rewrite driver by a
kthread interrupt handler and two wait_events
25 ns
![Page 8: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/8.jpg)
Result
![Page 9: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/9.jpg)
2-node test
פ A single SendRecv program running on 2 nodesפ Add a ‘!’ until the length is larger than specified value
Receiver
Sender
![Page 10: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/10.jpg)
Next
פ Try to setup Hadoop environment on proposed framework .פ Do on my laptopפ Setup JAVA on MaCube פ Setup Hadoop environment on MaCubeפ Modify the calling interface to my APIs
![Page 11: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/11.jpg)
Progress report
2012/05/15 boy13
![Page 12: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/12.jpg)
Last week
פ Completedפ Setup programming model and achieve framework covered with
software , hardware , GTPs .פ Combining examples פ API package פ Interrupt mechanism
פ To doפ Setup Hadoop on proposed framework
פ Setting Hadoop on this laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube Xפ Modify calling interface by my APIs X
![Page 13: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/13.jpg)
Setting Hadoop on my laptop
פ Single node settingפ Download Hadoop and jdk and extract them .פ <Hadoop Path>/conf/hadoop_env.sh
פ Export JAVA_HOME=<JdkPath>
פ <Hadoop Path>/conf/core-site.xmlפ hadoop.tmp.dir <a path to put /tmp>פ fs.defailt.name hdfs://localhost:9000פ Dfs.replication 1
פ <Hadoop Path>/conf/mapred-site.xmlפ Mapred.job.tracker localhost:9001
פ More on http://changyy.pixnet.net/blog/post/25245658
![Page 14: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/14.jpg)
פ Hadoop example executionפ HDFS format
פ <Hadoop Path>/bin/hadoop namenode -format
פ Start and stopפ <Hadoop Path>/bin/start-all.shפ <Hadoop Path>/bin/stop-all.sh
פ HDFS operation פ <Hadoop Path>/bin/hadoop dfs –mkdir inputפ <Hadoop Path>/bin/hadoop dfs –lsפ <Hadoop Path>/bin/hadoop dfs –put <filepath> input
![Page 15: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/15.jpg)
פ Hadoop example wordcount executionפ <Hadoop Path>/bin/hadoop jar hadoop-0.20.1-examples.jar
wordcount input output
פ See execution resultפ <Hadoop Path>/bin/hadoop dfs –cat output/*
![Page 16: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/16.jpg)
פ Multi-node settingפ Hosts
פ /etc/hosts : A table record machine name and its IPפ /etc/hostname : machine name
פ ssh connections פ sudo apt-get install openssh-server
פ <Hadoop Path>/confפ slaves : machine name of data-nodesפ masters : machine name of name-node
פ start-all.sh
![Page 17: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/17.jpg)
פ After start-all.shפ <Jdk Path>/bin/jpsפ <Hadoop Path>/bin/hadoop dfsadmin –report
פ More on : http://changyy.pixnet.net/blog/post/25612440
![Page 18: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/18.jpg)
Setting JAVA Runtime Environment
![Page 19: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/19.jpg)
פ cp /media/sda1/ejre1.6.0_32.tar.gz ./פ tar -xjf ejre1.6.0_32.tar.gzפ export PATH=/home/root/Desktop/ejre1.6.0_32/bin:$PATH
![Page 20: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/20.jpg)
Next to do
פ Try to setup Hadoop environment on proposed framework .פ Do on my laptopפ Setup JAVA on MaCube פ Setup Hadoop environment on MaCubeפ Modify the calling interface to my APIs
![Page 21: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/21.jpg)
Progress report
2010/05/29 boy13
![Page 22: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/22.jpg)
Recently
פ Goals : to build a multi-core system with hadoop environment
פ Work items פ Setup programming model and achieve framework covered with
software , hardware , GTPs .פ Combining examples Oפ API package Oפ Interrupt mechanism Oפ Router mechanism
פ Setup Hadoop on proposed frameworkפ Setup up on my laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube
![Page 23: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/23.jpg)
Router mechanism
פ A Light-Weight Parallel Router for FPGA-based Networks-on-Chip
פ Department of ECECS , University of Cincinnati, 2005
Application layer
Transaction layer
Physical Layer
Linux Device Driver
DMA in Tegra2Router circuit in FPGAFiber link
User programUser API
![Page 24: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/24.jpg)
Router mechanism
פ 2-D mesh architecture
פ Routing mechanism פ XY routing
פ Portפ Input Channel
פ FIFO interfaceפ Be requested from Output Channel of a port in another routerפ Request to X-bar switch to access Output Channel
פ Output Channel
פ Crossbar Switch
![Page 25: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/25.jpg)
![Page 26: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/26.jpg)
פ For out architecture פ 2-D mesh => 3-D meshפ Port => Local Port , Fiber Port
פ Local Port : interface for FPGA and DMA in tegra2 פ Fiber Port : interface for fiber link in proposed framework
Base board & FPGA
& fiber ports
Tegra2
![Page 27: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/27.jpg)
Setting hadoop in proposed frameworkפ Setting up namenode
פ applet not found
פ What is applet ?פ JRE v.s. Java Applet
פ By several weeks asking ,I conclude that JAVA Applet isn’t supported in embedded JAVA .
![Page 28: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/28.jpg)
פ Setting up datanode in MaCubeפ Laptop plays namenode role in this work .
פ Setting ssh connection פ Pre-requite : openssl , zlib
פ Download source package פ Extract it to local directoryפ ./configure –helpפ ./configure <options for cross compile>פ Modify makefile manually if necessary פ make && make install
![Page 29: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/29.jpg)
פ Error occur when I key make install for opensshפ Source pack : 5.7 , 5.9 פ Toolchain : arm-2009q3 , cc.2011.03
![Page 30: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/30.jpg)
Next
פ Router mechanismפ Complete design for Local port , Crossbar switch פ Change local port with fiber port פ Complete 2x2 2-D mesh architecture
פ Setup hadoop environment פ Build openssh and setup connectionפ Try datanode start service
![Page 31: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/31.jpg)
Progress report
2012/06/19 boy13
![Page 32: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/32.jpg)
Recently
פ Goals : to build a multi-core system without hadoop environment
פ Work items פ Setup programming model and achieve framework covered with
software , hardware , GTPs .פ Combining examples Oפ API package Oפ Interrupt mechanism Oפ Router mechanism Oפ Multi-core system
פ Setup Hadoop on proposed frameworkפ Setup up on my laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube Xפ Namenode software service
![Page 33: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/33.jpg)
Router mechanism
פ Router פ Input Channel , Output Channelפ Input Request mechanism , Round-Robin Arbitratorפ Parallel Router by cross-bar switch
פ Testing pattern פ Local port Oפ Local port + Router Oפ Local port + Router + 1 fiber port Oפ Local port + Router + 4 fiber port O
![Page 34: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/34.jpg)
DMA
Local port
Router
East port West port
GTP_J4
North portSouth port
GTP_J5
id
![Page 35: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/35.jpg)
Multi-core system
פ So far , only 1-1 send-recv pairs , and no packet overlapping occur .
פ Read mechanism in driver should be designed פ When read interrupt comes
פ configure DMA and put data into a bufferפ Wakeup kthread (interrupt handler)
פ Kthreadפ Put data in buffer in a list kept in driver
פ Read callפ Search list in driver פ If found , copy data into read buffer API allocatedפ Otherwise , sleep
פ Re-check circuit designפ Whether data not sent but be replaced or notפ Buffer empty register
![Page 36: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/36.jpg)
Setting hadoop environment
פ Hadoop settingפ Download hadoop source and JRE into MaCubeפ Modify <hadoop path>/conf/ * .xmlפ Build ssh connectionפ ./start-all.shפ ./bin/hadoop <execution commands>
פ 2 use-caseפ MaCube play both namenode and datanode rolesפ MaCube plays datanode role and laptop/PC act as namenode
![Page 37: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/37.jpg)
פ Problem encountered for both datanode and namenode roleפ bash : no such file or directory
פ What is bash ?פ ln –s /bin/sh /bin/bash
פ bash : applet not found
![Page 38: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/38.jpg)
פ By several weeks googling and asking , I conclude that JAVA Applet isn’t supported in embedded JAVA .
פ Taiwan Hadoop Forumפ http://hadoop.nchc.org.tw/phpbb/index.php?sid=021ae10a0e0609
ab818b3e7062aaa314
פ Embedded JAVA , no bash => no applet => hadoop can’t be built .
פ So ……
![Page 39: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/39.jpg)
Namenode software service
פ Motivation : hadoop is not friendly with embedded system .
פ We build a simple software service of cloud systemפ A simple service to submit job
פ Package source code , data , and execution argumentsפ cross-compile source codeפ Send package to datanode scheduled פ Result returned
פ Graphics user interface and it’s easy to use.פ Embedded JAVA !!פ New thread and system call to execute program received
פ Simple distributed file systemפ Connect with student of Prof. Shiao
![Page 40: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/40.jpg)
To do
פ achieve multi-core system hardware frameworkפ Complete read mechanismפ Put fiber port from 4 to 6
פ Design namenode software service פ A quick job submit flowפ Easy to use and graphical
![Page 41: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/41.jpg)
Progress report
2012/07/10 boy13
![Page 42: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/42.jpg)
Recently
פ Goals : to build a multi-core system
פ Work items פ Setup programming model and achieve framework covered with
software , hardware , GTPs .פ Combining examples Oפ API package Oפ Interrupt mechanism Oפ Router mechanism Oפ Multi-core system
פ Setup Hadoop on proposed frameworkפ Setup up on my laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube Xפ Namenode software service
פ Demo items
![Page 43: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/43.jpg)
פ Multi-core systemפ Transportation testing
פ X-axis transportation פ Y-axis transportation פ 2-D mesh transportation
0 1
2 3
0 1 2 3
![Page 44: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/44.jpg)
פ Multi-core systemפ Serial transportation
פ Motivation : sometimes a byte data shift
![Page 45: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/45.jpg)
פ Multi-core systemפ Serial transportation problem
פ Reference clock skewing
פ Solutions :פ Sink : make the initiator and target correct the timing when initial
the transactionפ Re-send : if data check is error by some mechanism , then request
initiator to resend
פ The author of GTP_EXAMPLE says , it’s locked when the first sink , and can’t be unlocked .
פ => only can try re-send method .
![Page 46: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/46.jpg)
פ Method thinking פ do something when error has checked at the first time .
פ Ack/Nak mechanism . פ Deadlock possible ?
TX_IF
RX_IF
GTPCORE
TX_IF
RX_IF
GTPCORE
![Page 47: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/47.jpg)
פ Namenode software serviceפ TCP communication
פ Try : a simple chat room program
פ Shell program
![Page 48: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/48.jpg)
Server
Initial socket
Listen client
Create thread to message passing
Create receive thread
Message recv
Take mutex and queue it
Check queue is empty
Take mutex , pop queue ,Socket send
Client
Initial socketGet
connection
Create thread to recv
chatSocket recv
![Page 49: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/49.jpg)
פ Demo items פ Demo item 1 :
פ Using Ethernet to do Map-Reduce or SPCP
פ Demo item 2 (possible) : פ Using Ethernet to transport program and data פ Using fiber to communication with components
1 2 3
4 5 6
7 8
![Page 50: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/50.jpg)
Progress report
2012/08/01
![Page 51: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/51.jpg)
Recently
פ Goals : to build a multi-core system
פ Work items פ Setup programming model and achieve framework covered with
software , hardware , GTPs .פ Combining examples Oפ API package Oפ Interrupt mechanism Oפ Router mechanism Oפ Multi-core system
פ Setup Hadoop on proposed frameworkפ Setup up on my laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube Xפ Namenode software service
פ Demo items
![Page 52: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/52.jpg)
Recently
פ Transport testing
פ Middle-ware implementation
פ Demo items implementation
![Page 53: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/53.jpg)
Transport testing
פ 2-D mesh architecture
פ X_testפ 0,1 OKפ 4,5 X -> OK
פ Y_testפ 0,4 X -> OKפ 1,5 X -> OKפ Synchronous problem occurred frequently .
0,0(0)
1,0(1)
1,0(4)
1,1(5)
![Page 54: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/54.jpg)
Middle-ware implementation
פ Shell program(JAVA) , TCP Agent(C++) , TCP Core(C++)פ connection built OK , but no data received
פ JAVA object serialization
פ It’s OK to ping Macube from Vmware , but it doesn’t work to ping Vmware from Macube
פ Double-system laptop
פ Viewing SPCP sources
פ => porting SPCP onto Macubeפ Cross-compile core program and takes related filesפ Possible problem : speed gap between PC and Macube
![Page 55: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/55.jpg)
Demo item implementation
פ Item 1 : porting SPCP
פ Item 2 : simple puzzle (possible)פ Ethernet version
פ Packet size , header configurationפ no puzzle change , 2 cores on PC , 4 cores on Macubesפ Speed gap
1 2 3
4 5
JAVA UI
Upload
System call and wait
Server
Core
Core
Core
Core
TCP/IP connection
![Page 56: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/56.jpg)
פ Item 2 : simple puzzle (possible)פ Fiber version
פ Serial transport synchronous problemפ Re-send , Sync at transport initial
פ re-send mechanism
1 2 3
4 5
TX_sendRX get Ack
timeout
done
Re-send
RX_recvCRC_error =
1 discard
Send ack packet
TX_sendAck
RX_getAck
CRC_error = 1
discard
![Page 57: Progress report 2012/05/01. Reminding Goals : to build a multi-core platform with Hadoop environment Base board & FPGA & fiber ports Tegra2 Application.](https://reader035.fdocuments.us/reader035/viewer/2022062511/551763215503460e6e8b4aea/html5/thumbnails/57.jpg)
Thanks for your attention !