Port Scratch to pcDuino with Hardware Support

pcDuino hardware interfaces

Overview Scratch is a tile-based visual programming

environment and toolkit. It lets kids make games, animated stories, interactive art, and share with others on the Net.

In this poster, we explain the work needed to let Scratch have access to Arudino compatible hardware interface on pcDuino when Scratch itself runs on pcDuino.

Introduction Scratch is based on squeak ( a type of smalltalk)

virtual machine. Scratch provides a mechanism to call host API,

named plugin. Through plugin, Scratch can access devices, such as camera, sound, on host.

pcDuino already implemented Arduino style API (in C/C++), so the key is to pacakge pcDuino’s Arduino-ish API into a form of plugin.

UI

Scratch uses block to organize the programming elements. Currently, there are 8 blocks.

We add a new block ‘hardware’ to hold the new elements.

Hardware functions implemented A portion of pcDuino hardware interface is

implemented: GPIO 0-23: Set input/output mode, high/low output signal level,

Read Status PWM5-6: Set the output frequency, duty cycle, turn off PWM ADC0-5: Read the voltage level

Referring to the UI, the following items are implemented: set pin (0-23) to (INPUT/OUTPUT) mode set pin (0-23) to (HIGH/LOW) level pin (0-23) level is (HIGH/LOW)? Voltage(mV) of pin (A0-A5) set pwm (5/6) (781/520/390/260/195) Hz (0-256) step stop pwm (5/6)

Screen Shot

Add a new block “Hardware” Open system browser in squeak, find method

‘blockSpecs’ in class Scratch-Objects -> ScratchSpriteMorph -> block specs.

Add a new block “Hardware”The symbol after % is the input variable. It needs to be declared in Scratch-Blocks -> CommandBlockMorph -> private -> uncoloredArgMorphFor. We find a symbol that is not used, and add our own:

Parameters Enumeration in hardware block The format of blockSpec is as following:

(‘block text’#identifier#selector ‘default’ ‘values’) The one after % in ‘block test’ is the input parameters,

we need to implement the following: %A #pinModeNames INPUT/OUTPUT %B #pinLevelNames HIGH/LOW %E #analogPinNames A0-A5 %F #digitalPinNames 0-23 %G #pwmPinNames 5/6 %j #pwmPinFreqs

781/520/390/260/195 %J #pwmPinLevels0-256

Calling Method in Hardware Block

# selector in blockSpec is the corresponding calling method. We need to implement the following methods: pinMode:to: digitalWrite:to: digitalRead:is: analogReadVoltage: analogSet:freqTo:stepTo: analogStop:

All the above methods eventually need to be implemented through hardware plugin which is implemented in C/C++.

We need to define the interface to hardware plugin.

Calling Method in Hardware Block

Open system-Browser, and create HardwarePlugin->Primtives Class, and add the following methods:

primPinMode: pin to: mode "This is the call to the pinMode primitive."

 <primitive: 'primPinMode' module: 'HardwarePlugin'>self primitiveFailed

primDigitalRead: pin"This is the call to the digitalRead primitive."

 <primitive: 'primDigitalRead' module: 'HardwarePlugin'>self primitiveFailed

primDigitalWrite: pin to: level "This is the call to the digitalWrite primitive."

 <primitive: 'primDigitalWrite' module: 'HardwarePlugin'>self primitiveFailed

Calling Method in Hardware Block

primAnalogReadVoltage: pin"This is the call to the analogReadVoltage primitive."

 <primitive: 'primAnalogReadVoltage' module: 'HardwarePlugin'>self primitiveFailed

primAnalogSet: pin freqTo: freq stepTo: step"This is the call to the analogSet primitive."

 <primitive: 'primAnalogSet' module: 'HardwarePlugin'>self primitiveFailed

 primAnalogWrite: pin to: level

"This is the call to the analogWrite primitive." 

<primitive: 'primAnalogWrite' module: 'HardwarePlugin'>self primitiveFailed

Calling Method in Hardware Block

Use primPinMode as the example, it will eventually call function ‘primPinMode’in scratch HardwarePlugin.so to set the mode of GPIO pin as instructed in <primitive: 'primPinMode' module: 'HardwarePlugin'>

Calling Method in Hardware Block

We define the interface in Scratch-Objects -> ScratchSpriteMorph -> hardware ops instance:

pinMode: pin to: mode| p m |p := pin asNumber.Transcript show: p.mode = 'INPUT' ifTrue: [ m := 0 ].mode = 'OUTPUT' ifTrue: [ m := 1 ].HardwarePlugin primPinMode: p to: m.

digitalRead: pin is: level| p ret |p := pin asNumber.ret := HardwarePlugin primDigitalRead: p.level = 'HIGH' ifTrue: [ ^ ret ].^ ret not.

digitalWrite: pin to: level| p l |p := pin asNumber.Transcript show: p.level = 'LOW' ifTrue: [ l := 0 ].level = 'HIGH' ifTrue: [ l := 1 ].HardwarePlugin primDigitalWrite: p to: l.

Calling Method in Hardware Block

analogReadVoltage: pin| p |pin = 'A0' ifTrue: [ p := 0 ].pin = 'A1' ifTrue: [ p := 1 ].pin = 'A2' ifTrue: [ p := 2 ].pin = 'A3' ifTrue: [ p := 3 ].pin = 'A4' ifTrue: [ p := 4 ].pin = 'A5' ifTrue: [ p := 5 ].^ HardwarePlugin primAnalogReadVoltage: p.

analogSet: pin freqTo: freq stepTo: step| p f s |p := pin asNumber.f := freq asNumber.s := step asNumber.HardwarePlugin primAnalogSet: p freqTo: f stepTo: s.

analogStop: pin| p |p := pin asNumber.HardwarePlugin primAnalogWrite: p to: 0.

  

Implement HardwarePlugin We can refer to the code of camera. The key is to implement setInterpreter

and the functions primXXX used in Primitives-Plugins->HardwarePlugin->primtives.

The functions pinMode, digitalRead, digitalWrite, and analogWrite are already implemented in libarduino of pcDuino. We can call them directly.

Implement HardwarePluginDLLEXPORT void primPinMode(void) { int pin = interpreterProxy->stackIntegerValue(1); int mode = interpreterProxy->stackIntegerValue(0); pinMode(pin, mode);} DLLEXPORT void primDigitalWrite(void) { int pin = interpreterProxy->stackIntegerValue(1); int level = interpreterProxy->stackIntegerValue(0); digitalWrite(pin, level);} DLLEXPORT int primDigitalRead(void) { int pin = interpreterProxy->stackIntegerValue(0); int level = digitalRead(pin); interpreterProxy->pop(1); interpreterProxy->pushBool(level != 0); return 0;} 

Implement HardwarePluginDLLEXPORT int primAnalogReadVoltage(void) { int pin = interpreterProxy->stackIntegerValue(0); int level = analogReadVoltage(pin); interpreterProxy->pop(1); interpreterProxy->pushInteger(level); return 0;} DLLEXPORT void primAnalogWrite(void) { int pin = interpreterProxy->stackIntegerValue(1); int level = interpreterProxy->stackIntegerValue(0); analogWrite(pin, level);} DLLEXPORT void primAnalogSetFreq(void) { int pin = interpreterProxy->stackIntegerValue(1); int freq = interpreterProxy->stackIntegerValue(0); analogSetFreq(pin, freq);} DLLEXPORT void primAnalogSet(void) { int pin = interpreterProxy->stackIntegerValue(2); int freq = interpreterProxy->stackIntegerValue(1); int level = interpreterProxy->stackIntegerValue(0); analogSetFreq(pin, freq); analogWrite(pin, level);

Implement HardwarePlugin

DLLEXPORT int setInterpreter(struct VirtualMachine* anInterpreter) { int ok;  interpreterProxy = anInterpreter; ok = interpreterProxy->majorVersion() == VM_PROXY_MAJOR; if (ok == 0) { return 0; } ok = interpreterProxy->minorVersion() >= VM_PROXY_MINOR; return ok;}

Appendex

Programming under Ubuntu (linux)

Two flavors Command line IDE

Arduino style programming ( C )

C Command lineSetup (one time) If not already done, set up git. Do this using the command: ubuntu@ubuntu:~$ sudo apt-get install git  Make sure you’re in your home folder by typing ubuntu@ubuntu:~$ cdubuntu@ubuntu:~$ pwd/home/Ubuntu Now download the distribution from github by typingubuntu@ubuntu:~$ git clone https://github.com/pcduino/c_enviroment

C Command line

C Command lineChange into the c_enviroment folder: ubuntu@ubuntu:~$ cd c_enviromentubuntu@ubuntu:~/c_enviroment$ lsMakefile hardware libraries output sample Now run make to make the libraries and the examples with the following command: ubuntu@ubuntu:~/c_enviroment$ make

Make[1]: Leaving directory `/home/ubuntu/c_enviroment/sample'The resulting binary files are found in the output/test folderubuntu@ubuntu:~/c_enviroment$ cd output/testubuntu@ubuntu:~/c_enviroment/output/test$ lltotal 660drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 27 06:59 ./drwxrwxr-x 3 ubuntu ubuntu 4096 Apr 27 06:49 ../-rwxrwxr-x 1 ubuntu ubuntu 13868 Apr 27 06:58 adc_test*-rwxrwxr-x 1 ubuntu ubuntu 28284 Apr 27 06:58 adxl345_test*-rwxrwxr-x 1 ubuntu ubuntu 14209 Apr 27 06:58 interrupt_test*-rwxrwxr-x 1 ubuntu ubuntu 13726 Apr 27 06:58 io_test*-rwxrwxr-x 1 ubuntu ubuntu 13712 Apr 27 06:59 linker_button_test*-rwxrwxr-x 1 ubuntu ubuntu 13907 Apr 27 06:59 linker_buzzer_test*-rwxrwxr-x 1 ubuntu ubuntu 13689 Apr 27 06:59 linker_hall_sensor_test*-rwxrwxr-x 1 ubuntu ubuntu 13760 Apr 27 06:59 linker_joystick_test*-rwxrwxr-x 1 ubuntu ubuntu 13769 Apr 27 06:59 linker_led_bar_test*-rwxrwxr-x 1 ubuntu ubuntu 13690 Apr 27 06:59 linker_led_test*-rwxrwxr-x 1 ubuntu ubuntu 14290 Apr 27 06:59 linker_light_sensor_test*……

C Command lineTo view the contents of a sample sketch, (this example we’ll look at the contents of linker_led_test.c) type: ubuntu@ubuntu:~/c_enviroment/sample$ cat

linker_led_test.c/** LED test program*/#include <core.h>int led_pin = 1; void setup(){if(argc != 2){goto _help;}led_pin = atoi(argv[1]);if((led_pin < 0) || (led_pin > 13)){goto _help;}pinMode(led_pin, OUTPUT);return; _help:printf("Usage %s LED_PIN_NUM(0-13)n", argv[0]);exit(-1);}

void loop(){digitalWrite(led_pin, HIGH); // set the LED ondelay(1000); // wait for a seconddigitalWrite(led_pin, LOW); // set the LED offdelay(1000); // wait for a second} 

Creating Your Own Sketchubuntu@ubuntu:~/c_enviroment/sample$ nano button_led.c An empty nano screen should appear.Copy and paste the following code into it. (Remember to paste in nano at the cursor,

just right click the mouse button). #include <core.h> // Required first line to run on pcDuinoint ledPin = 8;int buttonPin = 7;// variables will change:int buttonState = 0; // variable for reading the pushbutton status void setup() {// initialize the LED pin as an output:pinMode(ledPin, OUTPUT);// initialize the pushbutton pin as an input:pinMode(buttonPin, INPUT);} 

Creating Your Own Sketchvoid loop(){// read the state of the pushbutton value:buttonState = digitalRead(buttonPin); // check if the pushbutton is pressed.// if it is, the buttonState is HIGH:if (buttonState == HIGH) {// turn LED on:digitalWrite(ledPin, HIGH);}else {// turn LED off:digitalWrite(ledPin, LOW);}}

Creating Your Own SketchModify the Makefile and Compileubuntu@ubuntu:~/c_enviroment/sample$ nano Makefile You will see a section that lists all the OBJS something like: OBJS = io_test adc_test pwm_test spi_test adxl345_test serial_test liquidcrystal_i2c liquidcrystal_spi

interrupt_test tone_testOBJS += linker_led_test linker_potentiometer_test linker_tilt_test linker_light_sensor_test

linker_button_testOBJS += linker_touch_sensor_test linker_magnetic_sensor_test linker_temperature_sensor_test

linker_joystick_testOBJS += linker_rtc_test linker_sound_sensor_test linker_buzzer_test linker_hall_sensor_test

linker_led_bar_test linker_relay_testOBJS += pn532_readAllMemoryBlocks pn532readMifareMemory pn532readMifareTargetID

pn532writeMifareMemory 

Creating Your Own SketchWe’re going to add a line to the end of this with the name of the scketch we just created: OBJS += button_led Save the file and exit nano using <CTRL>X with a y and <enter>. We now run make by typing: ubuntu@ubuntu:~/c_enviroment/sample$ make You should see a whole bunch of text with the end being: button_led.c -o ../output/test/button_led ../libarduino.a If all went well, you can go to the output/test folder and find your executable you have created: ubuntu@ubuntu:~/c_enviroment/sample$ cd ../output/test/ubuntu@ubuntu:~/c_enviroment/output/test$ lltotal 676drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 27 07:51 ./drwxrwxr-x 3 ubuntu ubuntu 4096 Apr 27 06:49 ../-rwxrwxr-x 1 ubuntu ubuntu 13868 Apr 27 07:51 adc_test*-rwxrwxr-x 1 ubuntu ubuntu 28284 Apr 27 07:51 adxl345_test*-rwxrwxr-x 1 ubuntu ubuntu 13668 Apr 27 07:51 button_led*…..(not showing rest of listing here)  

Creating Your Own SketchRun Your Sketch To run it, once you have wired up a switch and led to the right pins, type: ubuntu@ubuntu:~/c_enviroment/output/test$ ./button_led To stop the program, <Ctrl>C A Quick Re-Cap Add #include <core.h> to the top of your sketch.Create your sketch in the samples folder (if your familiar with linux,

makefiles, and compiling code, you could set up your own)Add the filename to the Makefile in the samples folder in the OBJS section

without the .cRun makeRun the executable from the output/test folder.You can introduce command line arguments into your sketch to make it

more transportable. 

Arduino IDE

Arduino IDE

Arduino IDE

Arduino IDE

Arduino IDE

ArduBlock

Pythonubuntu@ubuntu:~/python-pcduino/Samples/blink_led$ more blink_led.py #!/usr/bin/env python# blink_led.py# gpio test code for pcduino ( http://www.pcduino.com )#import gpioimport timeled_pin = "gpio2"def delay(ms): time.sleep(1.0*ms/1000)def setup(): gpio.pinMode(led_pin, gpio.OUTPUT)def loop(): while(1): gpio.digitalWrite(led_pin, gpio.HIGH) delay(200)

OpenCV

OpenCVdef process(infile): image = cv.LoadImage(infile); if image: faces = detect_object(image) im = Image.open(infile) path = os.path.abspath(infile) save_path = os.path.splitext(path)[0]+"_face" try: os.mkdir(save_path) except: pass if faces: draw = ImageDraw.Draw(im) count = 0 for f in faces: count += 1 draw.rectangle(f, outline=(255, 0, 0)) a = im.crop(f) file_name = os.path.join(save_path,str(count)+".jpg") # print file_name a.save(file_name) drow_save_path = os.path.join(save_path,"out.jpg") im.save(drow_save_path, "JPEG", quality=80) else: print "Error: cannot detect faces on %s" % infile if __name__ == "__main__":process("./opencv_in.jpg")

OpenCV#!/usr/bin/env python#coding=utf-8import osfrom PIL import Image, ImageDrawimport cv def detect_object(image): grayscale = cv.CreateImage((image.width, image.height), 8, 1) cv.CvtColor(image, grayscale, cv.CV_BGR2GRAY) cascade = cv.Load("/usr/share/opencv/haarcascades/haarcascade_frontalface_alt_tree.xml") rect = cv.HaarDetectObjects(grayscale, cascade, cv.CreateMemStorage(), 1.1, 2, cv.CV_HAAR_DO_CANNY_PRUNING, (20,20)) result = [] for r in rect: result.append((r[0][0], r[0][1], r[0][0]+r[0][2], r[0][1]+r[0][3])) return result  

Cloud 9 IDE Cloud9 IDE is an online development

environment for Javascript and Node.js applications as well as HTML, CSS, PHP, Java, Ruby and 23 other languages.

You're programming for the web, on the web. Teams can collaborate on projects and run them within the browser. When you're finished, deploy it—and you're done!

Cloud 9 IDE

QT on pcDuino

QT on pcDuino

Scratch

$sudo apt-get install pcduino-scratch

Home Automation： IP controllable LED

Many users are asking if the hardware part can be programmed together with the Ubuntu linux?

Sure. This is the beauty of pcDuino. The Arduino compatible hardware is a native part of the OS.

pcDuino includes Ethernet port, USB Wifi dongle, so there is no need for Ethernet shield, Ethernet shield , USB host shield, MP3 shields and so on.

Now, we are going to implement a TCP/IP socket server on pcDuino to listen to the data coming from client.

When it receives character ’O', it will turn on the LED, and when it receives ‘F”, it will turn on the LED. No actions if receive something else.

Home Automation： IP controllable LED

#include “sys/socket.h”#include “netinet/in.h”#include “arpa/inet.h”#include “sys/types.h” int led_pin = 2; int listenfd = 0, connfd = 0;int n;struct sockaddr_in serv_addr;  char sendBuff[1025];time_t ticks;  

 void loop(){ n = read(connfd, sendBuff, strlen(sendBuff) );  if(n>0){if(sendBuff[0]=='O') digitalWrite(led_pin, HIGH); // set the LED onif(sendBuff[0]=='F') digitalWrite(led_pin,LOW); // set the LED off}}

void setup(){ led_pin = 2; pinMode(led_pin, OUTPUT); listenfd = socket(AF_INET, SOCK_STREAM, 0);memset(serv_addr, '0', sizeof(serv_addr));memset(sendBuff, '0', sizeof(sendBuff));  serv_addr.sin_family = AF_INET;serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);serv_addr.sin_port = htons(5000);  bind(listenfd, (struct sockaddr*) serv_addr, sizeof(serv_addr));  listen(listenfd, 10);  connfd = accept(listenfd, (struct sockaddr*)NULL, NULL);  }

Programming under Android

Two flavors to program under Android There are two flavors to program under

Android: Command line QT5 GUI

Command line

QT5 GUI

We can copy the apk though pcDuino OTG or SD card to pcDunio and install it there.

WiFi Real-time Video Surveillance remote control robot

Steps to Assemble Rover Turn pcDuino into aWiFi AP Install video stream server Motor driver shield TCP/IP Communication Android APK