FMTS: Flexible multi-touch sensor/track pad

30
TK3: Ubiquitous Computing 1 | Page TK3: Ubiquitous Computing Final Project Flexible Multi-Touch Pressure Sensitive Sensor-Track Pad (FMTS) Group Members Faheem Nadeem : 1605624 Fawaz Amjad Malik : 1594913

description

In the area of tangible interaction devices FMTS (Flexible multi-touch sensor/track pad provides a unique way of acquiring multi-touch synaptic sensing information in environments where flexibility of hardware forms a key role in assessing device applicability and use case. FMTS is a flexible, resistive and highly scalable solution for mapping multi-touch XY data as well as providing a third dimension of data sensing by acquiring pressure information from each registered touch point. Combined with standardized TUIO protocol fast data delivery and cost effective manufacturing; FMTS forms a reliable touch detection solution for rugged, flexible and scalable environments.

Transcript of FMTS: Flexible multi-touch sensor/track pad

Page 1: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

1 | P a g e

TK3: Ubiquitous Computing

Final Project

Flexible Multi-Touch Pressure Sensitive Sensor-Track Pad

(FMTS)

Group Members

Faheem Nadeem : 1605624

Fawaz Amjad Malik : 1594913

Page 2: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

2 | P a g e

Contents

� Acknowledgements

� Introduction

� User Case

o Problem Statement

o Solution

o Design

o FMC modeling

� Design and Implementation

o Objectives

o Design specifications

o Implementation

� Hardware

� Software

� Testing

o Testing methodology

o Results and expectations

� Future extensions

� Appendix

o Appendix A

o Appendix B

Page 3: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

3 | P a g e

Acknowledgements

We would like to thank Dr. Jurgen Steimle, Daniel Schreiber and Simon Olberding for their

excellent guidance, support and resourcefulness towards the completion of this project in due

time.

Page 4: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

4 | P a g e

Introduction

In the area of tangible interaction devices FMTS (Flexible multi-touch sensor/track pad

provides a unique way of acquiring multi-touch synaptic sensing information in

environments where flexibility of hardware forms a key role in assessing device applicability

and use case.

FMTS is a flexible, resistive and highly scalable solution for mapping multi-touch XY data

as well as providing a third dimension of data sensing by acquiring pressure information

from each registered touch point.

Combined with standardized TUIO protocol fast data delivery and cost effective

manufacturing; FMTS forms a reliable touch detection solution for rugged, flexible and

scalable environments.

Page 5: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

5 | P a g e

User Case

Problem Statement

Ishtiaq is a product design and analysis engineer at a multinational company

delivering small-scale mechanical parts used in a variety of manufacturing machineries. He

is the team lead for the Robo-Arm group responsible for designing control parts of an arm of

a manufacturing robot. He handles a team of five people all assigned with designing different

parts of the robot arm. He is also responsible for designing the sensitive parts and

overlooking the compatibility of each component produced by his team.

Ishtiaq’s job is tough. He has to utilize multiple technologies to collaborate, design, control

and analyze his work, which can sometime seriously hinder the workflow and undermine his

efficiency. While working on designing a model on his computer he has to use his mouse

and keyboard to size and create models. He has to learn the commands and the shortcuts to

perform simple repeated operations like changing zoom level of model, rotating model,

changing model view, making deeper alterations to model etc. It takes him a long time to do

these changes using a single click based mouse input.

After completion and during the design of a component, Ishtiaq has to collaborate with his

team members to make it interoperable; make changes based on their feedback and discuss

ad work on various suggestions. Currently this is down in a closed room with each of his

group members voicing their concerns and jotting their own notes, which makes the process

long and prone to considerable errors.

At various stages of the design process Ishtiaq has to print/manufacture prototype models.

He has to control the manufacturing machine and make on the spot adjustments to

manufacture a reasonable prototype. He has to take initial scans of the manufactured object

and present it to his team members so that they work out the remaining kinks.

Currently, Ishtiaq has to learn all the different utilities which enable him to complete the

design process. He usually gets frustrated as he has to move around a lot the different areas

of the building and take all his stuff with him. Majority of his concerns are that a single

mistake by his team members or a hindrance in their task delays the whole workflow or can

even take him back to the drawing board.

Page 6: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

6 | P a g e

Solution

Considering Ishtiaq’s need of work interoperability, we propose a ubiquitous solution

which will enable him to efficiently perform tasks such as modeling (requiring multi-

peripheral input), collaborating with his team and controlling manufacturing tasks with ease

and portability.

“Flexible multi-touch pressure sensitive sensor/pad (FMTS)” provides a low cost, portable,

flexible, malleable, rugged, multi-touch, pressure sensitive, haptic feedback surface capable

of performing all the aforementioned human-computer interaction with relative ease and

efficiency.

Now we take a peek at Ishtiaq’s regular day at his office using “FMTS”.

His day starts with making changes to his robot arm joint component on CAD software

installed in his computer. With FMTS he can get rid of his keyboard and mouse. Due to its

large surface area FMTS integrates a full QWERTY keyboard and mouse track pad onto a

single surface acquiring Ishtiaq to maneuver very little while requiring both interfaces. With

the multi-touch surface he can use various gestures to perform preset modeling tasks such as

easily make view angle changes, zooming etc. He can use the pressure sensitivity of the

surface to make terrain changes to his model with relative ease and precision.

After completing his task Ishtiaq wants to update his team member on his progress, ask their

suggestions and see where the others are headed with their tasks, so he calls a meeting.

Ishtiaq takes the light weight FMTS with him to the conference room by easily rolling it up

and putting it in his bag. Before starting the discussion he puts FMTS on a table top which is

then coupled with a camera projector system on top of the sensor. All his team members

surround the table. The camera measures his touch surface dimensions and aligns the

projector to project his model in 3D over the pad’s surface. He can explain his work to all his

team members and they can easily make real-time changes to the model using gestures on

FMTS. Every person can give his opinion simultaneously, while Ishtiaq jots down important

notes simultaneously on his resistive touch surface using a normal pen.

Once the meeting is over and Ishtiaq’s component is ready for prototyping all he has to do is

take FMTS to the manufacturing machine, which projects its HCI on the FMTS. Ishtiaq then

selects various machining configuration using haptic feedback to machine using widgets

displayed by the machines HCI on FMTS surface. Once the prototype is machined, Ishtiaq

can make further manual alteration to the surface of the prototype. After making changes if

he needs some scans of the prototype surface, FMTS can do this as well. All he has to do is

put the prototype on the FMTS surface. FMTS scans the surface of the prototype by

measuring pressure changes on the sensors surface and making a respective image on a

computer screen.

Thus FMTS is an all in one solution to Ishtiaq daily problems, making his work more

efficient and less error prone.

Page 7: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

7 | P a g e

Design

They HCI encompass the development of a flexible multi – touch pressure sensitive

surface and a camera – projector system projecting visual aids on the pad surface.

For the scope of this project we will be developing the flexible resistive multi touch sensor

capable of registering multiple touch inputs with pressure information on each input. The

surface will also be able to detect some predefined gestures to perform various computing

tasks.

FMC Modeling

Page 8: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

8 | P a g e

Design and Implementation

Design:

FMTS is an easy to use, do it yourself multi-touch solution based on Arduino based

hardware platform. This design proposes a novel and efficient method to implement a thin

pressure sensitive multi-touch surface using simple electronics and cheap everyday

materials.

Interpolated force sensitive resistive sensing technique is used for implementing a multi-

touch system. The multi-touch surface consists of a sensor array and accompanying

measurement electronics. The sensor array is constructed from a conductive material called

“velostat”, “conductive copper tapes” and “thin transparent plastic films”. For measurement

electronics, we use Arduino microcontroller, shift register, analog multiplexers and an

operational amplifier.

There are many conductive materials available that can be used to develop pressure sensors.

Velostat is among the suitable to construct a resistive multi touch sensor array.

The figure illustrates the structure of a proposed multi-touch surface. The sensor array

includes three layers. The top layer houses driving electrodes, which carry constant voltage,

and the bottom layer houses sensing electrodes, which measure the change of resistance at

crossings. The middle layer is simply a sheet of Velostat. The driving and sensing electrodes

are made of conductive copper tapes. The resistance at a crossing of two copper electrodes is

changed by applied force and detected by the measurement electronics.

Page 9: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

9 | P a g e

Measurement Electronics:

For the measurement, we employ a zero potential scanning method. For a crossing of

the ith driving electrode and jth sensing electrode, constant voltage is applied to ith driving

electrode to drive the sensor array and jth sensing electrode is connected to a signal

conditioning circuit for resistive sensing. All other electrodes remain grounded. This process

is iterated for all crossings to get all registered touch points.

Page 10: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

10 | P a g e

A 74HC595 shift register is used to drive 24 driving electrodes. For sensing electrodes,

74HC053 triple two channel analog multiplexers and 74HC595 shift register are used

through which the output of sensing electrode are connected to the signal of conditioning

circuit.

The signal conditioning circuit measures the amount of current and sends out voltage output,

which is directly connected to A/D (analog to digital) port of the Arduino.

The Arduino controls the aforementioned electronic components to successively scan the

entire crossings to construct two dimensional images and transmits them to a connected host

PC. The data packet size is 384 bytes for 24x16 multi-touch surface.

Hardware Requirements:

Given below is the list of hardware that was used in developing the project.

1. Arduino Board Duemilanove

2. Conductive Copper Tape

3. Breadboard

4. OHP transparency film

5. Velostat Material. Thickness: 100 Micron.

6. TL082 (Datasheet Link: http://focus.ti.com/lit/ds/symlink/tl081a.pdf)

7. 74HC595 (Datasheet Link: http://www.nxp.com/documents/data_sheet/74HC_HCT595.pdf)

8. 74HC4053 (Datasheet Link http://www.nxp.com/documents/data_sheet/74HC_HCT4053.pdf)

9. Resistors: 1kOhm , 10kOhm

10. DC/DC inverter 7662 A (Datasheet Link:

http://ww1.microchip.com/downloads/en/devicedoc/21468b.pdf)

11. Capacitors Electrolytic 10uF

Page 11: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

11 | P a g e

Circuit Diagram:

Page 12: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

12 | P a g e

Testing

Testing Methodology

The test setup employed the actual FMTS hardware connected using a standard usb

2.0 interface cable to a laptop running visualization and data processing software.

Two applications built around “OpenCV” framework were used to visualize data points send

by FMTS using TUIO protocol. Data visualization software provided both 2D and 3D depth

of view to further visualize pressure sensitivity and XY mappings. The two applications are

described as follows.

** Test Environment**

Page 13: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

13 | P a g e

1. VIS3d

VIS3D is a data visualization software for FMTS multi-touch capable of

delivering 3-dimensional perspective for data read from FMTS using TUIO protocol.

It can provide features such as connectivity to a serial port, data logging, start/stop

communication, noise threshold, and baud rate selection.

** Please find detail description of setting up both VIS3d and MTView in Appendix B

Page 14: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

14 | P a g e

2. MTView

MTView is a multi-touch data processing software. It acquires data from a multi-touch

surface through RS232C (a.k.a UART or Serial) communication, processes the data and

sends out resulting “XY” touch information in standard TUIO protocol.

MTView provides features such as.

• Fine tuning of FMTS parameters and application of tuned results to improve the

performance of application written with MTFramework.

• Use TUIO compatible multi-touch applications with the FMTS.

• Use Windows 7 multi-touch applications with FMTS. To enable this feature, Multi

Touch Vista must be installed.

• Provides data smoothing and filtering capabilities to improve touch recognition

accuracy.

** Please find detail description of setting up both VIS3d and MTView in Appendix B**

Page 15: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

15 | P a g e

Results and Expectations

The calculation of time for scanning the flexible multi-touch surface is very critical

especially when dealing with a large size of the surface. The scalability comes with a

tradeoff in fewer frames per second. But this can be improved which is discussed in the

future extensions.

Max data rate available with Arduino = 115200 bps

Number of bits per byte = 8 + 1(start bit) + 1 (end bit) = 10 bits

Number of bytes per second = 11520 Bps

Current design rows = 24

Current design columns = 16

Total time required to transfer 1 byte of data = 1/11520 = 86.68 microsecond

A/D conversion for Arduino = 10 KHz

For one analog to digital conversion it takes = 100 microsecond

Total time = 384 x (100+86.68) = 71.4 millisecond

Number of frames per second = 15 fps

The results produced by the prototype hardware were according to the expectations. Fifteen

frames per second data acquisition and display rate provided quick touch registration and

response on test programs. Touch points registered were accurate with little deviation from

the point of contact. Further improvements can be made to current hardware assembly and

are listed as follows.

Future Extensions: This project is just a stepping stone for great number of application and products

which currently depend upon multi touch inputs. With the addition of pressure sensing and

flexibility it open up a broader domain for the products to be realized which are currently

concepts.

• FMTS is easily extensible with little hardware increase. The first step of improvement

is to make a module prototype for the hardware that is easy to set up and can be

extended to bigger sized surfaces. The module would be a pcb design with smd

packages in order to have a small area for the hardware.

• The frame rates can be improved considerably. As it was mentioned in the timing

results the two major bottle necks are analog to digital conversion and secondly the

baud rate. Analog to digital conversion can be done with an external A/D. Also, if a

multiple channel A/D is used it would reduce the amount of time required for A/D

conversion by the number of channels directly. Secondly, use a communication

scheme other than RS232 for increased data rate.

• The flexible multi-touch surface should be accurately designed with a pcb design for

greater accuracy and better deployment.

• A faster and stronger microcontroller could be used for faster results.

• A zigbee module could be implemented with the hardware in order to have a wireless

connected touch surface.

Page 16: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

16 | P a g e

Appendix A

Arduino Code

//include this directive for printing values

#include <Stdio.h>

//Pin connected to master reset pin of 74HC595

const int masterResetD = 8;

const int masterResetS = 2;

//Pin connected to clock pin (SH_CP) of 74HC595

const int clockPinD = 9;

const int clockPinS= 3;

//Pin connected to Data in (DS) of 74HC595

const int dataPin = 11;

//Pin connected to latch pin (ST_CP of 74HC595)

const int latchPinD = 10;

const int latchPinS = 4;

int incomingByte = 0;

//Beginning value of time in evaluating time

unsigned long begin_time;

//End value of time in evaluating time

unsigned long end_time;

//Total time evaluated

unsigned long elapsed_time;

//Function to start evaluating time

void time_chk_start(void)

{

begin_time = micros();

}

Page 17: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

17 | P a g e

//Function calculating the final state of time

void time_chk_stop(void)

{

end_time = micros();

elapsed_time = end_time - begin_time;

Serial.print("Time (micro s): ");

Serial.println(elapsed_time);

}

//Pin setup

void setup()

{

pinMode(latchPinD, OUTPUT);

pinMode(latchPinS, OUTPUT);

pinMode(dataPin, OUTPUT);

pinMode(masterResetD, OUTPUT);

pinMode(masterResetS, OUTPUT);

pinMode(clockPinD, OUTPUT);

pinMode(clockPinS, OUTPUT);

//Initializing all registers to zero

initializeShiftRegisters(dataPin, clockPinD, latchPinD, 3);

initializeShiftRegisters(dataPin, clockPinS, latchPinS, 3);

//Baudrate Setup

Serial.begin(115200);

}

void loop()

{

int StartDriving = 0;

//number of rows which are driven

int EndDriving = 24;

int StartSensing = 0;

//number of columns which are driven

int EndSensing = 16;

int measurement = 0;

// char datastr[200];

byte command = 'S'; // 83

Page 18: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

18 | P a g e

if (Serial.available() > 0)

{

//Check if S char is sent in accordance with TUIO protocol

incomingByte = Serial.read();

if (incomingByte == command)

{

//in case time evaluation is required

//time_chk_start();

//reset the value of sensing registers

resetRegister(masterResetD,latchPinD);

digitalWrite(dataPin, HIGH);

//Clock is toggled and the register is shifted by one

togglePin(clockPinD);

digitalWrite(dataPin, LOW);

for (int i = StartDriving; i < EndDriving; i++)

{

togglePin(latchPinD);

resetRegister(masterResetS,latchPinS);

digitalWrite(dataPin, HIGH);

togglePin(clockPinS);

digitalWrite(dataPin, LOW);

for (int j = StartSensing ; j < EndSensing; j++)

{

// Driving, Sensing Electrode Control using 595

togglePin(latchPinS);

//reading analog voltage

measurement = analogRead(0);

//Sending the voltage measured to the connected device

Serial.print(measurement/4, BYTE);

togglePin(clockPinS);

}

togglePin(clockPinD);

}

//in case time evalution is required

//time_chk_stop();

Page 19: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

19 | P a g e

}

}

}

//Function to intialize all registers

void initializeShiftRegisters(int whichRegister, int clockPin, int latchPin, int numRegisters)

{

digitalWrite(latchPin, LOW);

for (int i = 0; i < numRegisters; i++)

{

shiftOut(whichRegister, clockPin, MSBFIRST, 0);

}

digitalWrite(latchPin, HIGH);

}

//Function to toggle the input pin

void togglePin(int whichPin)

{

digitalWrite(whichPin,LOW);

digitalWrite(whichPin,HIGH);

digitalWrite(whichPin,LOW);

}

//Function to reset the input register

void resetRegister(int resetPin , int latchPin )

{

digitalWrite(resetPin,HIGH);

digitalWrite(resetPin,LOW);

digitalWrite(resetPin,HIGH);

togglePin(latchPin);

}

Page 20: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

20 | P a g e

Appendix B

Setting up VIS3D

Prerequisites

VIS3D requires a graphic card that supports GLSL (OpenGL Shading Language). Most

recent desktop PCs and notebooks are equipped with GLSL supported graphic cards.

However, if you have a graphic card that does not support GLSL, please

use MTView instead.

Quick Start Guide

1. Find attached FMTS Distribution files

2. Go to the vis3d folder.

3. Open the vis3d.ini file to set the configuration of FMTS Hardware. Identify the serial port

number of FMTS connection and modify port field in vis3d.ini according to the identified

port number.* Leave other fields unchanged (You don't need to modify them). For more

detailed information about the configuration file, see Section Using Configuration File.

Page 21: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

21 | P a g e

**You can identify the port number through Control Panel - Device Manager**

4. Launch the vis3d application by opening vis3d.exe.

Page 22: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

22 | P a g e

• The application will show you current setting.

• If you want to see the help, press down F1. F1 will show you all vis3d instructions.

5. Press down 'o' to open serial port.

Page 23: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

23 | P a g e

6. Press down 's' to start data acquisition

7. You can change your view point using mouse right button.

Page 24: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

24 | P a g e

Commands

GUI Control

• Up/Down arrow keys: zoom in/out

• R/F: Conic color control

• T/G: Cone shape change

• Y/H: Mode/Polygon mode change

• Drag with mouse right button: view point control

Communication Control

• o: open COM port

• c: close COM port

• s: toggle data transfer (start / stop)

Data Control

• a: Remove background noise.

• E: Toggle threshold (0, 10, 20, 30, 40, and 50). Touch pixel whose value is less than

the threshold will be ignored in the visualization.

• L: Toggle logging on/off. Resulting data is stored in the "log" subfolder. It is useful if

you want analyze touch data using other applications.

• ex) ./log/S20110210155010.txt

Page 25: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

25 | P a g e

Others

It is recommended to use vis3d.ini configuration file instead of following commands.

• Number 1 ~ 0: Set COM port. 0 corresponds to PORT10

• A: Toggle background filtering on/ off.

• B: Toggle baud rate (921600 / 115200 bps)

• D: Toggle data protocol (S:8bit normal mode/ F:8bit encoded mode/ R:12bit normal

mode)

o Note: FMTS does not support 'R' mode.

• n: Decrease the number of rows

• N: Increase the number of rows

• m: Decrease the number of columns

• M: Increase the number of columns

• i: Load configuration file

Using Configuration File

You can use VIS3D.INI file to indicate your hardware information. The default

configuration file is for FMTS and as follows:

[Vis3D]

row=24

col=16

port=6

baudrate=115200

datathreshold=0

scale=0.5

requestcommand=s

You should modify 'port' according to the port number assigned to your FMTS hardware.

In the above configuration file, you may often modify following four parameters.

• Row = 24: The number of row (driving) electrodes of FMTS.

• Col = 16: The number of column (sensing) electrodes of FMTS.

• Port = 3: Your port number assigned to FMTS hardware.

• Baud rate = 115200: Baud rate of serial communication of FMTS.

• Data threshold = 0: data threshold value for noise filtering.

• Scale = 0.5: scale value for data visualization.

• Request command = s

o F: Fast data transfer. The device compresses the touch data to increase the frame

rate. In this mode, noise in the touch data is removed during the data transfer.

o S: Slow data transfer. The device sends out the touch data to a host PC without

compression. The frame rate is slow but a user can see raw touch data measured

by the device.

Page 26: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

26 | P a g e

Setting up MTView

Install Prerequisites

• Install Microsoft Visual C++ 2008 Redistributable Package(x86) file

o http://www.microsoft.com/downloads/en/details.aspx?FamilyID=a5c84275-3b97-4ab7-a40d-

3802b2af5fc2

Quick Start

1. Connect the FMTS hardware

2. Select your serial port

3. Click Open UART Port button

4. Click Start Button. MTView should display multi-touch data from the device and show multi-touch

Page 27: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

27 | P a g e

tracking results.

Detailed Usage of MTView

Tuning the FMTS parameters

See 6. Algorithm Parameters in the Section GUI Overview.

Using TUIO applications with MTView

1. Check Send TUIO.

Page 28: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

28 | P a g e

2. Launch your TUIO applications. You can use many existing TUIO compatible applications. Visit TUIO

Application Showcase for example applications.

Using Windows 7 applications with MTView

1. Install Multi-Touch Vista. Multi-Touch Vista is a Windows HID driver, input management layer with a

TUIO input provider.

2. Open mtview.ini and modify the contents. If your Multi-Touch Vista folder is "MultiTouchVista_-

_second_release_-_refresh_2", the configuration file should be:

[MTView]

Service.Console="C:\\MultiTouchVista_-_second_release_-_refresh_2\\Multitouch.Service.Console.exe"

Configuration.WPF="C:\\MultiTouchVista_-_second_release_-

_refresh_2\\Multitouch.Configuration.WPF.exe"

Driver.Console="C:\\MultiTouchVista_-_second_release_-_refresh_2\\Multitouch.Driver.Console.exe"

3. Click "Load Multi-Touch Vista" button of MTView.

4. Launch your Windows 7 compatible multi-touch application.

GUI Overview

Page 29: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

29 | P a g e

Note: For the FMTS, High pass and Smoothing2 are not necessary, therefore check them off.

1. Source Image

• Display the raw data image from the FMTS hardware.

2. Output window

• Print out current status of serial communication.

3. Multi-touch Gesture Recognition

• Display the image changed by the result of recognition user's gesture

4. Processed Image Group

• Source Image (Gray): Display the source image.

• Resized Data: Display the resized image data.

• Offset Data: Display offset image data.

• Difference: Resized image - Offset data

• Smoothing: Average filtered image of difference image

• High pass: High pass image of Smoothing image

• Smoothing2: Average filtered image of High pass image

• Amplification: Amplified image of Smoothing2

• BW: Binarization result

• Labeling: Result after labeling process

5. Menu

• Open and Close UART Port Button: Open and close serial port

• Flip Vertical: Flip the source image vertically.

• Flip Horizontal: Flip the source image horizontally.

• Send TUIO: Identified touch positions and IDs are sent in TUIO protocol. Turn on this option when

you use SGDriver.

• Start Button: Start UART communication.

• Stop Button: Stop UART communication.

• Offset Button: Take the current data as offset values

• FPS: Frames Per Seconds (ms and Hz)

• Algorithm Speed: Computing time of multi-touch data processing

• Load Multi-touch Vista Button: Run Multi-Touch Vista.

6. Algorithm Parameters Group

Parameters of the multi-touch tracking are well-tuned default values. However, you can apply your own

parameter values to further improve the performance of the FMTS.

• Gain: Raw data is multiplied by Gain

• COL / ROW: indicate the physical number of rows and columns of attached multi-touch surface.

• Width Scale (Column) / Height Scale (Row): Resize the touch data.

• Filter Size for Smoothing: Mask size of the average filter

• Filter Size for Smoothing2: Mask size of the average filter

• Amplification range (MIN) & Amplification range (MAX): Sensor data between the upper (MAX)

and lower (MIN) bounds will be regarded as a valid sensor data.

• Amplification Ratio: Higher value results in higher contrast.

Page 30: FMTS: Flexible multi-touch sensor/track pad

TK3: Ubiquitous Computing

30 | P a g e

• BW Threshold: Sensor data higher than BW Threshold are regarded as valid touch data

• Min. Size of Touch Blob / Max. Size of Touch Blob: Only touch blobs whose area between the two

values are regarded as valid touch points.

The fine-tuned parameters can be directly used for MTFramework.

In order to use your own parameter values, your mtCreateMultiTouch instead of mtCreatArduMT(). The

function prototype of mtCreateArduMT() is as follows:

mtCreateMultiTouch(int _nrow, int _ncol, int _scale, int _filter, float _minSignal, float _maxSignal, int

_bwTh, int _minBlob, int _maxBlob)

• _nrow and _ncol are 24 and 16, respectively.

• _scale: if the resized image is 72 by 48, then _scale should be 3 (=72/24 or 48/16). It should be

noticed that the _scale value is common for both rows and columns.

• _filter: apply the mask size of the first average filter (Smoothing).

• _minSignal and _maxSignal: Amplification range (MIN) & Amplification range (MAX)

• _bwTh: BW Threshold

• _minBlob and _maxBlob: Min. Size of Touch Blob / Max. Size of Touch Blob

To release the memory assigned by mtCreateMultiTouch(), use mtReleaseMultiTouch().

7. UART Communication Group

• Serial Port: Use the serial port number assigned to your FMTS.

• Baudrate: 115200 bps

• Data bit: 8bit

• Stop bit: 1

• Parity: none

SG Driver:

SGDriver is a multi-touch gesture recognizer for Windows.

It accepts TUIO protocol, recognizes multi-touch gesture and sends out TUIO corresponding “Windows”

events.

As FMTS broadcasts registered touch points in standard TUIO format, therefore they can be read by

“SGdriver” and various multi – touch gestures can be recognized and sent to other windows applications

supporting TUIO events e.g. photo viewer etc.