UI Issues, Neural Nets, RTS
description
Transcript of UI Issues, Neural Nets, RTS
![Page 1: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/1.jpg)
UI Issues, Neural Nets, RTS
CSE 490RAJanuary 27, 2005
![Page 2: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/2.jpg)
Lecture outline
Leftovers from UI Lecture Neural Networks RTS
![Page 3: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/3.jpg)
Uses of the stylus
Direct writing Abstract writing Pointing Selecting Gesture Direct manipulation Control
![Page 4: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/4.jpg)
Mode issues in pen computing Adding modes to the pen
Barrel button Secondary button with non-dominant hand Eraser tip Pressure Explicit mode buttons
Area based modes (writing area, gesture area, control area, etc.)
Cursor feedback Importance of visual cues for informing user Errors in crossing mode boundaries
![Page 5: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/5.jpg)
Control
Widgets design for stylus use
![Page 6: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/6.jpg)
Flow Menu
Use movement through octants for control information
ItemText
Shape
Item
Move
ZoomHighlight
Custom
400%
200%
100%
66.6%
50%
800%25%
Custom
400%
200%
100%
66.6%
50%
800%25%
75.0 100.0
![Page 7: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/7.jpg)
Interaction with direct manipulation
ItemMove
ZoomHighlight
![Page 8: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/8.jpg)
CrossY: Crossing based UI Specify
operations by drawing through
![Page 9: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/9.jpg)
Gestures Commands issued with a single
stroke May be drawn or invisible Support from SDK
Register gestures to be recognized UI Issues
Similar to keyboard short cuts Speed up for experts Hard to learn / remember
![Page 10: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/10.jpg)
Gestures Ambiguity
Distinction between gestures Distinction between gesture and other ink
Robustness Handling misrecognized gestures
False positive False negative
Gesture initiated actions should be undoable
![Page 11: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/11.jpg)
Neural Networks
Fundamentals for Handwriting Reco Lecture (Jay Pittman)
Recognition algorithm
Learning based recognition algorithm
![Page 12: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/12.jpg)
General considerations for learning algorithms Training sets
Collection Evaluation
Training cost Time and space
Algorithm cost Time and space
Robustness to error
![Page 13: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/13.jpg)
Neural networksPerceptrons Motivated by considerations of the brain
![Page 14: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/14.jpg)
Single layer neural networks
Bias weights Threshold activation function
Step function Sigmoid function: 1/(1 + e-x)
![Page 15: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/15.jpg)
What you can do with single layer networks
Any linearly separable dataset can be recognized with a single layer neural network
![Page 16: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/16.jpg)
Gradient descent algorithm
Choose initial weights While not at optimum
Compute derivative Move along derivative
It can be proved this converges
![Page 17: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/17.jpg)
However, single layer networks are very limited
![Page 18: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/18.jpg)
Multilayer networks with hidden nodes
Can recognized much wider range of data set
The gradient descent algorithm generalizes to this case
![Page 19: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/19.jpg)
Real Time Stylus
Allow for user computation on the ink thread
![Page 20: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/20.jpg)
Architecture (Overview)
Pen Service
RealTime Event Sink
RealTime Event Sink
RealTimeStylus
queue
RealTime Event Sink
“Inking” Thread UI Thread
Ink CollectingObject
storage
InkCollector
![Page 21: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/21.jpg)
Substroke operations
Examples Custom Inking Multiple Ink Distributed Ink
Rendering Dynamic – draw on ink packet Static – draw on paint event
![Page 22: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/22.jpg)
Custom Inking Create plugin to
listen for packets Registor for
Packets Draw triangle on
each packet
![Page 23: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/23.jpg)
public void Packets(RealTimeStylus sender, PacketsData data){ for (int i = 0; i < data.Count; i += data.PacketPropertyCount){ Point point = new Point(data[i], data[i+1]); // Packet data always has x, y
// followed by the rest
Point convertedPoint = new Point(); // We need to convert to Pixels... convertedPoint.X = (int)Math.Round((float)point.X * (float)myGraphics.DpiX/2540.0F); convertedPoint.Y = (int)Math.Round((float)point.Y * (float)myGraphics.DpiY/2540.0F);
if (this.pointCount == 0)this.firstPoint = convertedPoint;
if (this.pointCount > 1){ Color color = pointCount % 2 == 0 ? Color.Red : Color.Yellow;Point[] triangle = new Point[3];triangle[0] = this.firstPoint;triangle[1] = this.previousPoint;triangle[2] = convertedPoint;
myGraphics.FillPolygon(new SolidBrush(color), triangle); } this.pointCount++; this.previousPoint = convertedPoint; }}
![Page 24: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/24.jpg)
Distributed Ink
Capture ink packets on machine 1 Send packets to machine 2 Reconstruct ink stroke by inserting
packets
![Page 25: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/25.jpg)
Sample application
![Page 26: UI Issues, Neural Nets, RTS](https://reader036.fdocuments.us/reader036/viewer/2022062315/56814e7e550346895dbc1a8e/html5/thumbnails/26.jpg)
Distributed Ink
Sender: Collect packets in packet, when count
is above threshold, send message Receiver:
Insert packets as custom data Render custom data as it is received Assemble custom data as an ink
stroke