Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher...
-
Upload
oscar-hillson -
Category
Documents
-
view
219 -
download
0
Transcript of Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher...
![Page 1: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/1.jpg)
Reinventing Printed DocumentKeeping digital as digital with uPen
Reinventing Printed DocumentKeeping digital as digital with uPen
Jian WangSenior ResearcherMicrosoft Research AsiaJuly 19, 2005
Jian WangSenior ResearcherMicrosoft Research AsiaJuly 19, 2005
![Page 2: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/2.jpg)
OutlineOutline
Challenges and Approach
TechnologiesEmbedded Interaction Code(EIC) system
Computer vision system
HW/SW architectureHardware prototype design
EIC SDK: system design and APIs
Demos
Challenges and Approach
TechnologiesEmbedded Interaction Code(EIC) system
Computer vision system
HW/SW architectureHardware prototype design
EIC SDK: system design and APIs
Demos
![Page 3: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/3.jpg)
ChallengesChallenges
``non-digital way of doing things'' like paper and pencils
PCs still create non-digital documentSuch as printed document
Keyboard/mouse confine people to desktop
Mobile interaction device
``non-digital way of doing things'' like paper and pencils
PCs still create non-digital documentSuch as printed document
Keyboard/mouse confine people to desktop
Mobile interaction device
![Page 4: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/4.jpg)
GoalGoal
Digital original
Keep digital as digital
Makes every printed document digital
Brings digital computing to places where things are still being done in non-digital ways
Digital original
Keep digital as digital
Makes every printed document digital
Brings digital computing to places where things are still being done in non-digital ways
![Page 5: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/5.jpg)
Approach: EIC SystemApproach: EIC System
High data-embedding capacity to provide a gigantic address space up to 2192 bits.
A family of EIC visual patterns to adapt to various hardware implementations, applications and surfaces, and to meet the needs for usability and readability.
Multiple data-embedding channels to embed x and y coordinates, and both global (such as document ID) and local metadata in a digital document.
High data-embedding capacity to provide a gigantic address space up to 2192 bits.
A family of EIC visual patterns to adapt to various hardware implementations, applications and surfaces, and to meet the needs for usability and readability.
Multiple data-embedding channels to embed x and y coordinates, and both global (such as document ID) and local metadata in a digital document.
![Page 6: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/6.jpg)
10
0 1 0 1 1 0
0 0 1 1 1 0
0 1 0 0 1 1
0 1 0 0 1 0
1 0 1 1 1 0
EIC Pattern
EIC: Absolute X, Y Coordinates
![Page 7: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/7.jpg)
A pseudo-random 1D array of 0’s and 1’s with “window” property
An example with primitive polynomial of order 6:
000001000011000101001111010001110010010110111011001101010111111
h(x)=x6+x+1, code length = 26 - 1=63
The window property ensures the unique decodability
Window Property of m-sequence
![Page 8: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/8.jpg)
0
m1 = 9
m2=7
Construction of m-array
0
0
1
0
1
1
1
1
0
0
1
0
1
0
0
1
1
0
0
1
1
1
0
1
0
0
1
1
1
1
1
0
1
0
0
0
0
1
1
0
1
0
0
1
0
1
1
0
0
0
0
0
0
1
0
1
1
1
1
0
1
1
000001000011000101001111010001110010010110111011001101010111111
Create a 2D array with size of m1*m2, where gcd(m1,m2) =1
Folding the sequence
The 2D array retains the window property. If L is the bit location in m-sequence, then its location in m-array:
x= L mod m1
y= L mod m2
Create a 2D array with size of m1*m2, where gcd(m1,m2) =1
Folding the sequence
The 2D array retains the window property. If L is the bit location in m-sequence, then its location in m-array:
x= L mod m1
y= L mod m2
![Page 9: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/9.jpg)
Data-embeddingData-embedding
X, Y Position
67%
Encoded X, Y Position
Three Channels of Embedded-Data
Encoded Global Metadata
25%
Global Metadata(Document ID)
Encoded Local Metadata
Local Metadata
8%
![Page 10: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/10.jpg)
EIC pattern
Decodable areaEIC + Encoded
content
Non-decodable area
Bounding box
Action
Biding EIC with DocumentBiding EIC with Document
![Page 11: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/11.jpg)
Hardware ArchitectureHardware Architecture
MechanicalSize: 14x17x160mm
Ink cartage
Pressure sensor
OpticalCamera
Lens
IR LED
ElectricalProcessor/Memory
Bluetooth
Battery
MechanicalSize: 14x17x160mm
Ink cartage
Pressure sensor
OpticalCamera
Lens
IR LED
ElectricalProcessor/Memory
Bluetooth
Battery
![Page 12: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/12.jpg)
Computer Vision SystemComputer Vision System
To obtain high resolution (> 400 dpi), accurate x and y positions and metadata on printed digital documents (papers).
To obtain actual x and y position of the pen tip regardless of pen posture (tilting and rotating). Calibration error < 0.15mm.
To obtain sufficient spatial and temporal resolution (> 400 dpi x 120 Hz) of uPen digital ink (so that it’s compatible with the resolution of Tablet PC ink).
To obtain high resolution (> 400 dpi), accurate x and y positions and metadata on printed digital documents (papers).
To obtain actual x and y position of the pen tip regardless of pen posture (tilting and rotating). Calibration error < 0.15mm.
To obtain sufficient spatial and temporal resolution (> 400 dpi x 120 Hz) of uPen digital ink (so that it’s compatible with the resolution of Tablet PC ink).
![Page 13: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/13.jpg)
Document/EIC pattern segmentation
.32,,1,
else. ,
);,(),( and ),(),( if ,
);,(),( if ,
),(
ji
Z
jijiPjijiPA
jijiPT
jiLabel .32,,1,
else. ,
);,(),( and ),(),( if ,
);,(),( if ,
),(
ji
Z
jijiPjijiPA
jijiPT
jiLabel T: document content
A: EIC pattern
Z: background area
),( ji adaptive threshold of EIC pattern at (i, j)
),( ji adaptive threshold of document content at (i, j)
![Page 14: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/14.jpg)
31 1
,,13
21 1
1,,12
11 1
1,,1
01 1
,,0
/)||(
/)||(
/)||(
/)||(
NYXQ
NYXQ
NYXQ
NYXQ
i j
i j
i j
i j
n
i
n
jjiji
n
i
n
jjiji
n
i
n
jjiji
n
i
n
jjiji
nYXSW
nYXSB
kl
i
il
j
jk
klji
kl
i
il
j
jk
klji
/)),max((
/)),min((
,
1
1
1
1
,,
,
1
1
1
1
,,
EIC Pattern Processing
kkkkk
ijj
n
ikkkiik
YXXXb
bxyjSibxyJk
k
k
kk
'1'
1
2 }|||{,)(1
}),(,310,31,,1,0
,|),{(
)(
min
,,
,
,,
4,..,4
,,
TyxLabelyx
tktbxayyxP
N
PGray
t
iiiski
ki
Pski
ki
ski
}),(,310,31,,1,0
,|),{(
)(
min
,,
,
,,
4,..,4
,,
TyxLabelyx
tktbxayyxP
N
PGray
t
iiiski
ki
Pski
ki
ski
Determine EIC pattern grid
Determine orientation of pattern
Determine the value of each bits
![Page 15: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/15.jpg)
Efficient Image MatchingEfficient Image Matching
}),mod(,),mod(
],,[],,[|{
),(max
,,
0000,
,
,
yoffsetbarxoffsetbar
nm
jidocumentcamera
SP
ILnILm
dyydyyndxxdxxmPS
IICorrPji
}),mod(,),mod(
],,[],,[|{
),(max
,,
0000,
,
,
yoffsetbarxoffsetbar
nm
jidocumentcamera
SP
ILnILm
dyydyyndxxdxxmPS
IICorrPji
:search radius.dydx,
:Maximum grid alignment error.
CameraCamera IFI F : transformation matrix from camera coordinate to document image coordinate
![Page 16: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/16.jpg)
EIC SDK ArchitectureEIC SDK Architecture
uPen prototype/third-party hardwareuPen prototype/third-party hardware ApplicationsApplications
EIC Embedded SDKEIC Embedded SDK
EIC SDK EIC SDK
uPen FirmwareuPen Firmware
EICPattern
EICPattern MetadataMetadata
EICDocument
EICDocument.Net Compact Framework.Net Compact Framework
![Page 17: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/17.jpg)
DemosDemos
![Page 18: Reinventing Printed Document Keeping digital as digital with uPen Jian Wang Senior Researcher Microsoft Research Asia July 19, 2005 Jian Wang Senior Researcher.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551b0e615503462e578b599d/html5/thumbnails/18.jpg)
Thank youThank you