Taras Chaykivskyy - Computer Vision in Front-End
-
Upload
eastern-european-computer-vision-conference -
Category
Technology
-
view
1.468 -
download
0
Transcript of Taras Chaykivskyy - Computer Vision in Front-End
![Page 1: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/1.jpg)
eleks.com
Computer VisionIn Front-End (only)by Taras Chaykivskyy
![Page 2: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/2.jpg)
Q&ABut js is slow … ?
![Page 3: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/3.jpg)
Why ?Cross-platformNo installation
No huge video trafficWorks offline (cached)
![Page 4: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/4.jpg)
JavaScript CV instrumentsARToolkit.jsARUco.jsTracking.jsJSFeat
![Page 5: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/5.jpg)
Results :● QR marker (3 / 30 FPS)● Color QR markers (1 / 5 FPS)● Image matching (0 / 1 FPS)
“Web-Based Augmented Reality:To adopt or not to adopt ?”
* (LG Optimus L5 / Nexus 5 )
![Page 6: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/6.jpg)
Results :● Laptop/Desktop (15-30 FPS)● Smartphone* (5 FPS)
Virtual-Makeup
* Nexus 5
![Page 7: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/7.jpg)
How to boost CV on Web Application ?
![Page 8: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/8.jpg)
*dlib’s implementation**with -O3 optimization flag
SURF* on C++
height 128 256 512
C++** (ms) 0.77 6.2 29.67
![Page 9: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/9.jpg)
asm.jsIs low-level subset of JavaScript.
Emscripten (source-to-source compiler)
clang main.cpp -o mainem++ main.cpp -o mainModule.js
![Page 10: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/10.jpg)
asm.js
C codesize_t strlen(char *ptr) { char *curr = ptr; while (*curr != 0) { curr++; } return (curr − ptr);}
asm.js codefunction strlen(ptr) { ptr = ptr|0; var curr = 0; curr = ptr; while (MEM8[curr]|0 != 0) { curr = (curr + 1)|0; } return (curr − ptr)|0;}
![Page 11: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/11.jpg)
*dlib’s implementation**with -O3 optimization flag
SURF on asm.js
height 128 256 512
C++** (ms) 0.77 6.2 29.67
Asm.js** (ms) 3.5 21.5 104
![Page 12: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/12.jpg)
Call js from C++ code
Run js code from c++#include <emscripten.h>
int main() { EM_ASM( alert('hello!'); throw 'all done'; ); return 0;}
Arguments…EM_ASM_({Module.print('I received:'+$0);}, 100);…
![Page 13: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/13.jpg)
WebGLIs a JavaScript API for rendering interactive 3D computer graphics
![Page 14: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/14.jpg)
WebGL
http://webglstats.com/
![Page 15: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/15.jpg)
WebGL
js codevar gl = canvas.getContext("webgl");...var shaderProgram = gl.createProgram();gl.attachShader(shaderProgram, vertexShader);gl.attachShader(shaderProgram, fragmentShader);gl.linkProgram(shaderProgram);gl.useProgram(shaderProgram);...
![Page 16: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/16.jpg)
*dlib’s implementation**with -O3 optimization flag***upper bound
height 128 256 512
C++** (ms) 0.77 6.2 29.67
Asm.js** (ms) 3.5 21.5 104
+WebGL*** (ms)
~15 ~25 ~75
SURF on asm.js + WebGL
![Page 17: Taras Chaykivskyy - Computer Vision in Front-End](https://reader031.fdocuments.us/reader031/viewer/2022030223/5882a4e71a28ab92618b6bed/html5/thumbnails/17.jpg)
What’s next ?