US DTO VACE Phase III Meeting Washington DC October 2006
description
Transcript of US DTO VACE Phase III Meeting Washington DC October 2006
IIT-NRC Video Recognition Systems program
http://iit-iti.nrc-cnrc.gc.cahttp://iit-iti.nrc-cnrc.gc.ca/about-sujet/cv-vi_e.html
http://synapse.vit.iit.nrc.ca (www.perceptual-vision.com)
Leader: Dr. Dmitry Gorodnichy
US DTO VACE
Phase III Meeting
Washington DC October 2006
NRC-IIT Video Recognition Systems program
+
Canadian national interests in deploying video technologies
Leader: Dr. Dmitry Gorodnichy
US DTO VACE
Phase III Meeting
Washington DC 1/XI/2006
3. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Outline
Part 1: - Who we are: NRC IIT CVG VRS
- VRS projects and results
Part 2:- Canadian National/Regional interests in Video Technology– Government (OGD) contacts and programs– Universities, Industry references
(in particular wrt national security)
4. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
National Research Council of Canada (NRC-CNRC, or NRC)• Over 30 institutes
• Across Canada, Divided in clusters
• Logo: “From Discovery to Innovation…”
• Mandate: “Make science work for Canada”(i.e. provide solutions for OGD and industry)
• Funding: government & revenue
5. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Social values
driver
impact
NRCUniversitiesOGDsIndustry
Technology/Services
Knowledge/ Discovery
Industry
IIT
NRC’s role & place
ENG
IMTI
IMI
IOT
IAR
PBI
IBDIMB
BRI
HIAIBS INMS
NINT
IIT
ICPET
IMS
IRC
IFCI
SIMS
LS PS
GHIINH
IIT
6. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
CBSA, RCMP DRDC, DND/CF
CSIS, PPTC, CIC, PCOPSERC, TC, CATSA
CPRC
- VRS program
Flight Facility
papers,conferencesCVPR, CRV
Social values
driver
impact
NRCUniversitiesOGDsIndustry
wrt. Video Technologies
Technology/Services
Knowledge/ Discovery
Industry
IIT
Other clients:HealthMedia
Education
NRC’s role & place
…
7. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A closer look
8. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A closer look
9. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A closer look
FF
IIT
10. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Institute for Information Technology (IIT-ITI, or IIT)
Revenue & Forecast
$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
$1,400,000
$1,600,000
$1,800,000
$2,000,000
FY 03-04 FY 04-05 FY 05-06 (Forecast)
Fee forServiceRevenue
CollaborativeRevenue
LicensingRevenue
Level of Activityby Site
Fredericton
Gatineau
Moncton
Ottawa
Saint JohnSydney
Total: $18,976M
NRC-IIT generates most of its revenues through licensing its enabling technologies
• Funding: Govern + Revenues
• Location: Ottawa + Atlantic (e-IT)
Make smth that works (and sales)!
11. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
IIT: Areas of Specialty
•3D Imaging, Modeling and Visualization•Data and Text Mining •Computational Video
•Natural Language Processing •Information Security, Privacy and Trust•Human Computer Interaction
•Intelligent Internet Applications•Software Engineering
12. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
IIT: Some technologies
• From Space Shuttle (Canadarm & Canadarm2), to Hollywood, to Fine Arts, to Security applications
13. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Video Recognition Systems(VRS)
• Part of Computational Video Group (formed in 2001)
• Our Mandate: Develop Video Recognition technologies for Canadian companies and OGD’s
• Formerly called Perceptual Vision Technology
• Higher mission:
Computer Vision allows computers to see.Perceptual Vision allows computers to understand
what they see.™
Test-bed and Criteria: If you are able to recognize him / it,
why computer can’t ?..
14. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
What we do
• Research, coding… • Working with companies: consulting, making prototypes• Working with OGD: consulting, joint proposals
• “Outreach & influence”: raising awareness about Video Recognition Technology – as opposed to Video Technology– public (as you see surveill. cam in the hall or watch TV), – scientists (from diff. dept: CS,Phys,NeuroBio,Psych)– government (as they allocate money)
• Organizing conferences – CRV VideoRec’07
What we don’t do (vs. Universities): teaching (except specialized lectures)
15. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A bit of history
• First projects: Canadarm2• Then: Nouse™, Surveilance
16. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
VRS now • 4 permanent Dr.’s, • 3-10 contract / guest /
coop / student workers• 2 Tech.Support
• Collaborate with Ottawa U. (VIVA lab)
• Works closely with OGD that require Video Tech.
• Lead: D. Gorodnichy
• Collaborators: M. Fiala:- Markers Tracking algorithms - Panoramic Surveillance systems
G.Roth:- Feature detection, - Projective Vision algorithms
17. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Application directions
Domain 1: Security, Surveillance and Monitoring.
Domain 2: Visually-enabled computer-human interaction (inc. hand-free user interfaces)
Domain 3: Visually-enhanced communication and Intelligent video processing (inc. annotation of video)
• Canada Borders Services Agency (CBSA)• Canadian Police Research Center (CPRC, RCMP)
• Ottawa Health Center
• U of Ottawa Music Dept.• Several Canadian/Ontario private companies
(And partners)
18. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Research & Results Technologies developed
1. Neuro-associative memorization/ recognition Face Recognition from (low-res) Video (Better than correlogram) Object
learning
2. (Multiple) object detection and tracking Automated tele-operating, …
3. Nose (convex-shape) tracking Hands-free interfaces
4. Critical Evidence Snapshot extraction Intelligent Surveillance
5. Fingers, hands detection tracking Piano playing annotation
19. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
1. Neuro-associative memorization/ recognition Face Recognition from (low-res) Video (Better than correlogram) Object
learning
2. (Multiple) object detection and tracking Automated tele-operating, …
3. Nose (convex-shape) tracking Hands-free interfaces
4. Critical Evidence Snapshot extraction Intelligent Surveillance
5. Fingers, hands detection tracking Piano playing annotation
Research & Results Technologies developed
(X,Y)
name
20. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
1. Face Recognition in VideoGoal:
Recognize faces from regular video quality:
Bad news: low-resolution (TV:320x260) data i.o.d = 12 pixels !Good news: face detectors (Viola,CMU,MIT,Pitts) can detect such faces !• So, real problem is: Different recognition modality!• So, for solution: make use of temporal domain! (“Super-resolution” comment)
• Applications: (not for identification (yet), but …) for user recognition, TV annotation, soft biometrics, multiple-people tracking, multiple-camera tracking, back-tracking
21. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
12 pixels between the eyes should be sufficient
Nominal face resolution for video-based recognition
1) Efficient visual attention mechanisms (motion etc)
2) Decision based on accumulating results over t.
3) Three main principles of neuro-processing
1. non-linear processing,
2. massively distributed collective decision making
3. synaptic plasticity.
Allows:
a) to accumulate learning data in time by adjusting synapses,
b) to associate a visual stimulus to a semantic meaning based on the computed synaptic values
Keys principles we adhere to (from human vision recognition system)
(X,Y)
“Paul”
22. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Face Recognition in Video
Solution: - 12 pixel i.o.d. face model +
accumulation over time while tracking (both in training and recognition)
- combination of neuro-biological and computer vision approaches
Tested: on TV shows, on computer user recognition
23. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
See also:
Image and Vision Computing Journal, Special Issue on “Face Processing in Video Sequences”.
(Vol. 24, No. 6). June 1, 2006. – Six-page Editorial…
– On what makes video processing special– On nominal (optimal for video) face resolution– On promising directions and future trends– On video test-sets: Why not to use “Gone with the wind”?
Also• CVPR’04 Workshop on Face Processing in Video (FPiV’04), Washington
(www.vision-interface.net/fpiv04 )• CRV’05 Workshop on Face Processing in Video (FPiV’05), Victoria
(www.vision-interface.net/fpiv05 )
24. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Goals: - make surveillance
data manageable, - be suitable for
long-range monitoring, - be suitable for
low quality video (which they often are), - be affordable (time-wise, space-wise)Challenge: foreground is not easy to detect/track in low-quality
data: thru glass, blurred, against sun (24h), etc..Solution:- Automated extraction of A.C.E. (Annotated Critical Evidence) - Integrates recent advances in video detection and tracking
2. A.C.E. Surveillance
25. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A.C.E. Surveillance• Is currently used to monitor
several premises 24/7 (long-term)
• Fast/Succinctsummarization
obtained
(in terms of ACE snapshots)
• Little space
• Browsable by time
by objectby similarity*
26. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
From 22:00 to 7:00with web-cam
See also:CRV’06 Workshop on Video Processing for Security (VP4S-06), Quebec.
(www.computer-vision.org/4security )
27. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
3. Automated Tele-operator
• Replaces manual video operator: automatically switches cameras
• Code taken from surveillance monitoring application(another test-bed comment on Test Data)
28. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
4. Nouse™ (Nose as Mouse)
• Goal: To enable hands-free control
• Problem: sub-pixel precision, robustness and convenience is needed.
+ For truly hand-free user face recognition is required.
x y , z PUI
monitor
binary eventON
OFF
recognition /memorization
Unknown User!
29. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
5. Pianist playing annotationWhich hand/finger played a note ?
• Problem: Detect/Track hands and fingers. Multi-object, deformable object tracking is hard.
• Unbiased test-bed for hands tracking – very different from sign language + pianist hands (flexible and fast)
30. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Image Search for Security Apps.(Gerhard Roth)
• UAV takes large number of images–Returns to same environment at a later time
–Find images matching ones previously taken
• City has a host of cameras at intersections–Given a single view of a vehicle
–Want to find this vehicle at other intersections
• Take many views of an object or vehicle–Many images of vehicle at different viewpoints
–Want to find this vehicle in surveillance video
31. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
•bi-tonal (only black and white)•4 corners: for 6-DOF camera/pattern pose determination•Digital Methods: Error Correction, CRC-16 Checksum
ARTag Fiducial Marker System (Mark Fiala)
32. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
•Augmented Reality
•General Purpose Tracker
•Robot Navigation and control
Marker Detection Applications(Fiala, Roth)
33. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Part 2:
Interests in Video Technology from OGD
Receptor capacity for Video Technology: users and developers
National/Regional security programs and intentions
34. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Ottawa sector and national(by interest as customers vs. R&D)
* - expressed interest in participation in external evaluation for DTO VACE technologies
CSIS“CIA”
Also: PPTC, PSERC
* CF / DND(Forces)
Classified only
Industry - CRC (Communication
Research Center )
DRDC – NI“DARPA”
(Biometrics)
* RCMP(Police)
* DRDC – AIS (UAV)
DRE S/V…
* CBSA - LSSD
Border Services
* NRC – FRL(Flight
Facilities)
* NRC-IITCVG-VRS program
Capacity/Readiness for R&D
Capacity/Readiness for testing/using as customer
35. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Companies to know
• Point Grey Research (Vancouver, UBC) – firewire cams & video processing
• i3DVR – “Intelligent” DVR
• Deep Development Corp / Gatekeeper Systems “High Resolution for Tough Environments” – DVR (won US Force contract for Boeing Military planes)
• March Networks – DVRs
36. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Universities to know
• UBC (Vancouver)
• Ottawa U, VIVA Lab works closely with NRC
• York U (Toronto)
• …(see PC for CRV VideoRec’07 workshop)
(also www.vision-interface.net – for complete list)
37. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
17 slides omitted for online version
38. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Summary
• How can we contribute to VACE:– Science-wise: Reviewing, Testing
– Client-wise: Testing
– Liaisoning with OGD and national/regional/municipal surveillance programs
– Consulting, Tips, Feedback, Cheering-up :)
– Inviting (Making) everyone here to participate in:
CRV Workshop on Video Processing and Recognition(VideoRec’07)
Montreal, Canada, May 28-30