Videoconferencing Project Objectives - EECS …cs150/sp07/Lectures/11... · Videoconferencing...
Transcript of Videoconferencing Project Objectives - EECS …cs150/sp07/Lectures/11... · Videoconferencing...
CS 150 - Spring 2007 – Lec. #11: Course Project - 1
Videoconferencing Project
! Project Concept and Background
! Checkpoint Structure
! Bells and Whistles
CS 150 - Spring 2007 – Lec. #11: Course Project - 2
Objectives
! Broad “brush” overview of the project
! Details will be covered in the lab lectures, startingnext week
! NOTE: anything discussed in the lab lectures andproject checkpoint write-ups supercedes what Idescribe here!" Neil and Allen have a working implementation of the project
" They know the project better than I do! Listen to them!
CS 150 - Spring 2007 – Lec. #11: Course Project - 3
Course Project: VideoconferencingSystem
! Not quite this… but:" Video camera capture" CRT video display" Serial compressed video
2-way transmission betweentwo stations
" Wireless communications
" (no audio this semester)" Implemented in a
Xilinx FPGA on theCalinx boards in the lab
" Groups of two -- your Lab #4/#5 partner" Commit to a TA now for grading purposes
CS 150 - Spring 2007 – Lec. #11: Course Project - 4
Calinx EECS 150 Lab/Project Protoboard
Flash Card &
Micro-drive Port
Video Encoder &
Decoder
AC ’97 Codec &
Power Amp
Video & Audio Ports Four 100 Mb
Ethernet Ports
8 Meg x 32
SDRAM
Quad Ethernet
Transceiver
Xilinx
Virtex 2000ESeven Segment
LED Displays
Prototype
Area
CS 150 - Spring 2007 – Lec. #11: Course Project - 5
Complete Videoconferencing System
Display
Video Encoder
Video Encoder(Checkpoint #1)
Video Decoder
Camera
Videostream
Video
Decoder
Checkpoint #2
Checkpoint #4
SDRAM(Checkpoint #0)
Multiport SDRAM
Memory System
MultiportArbitration
Wireless Transceiver(Checkpoint #3)
CS 150 - Spring 2007 – Lec. #11: Course Project - 6
Checkpoint #0/#1/#2:SDRAM Interface
! Memory protocols" Bus arbitration" Address phase" Data phase
! DRAM is large, but few address lines and slow" Row & col address" Wait states
! Synchronous DRAM provides fast synchronous access currentblock" Little like a cache in the DRAM" Fast burst of data
! Arbitration for shared resource
CS 150 - Spring 2007 – Lec. #11: Course Project - 7
Checkpoint #1: Video Encoding! Pixel Array:
" Digital image represented bymatrix of values, where each is afunction of the informationsurrounding it in the image; singleelement in image matrix: pictureelement or pixel (includes info forall color components)
" Array size varies for differentapps and costs: some common sizesshown
! Frames:" Illusion of motion created
by successively flashing still
pictures called frames
High-Definition Television (HDTV), 2 Mpx
Workstation, 1 Mpx
PC/Mac, 1‡2 Mpx
Video, 300 Kpx
SIF,82 Kpx
High-Definition Television (HDTV), 1 Mpx
CS 150 - Spring 2007 – Lec. #11: Course Project - 8
Checkpoint #1: Video Encoding
! Video details fairly complex and involve many choices:" NTSC vs. PAL, HDTV, …
" Interleaved even-odd frames (TV) vs. progress scan (computer anddigital displays)
" Frame size, frame rate
" Pixel encodings: RGB, YUV/YCB (Luminance, Chrominance --brightness plus color difference signals)
" Subsampling to reduce data demands (compression trick)
" Inputs: ITU-R BT.601 Format (Digital Broadcast NTSC)
" Outputs: Component video, S-video to drive LCDs in lab
" Fortunately, Calinx board has a chip on-board that deals with muchof the grungy details …
CS 150 - Spring 2007 – Lec. #11: Course Project - 9
ITU-R BT.656 Details! Interfacing details for ITU-601
Pixels per line 858
Lines per frame 525
Frames/sec 29.97
Pixels/sec 13.5 M
Viewable pixels/line 720
Viewable lines/frame 487
! With 4:2:2 chroma sub-sampling,send 2 words/pixel (Cr/Y/Cb/Y)
! Words/sec = 27MEncoder runs off a 27MHz clock
! Control info (horizontal & verticalsynch) is multiplexed on data lines
! Encoder data stream show to right
! See video tutorial documents oncourse documentation web page!
718 719 720 721 0 1 2
359 360 0 1
359 360 0 1
736732( )
368366( )
368366( )
857863)(
Y 7
18
Y 7
19
C
360
B
Y 7
20
C
360
R
Y 7
21
C
359
B
C
359
R
Y 7
36(7
32)
C
368(3
66)
B
C 3
68(3
66)
R
Y 8
55( 8
61)
C
428(4
31)
B
Y 8
56( 8
62)
Y 8
57(8
63)
C
0B
Y 0
C
0R
Y 1
C
428(4
31)
R
C 0
B
Y 0
Y 1
C
0R
C
359
B
Y 7
18
Y 7
19
C
359
R
Last sampleof digital active line
Sample datafor O instant
First sampleof digital active lineH
Luminancedata, Y
Chrominancedata, CR
Chrominancedata, CB
Replaced bytiming reference
signal
Replaced bydigital blanking data
Replaced bytiming reference
signal
End ofactive video
Start ofactive video
Timing reference signals
Note 1 – Sample identification numbers in parentheses are for 625-line systems where these differ from those for 525-line systems. (See also Recommendation ITU-R BT.803.)
FIGURE 1
Composition of interface data stream
D01
CS 150 - Spring 2007 – Lec. #11: Course Project - 10
Checkpoint #1: Video Encoder
! Display driver processes pixels within frame buffer
! Drive ADV7194 video encoder device to output correct NTSC video
! Gain lots of experience reading data sheets
! Dictates the 27 MHz operation rate" Used throughout graphics subsystem
CS 150 - Spring 2007 – Lec. #11: Course Project - 11
Calinx On-Board Video Encoder! Analog Devices ADV7194: ITU 601/656 in, Composite Video Out
! Supports:" Multiple input formats and outputs
" Operational modes, slave/master
" Used in default mode: ITU-601 as slave
s-video output
! Digital input side connected to Virtex pins
! Analog output side wired to on boardconnectors or headers
! I2C interface for initialization:" Wired to Virtex
CS 150 - Spring 2007 – Lec. #11: Course Project - 12
SDRAM READ Burst Timing
CS 150 - Spring 2007 – Lec. #11: Course Project - 13
Checkpoint #2: Video Decode
! Pretty much the reverse of the encodingprocess of Checkpoint #1
! We will provide the base Verilog for video decode
! You will need to integrate video decode with yourSDRAM arbitrated write port
! Integrate with your Checkpoint #1
CS 150 - Spring 2007 – Lec. #11: Course Project - 14
Checkpoint #3: Wireless Transceiver
! This will involve interfacing tothe wireless transceiver chip onthe Calinx2 board
! Neil working on a cleardescription of how this works
CS 150 - Spring 2007 – Lec. #11: Course Project - 15
Checkpoint Build Up to Complete Project
! Week #7: Lab #6/Checkpoint #0 -- Basic SDRAM Subsystem
! Week #8: Checkpoint #1 -- SDRAM to Video Display (Encoder)
! Week #9: Checkpoint #2 -- Local Video System" Video Capture (Decoder) to SDRAM to Video Display (Encoder)
" Video Decoder Verilog will be provided to you
! Week #10/11 : Checkpoint #3 -- Wireless Transceiver" Midterm #2 scheduled for Week #10
" Spring break between Week #10 and #11
! Week #12/13: Checkpoint #4 -- Putting it altogether" Video Capture to SDRAM to Wireless Transceiver to SDRAM to
Video Display
! Week #14: Final Report
CS 150 - Spring 2007 – Lec. #11: Course Project - 16
Possible Bells and Whistles
! Still thinking about this but here are some ideas:" Performance tuning: larger remote display, higher refresh rate
" Sending more data per unit time via compression/decompressionthrough the wireless transceiver
" Your good idea here
" NOTE: We don’t necessary know how to implement these ourselves!(these haven’t been implemented in the TA solution, for example)
" NOTE: There will be a bonus for an early demo of the completeproject at the end of Week #12 (one week early)
" NOTE: Extra credit will be limited to 20% extra points and no extracredit unless the standard functionality works