UFG-07 Family User ManualMC... · /dev/video9(live video device) /dev/video10(H.264 main video...

UFG-10 MC Frame Grabbers

LINUX SOFTWARE

PROGRAMMING

GUIDE

Customized Property

General

2

Copyright This manual, Copyright © 2014 Unigraf Oy. All rights reserved

Reproduction of this manual in whole or in part without written permission of

Unigraf Oy is prohibited.

Notice The information given in this manual is verified in the correctness on the date of issue. The

authors reserve the rights to make any changes to this product and to revise the information

about the products contained in this manual without an obligation to notify any persons about

such revisions or changes.

Edition UFG-10 MC Linux Software Programming Guide, Rev 1

Document identifier:

Date: 02 October 2014

Trademarks Unigraf and UFG are trademarks of Unigraf Oy.

All other trademarks are properties of their respective owners.

Company information Unigraf Oy

Piispantilankuja 4

FI-02240 ESPOO

Finland

[email protected]

http://www.unigraf.fi

http://www.unigraf.fi/

General

3

Table of Contents 1. General ............................................................................................................ 4 2. Driver Architecture ........................................................................................... 5

Dual-Stream ..................................................................................................... 5 3. Module Dependency ........................................................................................ 7 4. Module Installation and Un-installation ............................................................ 8

Module Installation ........................................................................................... 8 Module Un-installation ..................................................................................... 8 System adjustment: ......................................................................................... 9

5. Encoder Header Parser ................................................................................. 10 6. Encoder Property Setting ............................................................................... 12 7. Resolution & Audio Detection ........................................................................ 16 8. Scaling down preview video........................................................................... 17 9. Scaling down H264 video .............................................................................. 18 10. Input Selection ............................................................................................... 19 11. UFG-10 MC sample program ......................................................................... 20

H.264 Recorded File Playback with MPlayer ................................................. 21 Audio ............................................................................................................. 22

12. GPIO setting .................................................................................................. 23

General

4

1. GENERAL

UFG-10 Linux driver is built based on Video 4 Linux 2 (V4L2) video and Advanced Linux

Sound Architecture (ALSA) audio. Application development is straightforward by adopting the

standard APIs. In addition, this document describes some customized properties for the user to

control some special features of the UFG-10 boards.

UFG-10 MC capture board features on board hardware compression. This manual describes the

hardware compression functions and parameters available and the way they are recommended

to use.

Please find the full V4L2 and ALSA resources in the link below.

V4L2 resource:

http://linux.bytesex.org/v4l2/

ALSA resource:

http://www.alsa-project.org/alsa-doc/alsa-lib/index.html

Note Please make sure that your Linux kernel compilation includes the V4L2 (video) and ALSA

(audio) modules built in.

Note If you are using a customized Linux kernel, please have the Virtual Video Driver built in your

kernel. It is located at Device Drivers -> Multimedia Devices -> Video capture adapters ->

Virtual Video Driver.

http://linux.bytesex.org/v4l2/

http://www.alsa-project.org/alsa-doc/alsa-lib/index.html

Driver Architecture

5

2. DRIVER ARCHITECTURE

Dual-Stream UFG-10 MC sends dual-streams. It transmits raw data (YV12) stream and H.264 main stream

and H.264 sub stream simultaneously. In this way, users can use raw data to preview image or

do some intelligent functions, and can save the H.264 streams into storage and transmit it

through network at the same time.

Raw Data Stream

YV12 format, SD and HD resolution.

Only one video device.: /dev/video0, /dev/video3, /dev/video6, /dev/video9.

H.264 Stream

Main Stream Sub stream

1920×1080p@30/25/24fps 960×540p@30/25/24fps

1920×1080i@60/50fps 960×540i@60/50fps

1280×720p@60/50fps 640×360p@60/50fps

1280×720p@30/25fps 640×360p@30/25fps

720×480i@60fps 352×240i@60fps

720×576i@50fps 352×288i@50fp

By default, the chipset on UFG-10 MC transmits two H.264 streams. The first four bytes of

each H.264 data contain UFG-10 custom information about the H.264 data. The remaining

payload is the real H.264 data.

Please refer Section Encoder Header Parser later in this document for detailed information

about this subject.

The UFG-10 driver implements two video devices for each chip. Therefore, you can find two

consecutive H.264 video devices in your system.

Note The H.264 video devices are enumerated after live video devices, and the enumeration rule is

one chipset after another.

Driver Architecture

6

Example ls /dev/video*

/dev/video0(live video device)

/dev/video1(H.264 main video device)

/dev/video2(H.264 sub video device)










Module Dependency

7

3. MODULE DEPENDENCY

Note Please make sure that your Linux kernel compilation includes the V4L2 (video) and ALSA

(audio) modules built in.

Note If you are using a customized Linux kernel, please have the Virtual Video Driver and BT848

Video for Linux (V4L) built in your kernel. It is located at Device Drivers -> Multimedia

Devices -> Video capture adapters category.

Module Installation and Un-installation

8

4. MODULE INSTALLATION AND UN-INSTALLATION

UFG-10 MC Linux driver module name is LXV4L2D.ko. In order to prevent the possibility of

installation failures, Unigraf will be glad to compile UFG-10 driver in an environment used by

the customer. Please contact Unigraf for details.

Module Installation

Copy LXV4L2D.ko to your module directory.

cp LXV4L2D.ko /lib/modules/`uname -r`/

Add LXV4L2D.ko to modules.dep.

depmod –a

Load LXV4L2D.ko module.

Modprobe LXV4L2D

If “invalid module format” error occurs, try to use –f parameter.

Modprobe –f LXV4L2D

Module Un-installation

Modprobe –r LXV4L2D

If you still have error message when loading the module, please provide your configuration file

to Unigraf. It can be found in the following path:

/usr/src/`uname -r`/.config.

Module Installation and Un-installation

9

System adjustment: Enlarge memory allocation at system boot:

Fedora:

Update menu.lst, add

"vmalloc=512M" after "quiet"

Ubuntu 11.10

Find "/etc/default/grub" and edit the line

"GRUB_CMDLINE_LINUX_DEFAULT".

Add the option:

vmalloc=512MB

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vmalloc=512MB" sudo update-grub

Encoder Header Parser

10

5. ENCODER HEADER PARSER

UFG-10 MC capture cards transmit H.264 bit-stream by means of standard V4L2 stream

capture interface. For each video frame, the driver sends the bit-stream. Additionally one

customized 4 bytes stream header data structure is embedded before the H.264 bit-stream

payload. The data structure is called _AH8400_H264_STREAM_HEADER-

That means that the 1028 bytes encoded data sent from the driver consists of a 4 bytes

customized header and 1024 bytes for the real H.264 bit-stream payload.

[4 BYTES HEADER] [H.264 BITSTREAM PAYLOAD] // FRAME #00



...

[4 BYTES HEADER] [H.264 BITSTREAM PAYLOAD] // FRAME #XX

The data structure _AH8400_H264_STREAM_HEADER is defined as below:

typedef struct _AH8400_H264_STREAM_HEADER {

DWORD nFrameBufferSize :24; // FRAME BUFFER SIZE (H.264 BITSTREAM

LENGTH) DWORD nFrameType :1; // 0 = I FRAME / 1 = P FRAME

DWORD nMotionStatus :1; // 0 = STATIC / 1 = MOTION

DWORD nLockStatus :1; // 0 = UNLOCK / 1 = LOCK

DWORD nReserved :5;

} AH8400_H264_STREAM_HEADER;

In data structure AH8400_H264_STREAM_HEADER, the H.264 frame buffer size and frame

type (I/P frame) are separately stored in nFrameBufferSize and nFrameType fields.

The nMotionStatus field, is used to detect the video motion status while the system runs the

motion detection function. The nLockStatus field is used to transmit current video signal status

to the user.

Example 1:

Get Stream Header.

struct MyStreamBuffer { void * pBuffer; size_t nBufferSize;

};

struct MyStreamBuffer * p_stream_buffers = NULL;

// to do memory mapping of buffer ...

// to queue buffer to driver ...

// to wait frame complete ...

// to dequeue buffer from driver

Encoder Header Parser

11

//

struct v4l2_buffer s_v4l2_buffer;

s_v4l2_buffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

s_v4l2_buffer.memory = V4L2_MEMORY_MMAP;

ioctl( fd, VIDIOC_DQBUF, & s_v4l2_buffer );

// to get the first 4 bytes of the H.264 bit-stream buffer

//

DWORD dwStreamHeader = (p_stream_buffers[ s_v4l2_buffer.index ]->pBuffer);

ULONG nBufferSize = (dwStreamHeader & 0x00FFFFFF) >> 0;

UINT bFrameType = (dwStreamHeader & 0x01000000) >> 24;

UINT nMotionStatus = (dwStreamHeader & 0x02000000) >> 25;

UINT nLockStatus = (dwStreamHeader & 0x04000000) >> 26;

// ...

Encoder Property Setting

12

6. ENCODER PROPERTY SETTING

V4L2_CID_MPEG_VIDEO_BITRATE_MODE

V4L2_CID_MPEG_VIDEO_BITRATE

V4L2_CID_MPEG_VIDEO_GOP_SIZE

#define V4L2_CID_MPEG_VIDEO_QUALITY (V4L2_CID_MPEG_BASE + 250)

#define V4L2_CID_MPEG_VIDEO_FRAMERATE (V4L2_CID_MPEG_BASE + 251)

#define V4L2_CID_MPEG_VIDEO_OSD_ENABLE (V4L2_CID_MPEG_BASE + 256)

#define V4L2_CID_MPEG_VIDEO_OSD (V4L2_CID_MPEG_BASE + 253)

#define V4L2_CID_MPEG_SET_KEY_FRAME (V4L2_CID_MPEG_BASE + 255)

#define V4L2_CID_ENC_DEINTERLACE (V4L2_CID_BASE + 298)

#define V4L2_CID_RESET_I2S (V4L2_CID_BASE + 299)

V4L2_CID_MPEG_VIDEO_H264_PROFILE

V4L2_CID_MPEG_VIDEO_H264_LEVEL

UFG-10 MC capture cards includes ioctl interfaces described below to allow the user to control

the hardware encoder setting.

vidioc_g_ext_ctrls

vidioc_s_ext_ctrls

Note These interfaces are based on your kernel version.

Currently the property controls listed below are implemented for the UFG-10 MC:

1. Bit Rate Mode: CBR or VBR

2. Bit Rate: In CBR mode, you can set the target bit rate.

RANGE: 128K ~ 12 * 1024 * 1024 BITS / SECOND

3. GOP: 0 ~ 255.

4. Quality: RANGE: 0 (WORST) ~ 10000 (BEST)

5. H264 frame rate: 1/10~1/2(1080P60)

6. osd(M/S both affected)

7. deinterlace type: 0 (progressive), 1~7(interleave) (M/S both affected)

8. force I frame

9. H264 profile, main frame or baseline (M/S both affected)

10. H264 level: from V4L2_MPEG_VIDEO_H264_LEVEL_1_0 to

V4L2_MPEG_VIDEO_H264_LEVEL_4_1 (M/S both affected)


13

Properties Available value Default Get / set SC3A0 / SC580 Stream

Bit Rate Mode optional CBR, VBR VBR both both M/S

Bit Rate optional 128K~12M 4M both both M/S

Quality optional 0~10000(BEST) 6000 both both M/S

GOP optional 0~255 25 both both M/S

H264 frame rate optional 2500, 3000, 5000, 6000, 6250, 7500, 12500,15000, 25000, 30000

25000 both both M/S

OSD enable optional 0, 1 0 both both M/S

OSD string optional 63 chars timer both both M/S

De-interlace must 0 (progressive), 1~7 (interleave)

1 both both M/S

H264 profile optional main, baseline main both both M/S

H264 level optional 1.0 ~ 4.1 4.1 both both M/S

The properties vidioc_g_ext_ctrls/ vidioc_s_ext_ctrls ioctlsare categorized in

V4L2_CTRL_CLASS_MPEG class. Please fill ctrl_class field of structure v4l2_ext_controls

with this macro. After that you can get/set the compression properties you request.

Record Mode, Bit Rate, and GOP are readily supported in V4L2, so you can use the

V4L2_CID_MPEG_VIDEO_BITRATE_MODE, V4L2_CID_MPEG_VIDEO_BITRATE and

V4L2_CID_MPEG_VIDEO_GOP_SIZE macros to specify these properties.

UFG-10 MC defines V4L2_CID_MPEG_VIDEO_QUALITY for video quality. Please

implement this macro into your application.

Example 1.

Set compression recording mode to CBR and bit rate is 4 MB/s.

struct v4l2_ext_controls sV4l2_ext_controls;

struct v4l2_ext_control sV4l2_ext_control;

memset( &sV4l2_ext_control, 0x0, sizeof(struct v4l2_ext_control) );

sV4l2_ext_controls.controls = &sV4l2_ext_control;

sV4l2_ext_controls.ctrl_class = V4L2_CTRL_CLASS_MPEG;

sV4l2_ext_controls.count = 1;

sV4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_BITRATE_MODE;

sV4l2_ext_control.value = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR; ioctl ( fd,

VIDIOC_S_EXT_CTRLS, &sV4l2_ext_controls );

sV4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_BITRATE;

sV4l2_ext_control.value = 4 * 1024 * 1024;

ioctl ( fd, VIDIOC_S_EXT_CTRLS, &sV4l2_ext_controls );


14

Example 2:

De-interlace type

struct v4l2_control g_ctrl;

g_ctrl.id = V4L2_CID_INTERLEAVE_DETECTED;

ioctl( pDevice->m_hHwDevice, VIDIOC_G_CTRL, &g_ctrl );

if(g_ctrl.value == 0)//progressive

{

struct v4l2_control s_ctrl;

s_ctrl.id = V4L2_CID_ENC_DEINTERLACE;

s_ctrl.value = 0x0000;

ioctl( pDevice->m_hHwDevice, VIDIOC_S_CTRL, &s_ctrl ); //do only once

}

if(g_ctrl.value == 1)//interleave

{


s_ctrl.id = V4L2_CID_ENC_DEINTERLACE;

if( width > 720 )

s_ctrl.value = 0x0007;//HD

else

s_ctrl.value = 0x0003;//SD

ioctl( pDevice->m_hHwDevice, VIDIOC_S_CTRL, &s_ctrl ); //do only once

}

Example 3:

Frame rate (based on 1080p60)

sV4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_FRAMERATE;

sV4l2_ext_control.value = 30000;

//sV4l2_ext_control.value = 12500;//1/2








ioctl(pDevice->m_hHwDevice, VIDIOC_S_EXT_CTRLS, &sV4l2_ext_controls );


15

Example 4:

Set key frame

sV4l2_ext_control.id = V4L2_CID_SET_KEY_FRAME;

sV4l2_ext_control.value = 1;


Example 5

SET KEY FRAME, separate by ‘\n’, maximum length 64

sV4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_OSD_ENABLE;

sV4l2_ext_control. value = 1;// 0 is disable


//without below, default OSD timer will display

CHAR OSD[64] ;

memset(OSD, 0x00, sizeof(OSD));

memcpy(OSD, " $$$Y.$M.$d $h.$n.$s $$$f\n Unigraf UFG-10 MC\n", 50);

sV4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_OSD;

sV4l2_ext_control.string = OSD;


Example 6

Profile

sV4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE;

sV4l2_ext_control.value = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN;


sV4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_H264_LEVEL;

sV4l2_ext_control.value = V4L2_MPEG_VIDEO_H264_LEVEL_4_1;


Example 7

Reset I2S


s_ctrl.id = V4L2_CID_RESET_I2S;

ioctl( pDevice->m_hHwDevice, VIDIOC_S_CTRL, &s_ctrl );

Resolution & Audio Detection

16

7. RESOLUTION & AUDIO DETECTION

#define V4L2_CID_X_RES_DETECTED (V4L2_CID_BASE + 294)

#define V4L2_CID_Y_RES_DETECTED (V4L2_CID_BASE + 295)

#define V4L2_CID_FPS_DETECTED (V4L2_CID_BASE + 296)

#define V4L2_CID_AUDIO_SAMPLING_FREQ (V4L2_CID_BASE + 211)

User can use above properties to get correct resolution detected by decoder.

Available value Default Get / set Stream

optional get RAW

Example

struct v4l2_control g_ctrl;

g_ctrl.id = V4L2_CID_X_RES_DETECTED;

ioctl( fd, VIDIOC_G_CTRL, &g_ctrl );

LINUXXWIN_DEBUG( "( X_RES = %d ) (2)\n", g_ctrl.value );

g_ctrl.id = V4L2_CID_Y_RES_DETECTED;


LINUXXWIN_DEBUG( "( Y_RES = %d ) (2)\n", g_ctrl.value );

g_ctrl.id = V4L2_CID_FPS_DETECTED;


LINUXXWIN_DEBUG( "( FPS_RES = %d ) (2)\n", g_ctrl.value

g_ctrl.id = V4L2_CID_AUDIO_SAMPLING_FREQ;

ioctl( pDevice->m_hHwDevice, VIDIOC_G_CTRL, &g_ctrl );

LINUXXWIN_DEBUG( "FREQ = %d \n", g_ctrl.value );

Scaling down preview video

17

8. SCALING DOWN PREVIEW VIDEO

#define V4L2_CID_HV_SCALE_FACTOR (V4L2_CID_BASE + 204)

This function is to down scale the preview video resolution. It can be applied for 1920×1080

and 1280×720 resolutions.

If scale value is 0x00020002 1920×1080 will become 960×540 1280×720 will become 640×360

If scale value is 0x00040004 1920×1080 will become 480×270


optional 0x00020002 for 1280*720

0x00020002 / 0x00040004 for

1920*1080

0x00010001 get/set RAW

Example:


s_ctrl.id = V4L2_CID_HV_SCALE_FACTOR;

s_ctrl.value = 0x00010001;//not scaled


Scaling down H264 video

18

9. SCALING DOWN H264 VIDEO

#define V4L2_CID_H264_HV_SCALE_FACTOR (V4L2_CID_BASE + 205)

This function is to down scale the H264 video resolution.

This setting will effect both main and sub H264 stream.


optional 0x02020202, 0x03030303, 0x00000202

0x00000202 get/set M/S

Example:


s_ctrl.id = V4L2_CID_H264_HV_SCALE_FACTOR;

//s_ctrl.value = 0x02020202;// 1/2 of original resolution(M&S)

//s_ctrl.value = 0x03030303;// 1/4 of original resolution(M&S)

//s_ctrl.value = 0x00000202;//main original resolution,

//sub 1/2 of original resolution


Note The resolution of sub stream must be smaller than main stream

Input Selection

19

10. INPUT SELECTION

You can use V4L “VIDIOC_S_INPUT” for video.

You can use V4L “VIDIOC_S_AUDIO” for audio.

UFG-10 MC supports multiple inputs. User has to set the correct input.

//HDMI 0

//DVI.DIGITAL (TMDS.A) 1

//COMPONENT 2

//DVI.ANALOG 3

//SDI 4

//COMPOSITE 5

//S 6


optional 0~6 4 get/set RAW M/S

Example:

Set video input signal to SDI

ULONG input = 4;//SDI

ioctl(fd , VIDIOC_S_INPUT, &input );

Example:

Set audio input signal to SDI embedded

struct v4l2_audio v4l_a;

v4l_a.index = 0;//embedded audio

//v4l_a.index = 1;//exernal line in

ioctl( pDevice->m_hHwDevice, VIDIOC_S_AUDIO, &v4l_a );

UFG-10 MC sample program

20

11. UFG-10 MC SAMPLE PROGRAM

The included UFG-10 MC sample program will display one live channel and record two H.264

channels at the same time. Recorded file names are the same as their node name.

Press 'q' to exit the program.

Installation

Step 1.

If you want to change video standard (NTSC/PAL) or video resolutions, please modify line

16~20 in LINUXXWIN.c.

line16: #define DEFAULT_VIDEO_STANDARD V4L2_STD_NTSC_M

line17:// #define DEFAULT_VIDEO_STANDARD V4L2_STD_PAL_B

line19: int width = 720, height = 480; // NTSC

line20:// int width = 720, height = 576; // PAL

Step 2.

Type

make.

Step 3:

./LINUXXWIN


21

H.264 Recorded File Playback with MPlayer

Playback

The recorded H.264 file can be playback with MPlayer (www.mplayerhq.hu). After installing

the codecs and MPlayer, you can type the command to playback the H.264 file.

mplayer –fps 30 “H.264_FILE”

Playback with de-interlacing filter

You can get a list of de-interlacing options by typing mplayer -pphelp at the command line.

Find the option that you want to use, and then use the -vf pp=<option> syntax to activate the

option. The command shown below is using ffmpeg de-interlacer to process the h.264 file.

mplayer –vf pp=fd -fps 30 “H.264_FILE”

http://www.mplayerhq.hu/


22

Audio UFG-10 MC will output one stereo audio data.

UFG-10 MC audio driver is designed based on ALSA (Advanced Linux Sound Architecture).

You can use “arecord” to record sound and “aplay” to play the recorded sound.

List your audio device.

arecord -l

Record audio of capture card and play the sound instantly.

arecord -D hw:1,0 -r 48000 -c 2 -f S16_LE | aplay

(Usually hw:0,0 is your sound card device, 48000 is audio sampling frequency, it could be

48000, 44100, 32000. User should use V4L2_CID_AUDIO_SAMPLING_FREQ property to

get correct value)

GPIO setting

23

12. GPIO SETTING

#define V4L2_CID_GPIO_DIRECTION (V4L2_CID_BASE + 270)

#define V4L2_CID_GPIO_DATA (V4L2_CID_BASE + 271)

The property allows you to access the GPIO interface. The property

V4L2_CID_GPIO_DIRECTION allows you to control its direction. Writing 1 to bit enables

this pin as output pin.

GPIO 1~16 is controlled by the first chipset in one board.

SUPPORT VALUE: 0 ~ 1 – INPUT ~ OUTPUT

The property V4L2_CID_GPIO_DATA allows you to access GPIO‟s data.

SUPPORT VALUE: 0 ~ 1 – LOW ~ HIGH

EXAMPLE 1

To define GPIO pins [1:16] to output and at high voltage

Use “/dev/video0”


s_ctrl.id = V4L2_CID_GPIO_DIRECTION;

s_ctrl.value = 0xFFFF;

ioctl( fd, VIDIOC_S_CTRL, &s_ctrl );

s_ctrl.id = V4L2_CID_GPIO_DATA;

s_ctrl.value = 0xFFFF;


EXAMPLE 2:

To define even pin to output, odd pin to input

Use “/dev/video0”


s_ctrl.id = V4L2_CID_GPIO_DIRECTION;

s_ctrl.value = 0xAAAA;


UFG-07 Family User ManualMC... · /dev/video9(live video device) /dev/video10(H.264 main video...

Documents

Transcript of UFG-07 Family User ManualMC... · /dev/video9(live video device) /dev/video10(H.264 main video...