Ultrahigh-Definition Television System With 4000 Scanning Lines

F.Okano1, M.Kanazawa1, K.Mitani1, K.Hamasaki1, M.Sugawara1,

M.Seino2, A.Mochimaru3 and K.Doi4

1NHK, 2Ikegami Tsushinki Co., Ltd., 3BOSE Japan and 4 Victor Company of Japan

ABSTRACT An ultrahigh-definition television system with 4000 scanning lines has been developed. A camera and a projection display together with a hard disc recorder have been produced as a trial version. Two green channels are arranged by the diagonal-pixel-offset method to achieve the 4000 scanning lines. The 22.2 multichannel audio system is also constructed for reproducing 3-D sound and natural impression. INTRODUCTION The HDTV system has been developed to provide high resolution images, and we can enjoy realistic atmosphere in our home, nowadays. Sensation that observer perceives is affected by many factors, for example, resolution, screen size, colorimetory, smoothness of motion, viewing distance, and etc. Especially, the viewing angle is significant and deeply related to realistic and immersive sensation. If we could use the factors more effectively and establish the new television system beyond HDTV, it can open the new era for the future imagery. Our project has shown that the viewers prefer larger angle with wide-screen and high-resolution images than that with HDTV where the angle of 30 degrees is recommended. The sensation of reality induced to an observer by watching a display screen increases with the viewing angle and saturates at an angle around 100 degrees horizontally [1]. Considering human vision characteristics, the spatial resolution of a display is desired to have 60 pixels per degree [2]. To meet the above conditions, a display with approximately 4k(V)×8k(H) pixels is necessary. Table 1 shows the specifications of the ultra-high definition television system with 4k×8k pixels, comparing with those of HDTV. Figure 1 shows the relationship between human visual field [1] and display areas of HDTV and the 4k×8k system. While the display area of HDTV is covering the effective field [2], that of the 4k×8k system covers the induced field, which is related to the observer’s posture judgment and therefore the sensation of reality. At a viewing angle of 100 degrees under the aspect ratio of 9:16, which corresponds to a viewing distance of 0.75 H (H: picture height), viewers can enjoy much higher sensation of reality with 4k×8k video images than with HDTV. The resolution of the 4k×8k system is 16 times of that of HDTV and is

more than twice of that of 70 mm motion film horizontally and vertically, assuming the resolution of 35mm motion film to be 700TVL [3].

HIGH RESOLUTION TECHNOLOGY

At the present, both image sensor and display panel with such a huge number of pixels of 4k×8k are not available. On the other hand, recently, the sensor and panel with 2k×4k pixels have been developed. Therefore, introducing them is the most feasible way to achieve ultrahigh-definition system. We increased the number of pixels in both the horizontal and vertical directions. Both camera and display are applying two channels to green (G1,G2) in the diagonal-pixel-offset method, as shown in the figure 2. In addition, we set one panel of 2048×3840 pixels to red (R) and blue (B) each. The figure 3 shows the equivalent pixel structure for the scheme that is well known as a Green Bayer Pattern [4]. It makes use of human visual characteristics. Specifically, while the

More than twice of 70 mm motion film

Equivalent to 35mm motion film

Comparison with movie

30 deg. horizontally

Viewing angle (pixel invisible)

4320×7680 1080×1920

4k×8k systemHDTV

Number of pixels

More than 100 deg.horizontally

Table 1 Specifications of ultrahigh-definition television system with 4000 scanning line

100 deg.

85 deg.

30 deg

20 deg.

HDTV

4K×8K system

Effective field Induced field

Fig 1 Human visual field and display area

2004 NAB BEC Proceedings — 437

human vision features high spatial frequency response with respect to brightness, it has relatively low response with respect to hue and chroma. Because green makes a bigger contribution than red and blue to brightness components, achieving high resolution even in only green can still be effective.

SUBJECTIVE EVALUATION

The authors conducted a subjective assessment [5] test in order to confirm the picture quality of the pixel-offset method. The picture quality related to pixel structures of the four methods shown in Table 2 was investigated. The image from the Method-A has full 4k×8k pixels for red, green and blue. The Method-B and Method-C are the spatial pixel-offset method, where the image from Method-B has 2k×4k×2 pixels for red, green and blue, and that from Method-C has 2k×4k×2 pixels for only

green. The image from Method-D has 2k×4k pixels for red, green and blue (half resolution of Method-A). The simulated evaluation was performed by conventional video monitor and its result is shown by figure 4. Because the scores of the Method-B and Method-C were as high as that of Method-A (Original) and the score of the Method-D was very low, the spatial pixel-offset method was proved to be effective. The Method-C is the most cost effective, therefore, this method was adopted to develop the ultrahigh-definition system. EXPERIMENTAL EQUIPMENTS 1) Camera The optical system in this camera is composed of a lens and color separation prism, shown in figure 5. The color separation prism divides an optical image into two G1, G2, R, and B on each 2k×4k-pixel image sensor. The four image sensors are attached to the prism with half-pixel pitch offset so that the image sampling pattern of their pixel is equivalent to that of a single chip color image sensor with 4k×8k pixels [6], as shown in figure 3.

The first proto-type camera system uses four 2k×4k-pixel CCDs with 60 frames per second progressive operation capability [7]. The 34 mm by 17.2 mm image area has 2048(V)×4046(H) active pixels with 8.4 µm squares. The multiple-output structure with its 16 outputs enables high-data-rate imaging for ultrahigh-resolution moving pictures. By using this camera system, we have performed several field tests to get a various kinds of images for evaluating the system and making demonstration programs. Moreover, aiming at the practical camera system,

Table 2 Result of subjective evaluation

A

B

C

D

Green Red/Blue

4k×8k 2k×4k ×2 diagonal offset

2k×4k ×2 diagonal offset 2k×4k

2k×4k

Number of pixels Method

1

2

3

4

5

95% reliable

Method A Method B Method C Method DEvaluated method

Fig 4 Result of subjective evaluation

Opinion scores

G1-sensor

G2-sensor

R- sensor

B-sensor

Lens

Fig 5 Color separation prism for 4-image sensors

Ｇ１

Ｇ2

Ｒ

Ｂ

G: 4320×7680 pixels equivalently R,B: 2160×3840 pixels

Fig 3 Pixel arrangement on 2G/R/B method

diagonal offset

One pixel interval

7680 pixels

4320 pixels

Fig 2 2G/R/B method

G1

R

G2

B

Original image

2160×3840 pixelsfor each channel

438 — 2004 NAB BEC Proceedings

we have studied to make a camera head smaller. A 2160×3840-pixel CMOS image sensor with the 16 mm by 9 mm image area has been developed for this system [8]. The main features of CMOS, for example, the low power consumption, the on-chip signal processing circuits and the column parallel structure, become remarkable in this application. The pixel size of the sensor is 4.2×4.2 micron squared, smaller than that of the CCD. A 10-bit analog to digital converter is placed at every two columns on the chip, and the 16-column parallel digital output structure is built in. The authors have developed an experimental image pickup system using four CMOS sensors. Figure 6 shows the external

views of the first camera head (80Kg with lens) and this experimental one. The size of the camera head with CMOS image sensors has become almost half that of the first one. The resolution characteristics of the camera is shown in figure 7. The vertical limiting resolution of the system can attain to more than 3200 TV lines successfully. Based on this result, the compact practical camera system will be developed.

2) Display

developed a front projection type display

Silicon) with 2048×3840-pixels in order to obtain the

s the sound with an extra large screen is expected to sation of reality and presence, the

roper “listener’s envelopment”

- nd

Asyst veloped. This system is proposed as the

The authors using four 1.7-inch LCoS panels (Liquid Crystal on

resolution of 4k×8k with a wide screen [9]. Here, the relative positioning of the two LCD panels for green has a great effect on resolution characteristics, and these two panels must be accurately offset by 0.5 pixel. To meet this requirement, we employed two projection units, one for color green, where two green panels are installed and the other for colors red and blue, where red and blue panels are installed. Figure 8 shows the optical system for green projection unit [10]. Polarizing beam splitters (PBS) with phase shifting plates are used to separate and combine G1 and G2. A compact stepping motor is attached onto the G2 panel to make fine adjustment of the 0.5-pixel offset between these two panels. The optical output of the display is approximately 5000 lumen, resulting in the peak brightness on the screen of about 50 cd/m2 with a 320-inch screen whose screen gain is 0.9. (about 40 cd/m2 with a 450-inch screen whose screen gain is 1.5 ) Because there are two projection units for dual-G and R/B, the images from these units are not projected on the same position on the screen without any correction. Therefore a convergence error correction scheme was developed, which converts red and blue images so that the convergence error on the screen is corrected.

3) Audio system Aenhance the sennecessary conditions of sound system for ultrahigh definition video system are decided as follows;

- Stable localization of a frontal sound image at any point over the screen

- Reproduction of a sound image in all directions around a viewer

- Reproduction of three dimensional spatial impression with p

- Wide listening area with adequate quality of sound Compatibility with existing multichannel sousystems

ccording to these conditions, 22.2 multichannel sound em is de

highest sound system in the hierarchy of compatible

G1 panel

G2 panel

PBS PBS

PBS

Projection lens

Phase shifting

White light

plate Green filter

(with stepping mo

of G projection unit

Glight

tors)

Fig 8 Optical structure

reen

Fig 6 External views of the first version camera head (left) and the experimental compact camera head (right).

: Measured value

MTF [%] 100

80

60

40

20

Spatial frequency [TVL]

1000 2000 3000 4000 5000

: Calculated data Lens F-number 5.6

Fig 7 Vertical resolution characteristics of the experimental compact camera head

2004 NAB BEC Proceedings — 439

multichannel sound systems for the theatre reproduct

various multichannel sound ion

including systems such as

nts and interface disc recorder system was also developed. It consists of 16

record approximately

stem e number of HDTV pixels, the

authors sim ts an terfaces between the sing and synchr ultiple devices based on the HDTV form . The video signal smitted through 16 channels of HDTV-SDI (Serial Digital Interface) format signal, in total, ap CONCULUTION An ultrahig ms have a lot of opportunities for various potential applications. T

stem with 4k×8k pixels (16 times that of HDTV) was eveloped to realize nearly the maximum of sensation

of sensation of ality with the experimental system described in this

stration program that uses this

he Quality f Television Pictures, ITU–R Recommendations, BT.500

et al., "Four-Chip CCD camera for HDTV",

The experimental system is shown in figure 10. The syis to have 4×4 tim

5.1, 6.1, 7.1 and 10.2. [11][12] The 22.2 multichannel sound system consists of three layers of loudspeakers; upper layer with 9 channels, middle layer of 10 channels, lower layer of 3channels and 2 channel for LFE (Low Frequency Effects). Figure 9 shows detailed arrangement of loudspeakers. 4) Other equipmeAof HDTV disc recorder units and can18 minutes of the ultrahigh-definition video signal in real time with a capacity of nearly 3.5 TB.

es thtplified he equipmen d in

m by u onising mat is tran

proximately 24 Gbps.

h-definition television systehe

sydof reality. There are 4k×8k pixels equivalently for green, while 2k×4k pixels for red and blue. A spatial pixel-offset method was used for green. The system is still at a basic stage. The performances of the devices must be improved. In order to investigate the possibilities for future broadcast services, the authors will make a quantitative evaluation

Screen

(b) Side view

Upper layer Lower layer

LF

(a)Ground plan

repaper. A demonultrahigh-definition television system will be exhibited in the 2005 World Exposition, Aichi, Japan. REFERENCES

[1]T.Hatada, et al., 1980. "Psychophysical analysis of the sensation of reality induced by a visual wide-field display", SMPTE Journal, Vol.89 pp.560-569 (1980) [2]"High-definition television", Van Nostrand Reinhold、pp.7 1992, [3]A.Kaiser, H.W.Mahler, R.H.McMann, "Resolution requirements for HDTV based upon the performance of 35mm motion-picture films for theatrical viewing", SMPTE Journal, pp.654-659 (1985) [4] B.E Bayer, "Color Imaging Array", US Patent 3971065 (1976) [5]Methodology for the Subjective Assessment of to[6] M. Sugawara SPIE Proceedings, vol.2173, pp.122-129 (1994) [7]K. Mitani, et al., "Ultrahigh-definition color video camera system with 4K-scanning lines", SPIE Proceedings, Sensor and Camera Systems for Scientific, Industrial and Digital Photography Applications IV, pp.5017-18 ( 2003). [8] I. Takayanagi, et al., "A 1 1/4 inch 8.3M pixel digital output CMOS APS for UDTV application", Proceedings of ISSCC2003, pp.216-217 (2003) [9]M.Kanazawa, et al., "An Ultrahigh-Definition Display using the pixel offset method", (under reviewed), 2003. [10] W.P.Bleha, et al., "D-ILA technology for high resolution projection displays", SPIE AeroSense-03, pp.5080-29 (2004) [11] Tomlinson Holman, “5.1 Surround Sound, Up and Running”，Focal Press, Woburn (2000) [12] Francis Rumsey, “Spatial Audio”，Focal Press, Oxford (2001)

Camera

HD-SDI 16ch Camera

control unit

Hard disk recorder

Display processor

R,B

G1,G2

Fig 10 Experimental system

ector

Video material preservation

Proj

Middle layer

E

Fig 9 Louds rrange 22.2 multichannel sound system

peakers a ment for

440 — 2004 NAB BEC Proceedings