Android Live Streaming Box Technical

MULTI PURPOSE

ANDROID TAB

By Jimmin J Kurichiyil

We are planning to design an android tablet with extra

features required.Since many android tablets are available in

market now,its easy to modify it according to our needs. The

main additional feature is live streaming ,which can be

connected to an external output , where converted high

quality video is displayed from the tablet.Another feature is

landline connectivity, through which we can make calls as

well as use dial up networking.This presentation gives its

emphasis on converting baseline video to high quality one

following a quite new algorithm

Processor: AM335x ARM Cortex-A8 microprocessorFeatures:

ARM Cortex A8 with NEON upto 720mhz , offers robust performance and OS support to enable network connectivity,UI and system control

Flexible power management-7mW in standby,700mW in active mode UI and 3D capabilities-touch screen controller,3D graphics accelerator Integration of flexible peripherals

CAN – connection to sensors,actuators and control devicesGigabit Ethernet two port switch – fast network connectivityDual USB+PHYs-Host and device support allows flexibilityLPDDR1/DDR2/DDR3-multiprogrammingPRU-offers full user configurabilty when desiredADC-connectivity to analog sensors and motor controlCryptographic acceleration-hardware acceleration of data

protection

Price – 5$

Development Board: BEAGLE BONE(from Beagle Board)

Price : 89$

Other features of tab:

Android OS v2.3

Can be used as a normal phone using sim card

Ability to connect to a landline and to use its features

Can stream videos using vlc player using isml files

HDMI out gives high quality video that can be connected to a TV

Internal memory of 8GB

ON/OFF

SWITCH

HDMI

OUT

USB

PORTS

DSL

PORT

POWER

PORT

PORT FOR SIM CARD

LOCATED NEAR BATTERY

INTERFACE

WIFI

LIVE TV

POWER

CALL

CONTACTS

HDMIMESSAGES

USBWEB

BATTERYFILES

KEYBOARD LAYOUT

LIVE STREAMING

Through Windows Azure, we are able to create isml stream points. The media

player named VLC (Video Lan player ) is capable of playing isml files. So for live

streaming, what we need to do is to create a code for linking the isml file which

we want to stream as the play file of vlc.

We will consider the isml file for each channel as different stream points . For

controlling channels, we may modify the code such that vlc can point to different

isml files as we are changing channel. We can use the VLC telecontrol app in our

tab for controlling VLC while watching stream in TV via bluetooth.

• Expression Encoder 4 Pro SP1

• Live Broadcasting Project

• Broadcasting to Live Smooth Streaming Publishing Point of IIS server

• Output Format – IIS Smooth Streaming

• Video – H.264 – Baseline

• Audio – AAC – LC

• Point the output of encoding to : http://ourserver.cloudapp.net/LiveSmoothStream.isml

• The isml File is the publishing point located in the IIS server

http://ourserver.cloudapp.net/LiveSmoothStream.isml

• To perform live video broadcasting through Windows Azure Web Roles.

• Using IIS Media Services 4

• Silverlight player as the client side for viewing streams.

SILVERLIGHT PLAYER

Abstract- We presents a technique for an Android mobiledevice to best utilize its surrounding external display devicesuch as high-definition TV, in order to overcome the size andresolution limitation of the small LCD in the mobile device.Unlikecurrent techniques that simply expand the LCD screen imageto an external large display, our technique dynamically changesthe resolution adapting to capability of the externaldisplay. For this, we first identify technical hurdles in theAndroid platform for dynamic resolution adaptation. Then, wepropose novel mechanisms to effectively address the technicalhurdles.

Mobile devices like cell phones and PDAs set us free from locational restrictions.Moreover, thanks to increasing processor speeds and radio communicationbandwidths, it becomes real for emerging mobile devices to provide generalcomputing and IT services including web browsing, video streaming, and evendocument editing. The quality of experiences (QoEs) of those services arestill very limited ironically because of the most appealing aspect of mobiledevices, that is, “small and thus easy to carry”. To be small and easy to carry,mobile devices cannot be equipped with large and highly capable UI (userinterface) devices. Under this inherent limitation, for mobile devices toprovide high QoEs of general computing like IT services, it is essential to exploitsurrounding gadgets and appliances such as Bluetooth keyboards and HDTVsas their alternative input/output devices depending on their locations. With thisregard, we presents our initial work that makes a mobile device to automaticallyswitch its display to its surrounding high-definition display device

Now take a look at the case of an ordinary hd video converter chip. Here we aretaking AITech’s AIT 1168.

Features : 3-channel 8 bit input digitalizer Multiple input formats Hardware image size scaling for TV size Advanced Flic free 3 channel 10-bit output D/A convertors 80 pin MQFP package

DYNAMIC RESOLUTION ADAPTATIONHDMI CONVERSION

The AIT 1168 simply expand the LCD display to a simply large image. Switching is possible by capturing the display frame buffer and transmitting the captured data to the external display device such that it can display the received data. However, this approach is simply magnifying the same quality image to a large screen as shown in Fig. 1(a). The problem here is that picture quality will not be enhanced. So we are introducing a technique called “DYNAMIC RESOLUTION ADAPTATION.Here we change the resolution of the image content itself adapting to the resolutionof the external display device as shown in Fig. 1(b).

Fig 1.a - simple magnification Fig 1.b-resolution adaptation

For the existing mobile platforms such as Android , however, dynamically changing theresolution is not straightforward because they are designed without anticipating the dynamicchange of display devices. Specifically, they have following technical limitations:

• Hard-coded reference resolution,• Pre-fixed kernel memory boundary for frame buffer, and• No redrawing mechanism after resolution change

We aim at adapting the resolution to fully utilize the capability of the target display, as shown inFig. 1(b). Such an adaptation should be realized in the embedded software platformlevel including Android platform and kernel device driver such that it works for any types ofapplication programs including web browsing, video streaming, and documentediting.

OVERVIEW OF MODIFIED ANDROID STRUCTURE FOR DISPLAY SWITCHINGWe briefly describe our modification of the Android platform for automatically detecting andswitching to an external display device. Fig. 2 shows such modified structure wherewhite boxes are newly added modules, gray boxes are modifications of existing modules, andblack boxes are unmodified existing modules.

The first step is for monitoring the external display devices. For this step, the Plug-InService Daemon uses Network Plug- In library to monitor the beacon signals fromsurrounding display devices such as a HDTV and a computer monitor that can beconnected through wireless networks such as Wi-Fi and Bluetooth. Once the Plug-InService Daemon detects a new display device, it requests the device’s profile thatincludes device name, type, and resolution information. The reported profile is storedand managed by the Profile Manager.

The second step is the choice of preferred external display device. For this step, the UISwitching application is invoked whenever the list of external display deviceschanges. It notifies the user with “new device detection”, “disconnection with anexisting device” through both voice and GUI. This notification allows the user tochoose the most preferred display device. After the choice of external display device,the third step is to actually change the resolution targeting for the selected externaldisplay device.



For this step,the UI Switching module gives the resolution information of the selected

device to the Dynamic Resolution Switching module. Then, the Dynamic

Resolution Switching module conducts all the necessary jobs for resolution change.

The jobs conducted are

• It changes the reference resolution and provides it to the Surface Manager so that

the Surface Manager can draw the screen and fill in the frame buffer referring the new

resolution.

• It commands the Frame Buffer Capture module to capture the proper address space of

the frame buffer according to the new resolution.

• It also triggers the most frontend application to redraw its contents so that the changed

resolution applies immediately.

After this resolution change, the Surface Manager, and hence all the applications

draw their contents into the frame buffer based on the new resolution. This frame

buffer is continuously captured and transmitted to the external display device so that the

best fitting screen images can be displayed on the selected external display device

. PROPOSED MECHANISMS FOR DYNAMIC RESOLUTION ADAPTATION

A.Hard-coded Reference ResolutionThe GUI mechanism of Android is well structured such that all the GUI relateddrawings are managed by the Surface Manager that is responsible for the final screendrawing using the reference resolution. Thanks to this structure the same softwaresystem can be easily ported to different mobile devices with different LCDresolutions. Only thing we have to do is to change (x, y) resolution informationhardcoded in the display driver source code file (in our case, s3c-fb.c) of Androidand Linux kernel source files, and to rebuild the kernel for the target mobile device.

However, dynamically changing the resolution while the system is running is adifferent story. In order to enable the dynamic change of the reference resolution,we propose to make the Dynamic Resolution Switching module manage the referenceresolution. This dynamic reference resolution is modified by the command ofthe UI switching module whenever the user chooses a difference display device. Forthe Surface Manager uses this dynamic reference resolution rather than theoriginal hard-coded reference resolution, we modify the GetResolution() function suchthat it gets the resolution information from the Dynamic Resolution Switching module

B.Kernel Memory Boundary of FrameBufferThe embedded Linux kernel onwhich Android is implemented uses afixed memory area sized of 2 Mbytes forthe display frame buffer as shown in Fig.3. The memory area is written by Androidbased on the application activities. Atthe same time the memory area isaccessed by the display controller viaDirect Memory Access (DMA) so thatthe content of the display frame buffercan actually show up on the displayscreen. Since the display framebuffer is concurrently accessed byboth the “provider”, i.e., Android andthe “consumer”, i.e., displaycontroller, there is a possibility thatwhile the Android is filling up the framebuffer, the incompleted image cantemporarily show up on the displayscreen, which causes displayflickering, tearing, and afterimage.


In order to avoid this problem,Android now uses doublebuffering scheme in which, the display frame buffer area isdivided into two sub-areas, called Buffer 1 and Buffer 2. Atan instant, the provider fills the image content on Buffer 1and the display controller reads and displays the imagecontent from Buffer 2. At the next instant, the roles ofBuffer 1 and Buffer 2 are switched. This flip of thetwo buffers continues. This way, the display controllercan continuously display only the finished images onthe LCD screen.Considering this double buffering scheme, out of 2 Mbytes, we have only 1 Mbytes torepresent one screen image. Since the typical graphic representation use 16 bpp (bits perpixel), the 1 MBytes capacity can hold only 524,288 pixels, which is equivalent to 836 ×627 resolution.

Due to this fixed 2 MBytes frame buffer with double buffering scheme, Androidinherently cannot support a higher density resolution such as 1024 × 768. This is notAndroid’s design error since it is designed targeting a fixed small LCD device on a mobiledevice. However, in our targeting scenario where the mobile device can evenexploit a surrounding display such as HDTV, we have to overcome the limitation.

Our approach uses the entire frame buffer space size of 2 MBytes as a singlebuffer. For this, we change the kernel code related to the display driver so thatboth the provider and consumer view the entire frame buffer space as a single bufferto hold a large screen image. The positive aspect of this change is that the framebuffer can now hold a screen image with a larger resolution like 1024 × 768. On theother hand, the negative aspect is loosing the benefit of double buffering, thatis, avoiding display flickering, tearing, and afterimage.In order to retain the benefit of double buffering with our single bufferingmechanism, we propose a hybrid buffering mechanism as shown in Fig. 5 Inour hybrid buffering mechanism, the mobile device uses a single bufferingbut ensuring mutual exclusive access to the frame buffer with a mutexsemaphore. This way, while the provider, i.e., Android fills in the frame buffer, theconsumer, i.e., the transmitter, waits. Only after the provider completely fills in theframe buffer, the consumer transmits the frame buffer content to the externaldisplay device. This way, we can avoid the scenario that an unfinished image istransmitted. In the external display device such as HDTV with a plenty ofmemory space, we can simply use the double buffering such that the provider, i.e.,receiver fills in the first half of the frame buffer while the consumer, i.e., displaycontroller, is reading and displaying the second half.

C.No Redraw After Resolution ChangeEven after the Dynamic Resolution Switching module changes theresolution as described above, the Android application on top of the screenis simply waiting for a user input without re-drawing its content based onthe new resolution. Due to this reason, the frame buffer content based on theold resolution can be captured, transmitted, and show up on the externaldisplay, which shows a partially filled incomplete image.

Therefore, we need to make a mechanism that triggers the application toredraw the content immediately after the resolution change. For this, wepropose to use a simple but effective approach by exploiting the application’slife cycle as shown in Fig. 6.

Once a user starts an application, it isadded to the application list of so calledan “activity manager” that manages allthe JAVA applications on top of the Dalvikvirtual machine. Then, the activitymanager calls the application’sOnCreate(), OnStart(), OnResume() methodfunctions. The OnCreate() functionconstructs the application’s layoutincluding window size, button positions, etc.based on the reference resolution. Then,OnStart() and OnResume() functions dojobs that are needed in the beginningand after resume, respectively. These twofunctions do not change the application’sscreen image. After that, the application runson the top of the display interacting with theuser changing its display image as needed.

When a user starts a new application, the activity manager adds it in the list and callsOnPause() method of the previous application. If the new application has nothing to do withthe display and stops, the activity manager resumes the previous application by callingOnResume(). On the other hand, if the newly started application uses the display, theprevious application should move to the background and the new application imageshould pop-up to the foreground. For this, the activity manager calls OnStop() of the previousapplication,from which point, the previous application’s content is not visible and the newapplication’s content becomes visible on top of the display screen. If the new application stopsor the user changes its focus to the previous application, the activity manager calls theprevious application’s OnRestart() method function, for the previous application tomove to the foreground. At this point, the OnRestart() method reconstruct the application’slast image just before its previous stop.

Note that OnRestart() method redraws the last image shot of the application and themethod is called when a new application starts and stops. Motivated by this, whenthe resolution changes, our approach is to start a dummy application. The dummyapplication does nothing but simply touches the display and then stops. By this dummyapplication, the most foreground application just before the resolution change stops andrestart, which causes calling OnRestart(). The OnRestart() method redraws the application’slast image referring the changed resolution. By this explicit triggering of redraw after theresolution change, the resolution changed image shows up on the external displayimmediately after the resolution change without showing any partially filledincomplete image.

Concluding,

We can develop an android tablet that is capable ofperforming live streaming along with dynamic videoconversion for viewing on TV.Cost of this project is approximately 5000 rs.We canreduce the cost by mass production. Hope that thisproduct will definitely conquer the market since there isnot at all a product with this much capabilities under 10Krs.

THANK YOU

JIMMIN J KURIHIYIL

Android Live Streaming Box Technical

Technology

Transcript of Android Live Streaming Box Technical