Under the Hood 2009

8/6/2019 Under the Hood 2009

http://slidepdf.com/reader/full/under-the-hood-2009 1/38



>

03



The Video Lifecycle is an

involved, multi-step process that requires technical solutions and services at every stage.

blinkx is the only technology company to o er a ull suite o

solutions or media owners’ and publishers’ requirements at

every step o the Video Li ecycle.



LOCATE

PROCESS

INDEX

CONTROL

MONETIZE

DELIVER



LOCATE

Today, most video content is captured and storedin encoded digital les. These les may be storedin a private content management solution on asecure, private network or they may be public-ly available on the Internet. The remainder o video content is maintained as legacy archivesin non-digital orm. blinkx has developed the blinkx distributed Video Fetch Server (dVFS)and blinkx o f ine Video Fetch Server (oVFS) totraverse and identi y content in digital and non-digital orms, respectively.

BLINKX DISTRIBUTED

VIDEO FETCH SERVER (dVFS)The blinkx distributed Video Fetch Server is acon gurable server that is capable o inter acing with a variety o video repositories (includingover 50 distinct database and content manage-ment systems) in order to locate and extract videothat is contained within them. The blinkx dVFSis architected as a parallel server that runs multi-ple spider modules which each trawl the reposi-tories to be indexed. While some private systemscan be relatively straight orward to inter ace with, o ering some orm o public inter ace orexport unction, others are extremely complex

to spider. The blinkx dVFS is able to automati-cally aggregate rom all with minimal manualcon guration.



The award-winning blinkx dVFS Web Edi-tion Spider has played a major role in building blinkx’s video index - now the largest in the world. The Web Edition module is available ex-clusively to our customers or spidering any sitesthat are o interest to them.As with the standard dVFS, the Web Edition is

ully parallelized in order to improve scalabilityand is built on a complex process architecturethat supports dynamic resource allocation andmodule sel -replication in order to automati-cally match the scale o the task set be ore it. The

spider trawls link structure, automatically bias-ing towards likely sources o video content in or-der to improve video yield.

In addition to static and dynamically generatedHTML, the blinkx dVFS is capable o analyzingand processing popular Web scripting languages(including JavaScript/AJAX and Flash) and uses variable substitution and code pre-compilingtechniques to in er content that is hidden in dy-namically generated pages.

09



BLINKX OFFLINE

VIDEO FETCH SERVER (oVFS)I stored on tape or disk, o f ine video contentmust be captured and encoded be ore it can be used within a digital medium such as theInternet. The blinkx o f ine Video Fetch Server(oVFS) acilitates this process by inter acing with standard video capture cards in order tocapture streams o analog content rom o f inestores (such as tapes) or directly rom live-air broadcast.

Parallel and massively scalable automated spiders

that intelligently ocus to nd audio/video content Render page ully in memory, allowing videos’ contextual metadata to be indexed

Ability to index any video rom any page,regardless o ormat (including Flash)

Automatic generation o thumbnails, previews and word-timing

Agnostic approach, metadata, video analysis, speech recognition and closed captioning

blinkx dVFS supports MySQL, Oracle and all other common databases and standard C.M. systems

blinkx oVFS inter aces directly with o fine video sources including most tape and disk ormats, satellite, cable and terrestrial broadcast

Native support or over 50 common database and content management systems

10



PROCESS

Once a digital video has been ound or createdrom an analog original, the next step is analysis.

This process involves breaking down the piece o content into several o its constituent compo-nents and processing each one to ully under-stand the video’s overall meaning. The resultingdata can serve as a oundation or unctions suchas search, organization, selection or suggestion.

LOCATEPROCESS

INDEX

CONTROL

MONETIZE

DELIVER



BLINKX VIDEOLOGGER

VIDEO ANALYSIS MODULE blinkx processes video content using a multi-threaded server technology known as the Vid-eoLogger. In constant development or the pastten years, blinkx’s VideoLogger is a central man-agement service that is responsible or marshal-ling each piece o content through one or moreanalysis modules that extract in ormation.

The VideoLogger can be used in isolation but is usually coupled directly with the Video FetchServer (VFS) modules.

The VideoLogger analysis modules use a variety o advanced image and audio analysis techniques toautomatically extract in ormation about a videoin real-time, acilitating the creation o a rich

index that accurately describes the video content.

This precise, time-stamped index provides ne- grained access to the video content that can be used to e ciently search and locate a speci c video segment or playback. Used together, theVFS and VideoLogger modules can simultane-ously index and digitize (encode) input contentto trans orm the video and audio assets intoaccessible, Web-ready content.

Existing methods o making audio and videosearchable rely on either textual metadata (added by pro essional editors or end-users as part o a

‘tagsonomy’) or closed caption data that is addedduring the television production process. Botho these approaches are signi cantly f awed.

12



Metadata is descriptive in ormation (e.g., sum-

maries and tags) created by a video’s original edi-tor but o ten omits aspects o a video that may beo interest to others.

Typically, it is only applied on a per-video basis, o ering generic summaries o a video, andis there ore an ine ective method o providing users with precise, granular descriptions o thesubtle details in a clip. Additionally, the prac-tice o ree tagging, especially when opened to acommunity, is prone to spamming - where users

alsely apply descriptors to content to subvert thesearch process.

Closed captioning is f awed primarily because it

is generated by human transcribers who can su -er rom high error rates. Furthermore, closed

captioning is extremely rare on the Internet;recent research by blinkx suggests that less than0.001% o all Internet video content contains anyclosed captioning. Even in cases where closedcaptioning exists, the majority o these videosonly have basic titles that mark a content seg-ment’s beginning and end - rather than a com-plete transcript.

I they exist, the blinkx VideoLogger does ex-tract and use metadata and closed captioning as

the rst step in the indexing process. In addition, blinkx’s technology utilizes advanced speechrecognition and visual analysis techniques toanalyze and understand the spoken word and



visual content o an audio/video le, ensuring unparalleled comprehension o online multi-media content.

The VideoLogger can also control the encodingprocess using third-party encoders that outputin popular ormats including MP4, Flash, Realand Windows Media. Using an output module,the VideoLogger controls both indexing andencoding processes to ensure synchronization between the metadata captured rom the videoasset and the associated digital le. The outputo these analytical modules are stored as urther

metadata tracks alongside the digitally encodedcontent itsel ; not only does blinkx know what was said, blinkx knows exactly when it was said.

Once the video or audio stream has been indexed,encoded and analyzed, the digital video les andthe descriptive analysis output are stored in the blinkx video index.

14



AUDIO ANALYSIS MODULE

blinkx’s audio analysis technology uses advancedstatistical methods to deal with all aspects o processing the digital audio signal rom anaudio or video stream. It employs a wide rangeo recognition technologies – rom keyword-and phrase-spotting to continuous vocabularyspeech, speaker and language recognition.

In order to analyze the spoken words o an audioor video stream, blinkx uses audio analysis tech-niques that are based on neural network tech-nology and Hidden Markov Models (HMMs) toconstruct an e cient acoustic model that can

provide a ast, accurate and dynamic solution within rapidly changing acoustic environments,such as radio and television.

The technology is based on decomposing digi-tized speech into its phonetic constructs. Thephonetic sequence is then analyzed in conjunc-tion with the acoustic model and statistical prob-abilities to calculate the most probable sequenceo words and utterances.

LARGE VOCABULARY RECOGNITION

Unlike traditional speech recognition systems which have xed vocabularies, blinkx supportslarge text corpora, including hundreds o mill-ions o words that can train the system andre ne its accuracy according to the speci c

requirements o its customers.



Using patented predictive technology, blinkx’s

speech analyzer can o er users the bene ts o a large vocabulary speech recognition system without the overhead o a vast search space.Rather than relying solely on existing metadatato describe an audio or video clip, blinkx has theability to retrieve a wide range o multimediacontent based on the words spoken in the televi-sion or radio clip.

SPEAKER INDEPENDENCE

Whereas other approaches require training datarom speci c speakers to realize their ull poten-

tial, blinkx per orms consistently well across a

wide variety o previously unrecognized speechsources. Transcription o speech and segmenta-tion by speaker requires no initial per-speakertraining because blinkx’s underlying technology

was developed as a tool to maintain inter-speaker

independence - not to be a single user transcrip-tion tool.

NON-DICTATED SPEECH

In ormation eeds, such as news broadcasts andradio, are o ten di cult to transcribe due tonoisy conditions and less-than-per ect articula-tion. blinkx’s sophisticated signal processing andstatistical techniques enable the transcriptionengine to lter out extraneous noise, compensate

or low volume levels and probabilistically pre-dict intended dialogue.



PHONEME-LEVEL PHRASE

AND WORD SPOTTING blinkx breaks down all orms o speech intophrases, words and phonemes (the smallestsound units in a particular language), o eringan exceptional granularity o understanding.

MULTIPLE LANGUAGE

MODEL ARCHITECTURE blinkx’s core technology is entirely languageindependent, enabling multiple languages to besimultaneously processed and searched. blinkxcombines both phonetic and conceptual methodsto disambiguate the limitations inherent in tra-

ditional approaches, combining more accuratelanguage recognition with e ective in orma-tion retrieval. Besides also supporting traditionallegacy techniques such as keyword-spotting and

Boolean protocols, blinkx VideoLogger enables

users to search audio data rom a range o sources using multilingual natural language queries.

IMAGE ANALYSIS MODULE

blinkx VideoLogger’s advanced video captureand analysis technology also utilize neural networks and HMMs to optimize the encoding o content in real-time. A comprehensive rangeo media analysis plug-ins allow or the auto-matic creation o metadata and the ability tosearch entire media streams or clips by a range o parameters such as audio, scene, speaker, loca-tion, key rame, image, on-screen text, ace,

token and concept.

17



By making video easy to identi y, locate andre-use, blinkx’s VideoLogger allows the elementsto be assembled and repurposed aster and with greater accuracy than ever be ore. blinkx is capable o a wide range o intelligent video analytics

unctions, including:

LOGO AND SCENE-CHANGE DETECTION

The blinkx VideoLogger is capable o automati-cally detecting, analyzing and interpreting allactivity within video data and can, or exam-ple, interpret and understand the signi cance

o speci c images or note scene changes. Usingadvanced techniques, the blinkx VideoLoggeridenti es and categorizes objects in a scene bysize, shape, color, speed, direction, location and

time o day. Additionally, it utilizes techniquessuch as comparing object histories, motion detec-tion, object sizing, object tracking, object count-ing and behavioral analysis; putting each objectand motion in context.

FACIAL IDENTIFICATION blinkx’s video analysis also o ers power ul bio-metric identi cation tools to enable acial recog-nition. However, traditional 2-D acial recogni-tion has undamental limitations with regard toposture, expression and lighting. The blinkx Vid-

eoLogger employs superior three-dimensionalrecognition techniques rather than 2-D acialmatching processes or optimum per ormance.



ON-SCREEN CHARACTER RECOGNITION

Neural networks based on optical character rec-ognition techniques allow the blinkx Video-Logger to support advanced character recogni-tion. Unlike template-matching used by othersystems, which is dependent on receiving high-quality images, blinkx’s visual analysis techniquesprovide much greater tolerance or matchingpoorly-de ned characters.

With the ability to integrate with multiple databases and automatically cross-re erence and cor-relate identi ed characters with other data,the blinkx VideoLogger o ers the most sophis-

ticated, comprehensive, end-to-end solution which encompasses every aspect o characterrecognition together with advanced recording,retrieval and analytical capabilities.

19



INDEX

When a user requests a speci c piece o contentor a suggestion o something new to watch, thatrequest is processed by the blinkx Index.

The Index uses a complex, multi-dimensional,pattern-matching process to compare the requestto its records on each piece o available content,and then uses its ndings to create a list o themost relevant suggestions. These suggestions are

ed back to the user, either in an ordered resultslist that can be organized in a number o ways,or by creating a channel or playlist o contentpieces that can be consumed sequentially.

The blinkx Index is a plat orm-indepedent serv-er that typically runs as a single virtual service

LOCATE

PROCESS

INDEX

CONTROL

MONETIZE

DELIVER



SEARCH FUNCTIONS

These unctions allow a user to explore andsearch the blinkx Index with a high degree o control and accuracy. blinkx’s technology sup-ports keyword search, Boolean search, concep-tual search, automatic hyperlinking, elded ormeta-search, ederated search, parametric searchand guided-navigation.

keyword boolean phrase conceptual contextual

hyperlinked parametric guided clustered

22



COMMUNITY FUNCTIONS

With the Community unctions, a blinkx-pow-ered system can in er pro les based upon theconsumption and creation o content by usersand groups.

blinkx supports both implicitly generated pro-les that are built on automatic observation

o user actions, and explicit pro ling based on user-driven pre erence setting and training. Inall cases, blinkx understands that individual us-ers can have many diverse interests - blinkx pro-

les are multi- aceted by design.

These pro les can be used to support a wide va-riety o recommendation strategies: alert usersto new content that will be o interest to them,automatically identi y users who are topic

‘leaders’, suggest user groups based on those whohave similar interests and recommend content based upon the creation, consumption and shar-ing o content.

- thematic clustering - explicit and implicit pro ling - alerting - community suggestion- group selection- audience/content demand pro ling

SORTING AND FILTERING

Regardless o the unction used, blinkx’s Indexresults lists can also be sorted and ltered in anumber o ways:



SORTING

BY CONTENT PROVIDER:

pre erential weighting o content rom speci-ed sources over all other content providers in

an index

BY RELEVANCE OR DATE :

view content results based on best possible matchto search query or reshness o content

BY ARBITRARY METADATA :

sort a results list by any o the elds o metadata which describe each piece o content (e.g., sort byauthor, security clearance, number o commentsor number o views)

FILTERING

SAFE FILTER:

pre-populated lter which blocks inappropriatecontent in order to acilitate amily- riendlyresults lists

QUALITY FILTERS:

ensure that only content o certain quality levels(measured by rame rate, resolution, bit rate ordestination network latency) are returned

FORMAT FILTERS:

block content encoded in ormats the user doesnot wish to view, allowing or return o lesonly in Flash, Windows Media, RealPlayer anyrequired combination o content ormats



MONETIZE

Persuasive automated advertising must combinehistorical, demographic knowledge o a user with an understanding o the content that he orshe consumes at a given moment in time.

Put simply, knowing who is watching what and why makes it easy to select highly relevant advertising. blinkx’s ad plat orm, AdHoc, achievesthis by rst capturing these inputs and then au-tomatically synthesizing them to deliver a selec-tion o advertising that aims to best monetize aparticular user or content event.

LOCATE

PROCESS

INDEX

CONTROL

MONETIZE

DELIVER



Once relevant ads have been identi ed, AdHoc

can deliver them in a variety o ways.BLINKX AD DELIVERY FORMATS

PRE-, POST- AND INTERSTITIAL-ROLL

These popular ads are ull-screen video ads thatare played be ore, a ter or during a piece o con-

tent (respectively). Though extremely arrestingas they play directly in the video player window,roll ads are less popular because they inter ere with the clip playback experience. Roll ads arepriced based on an impressions model.

BLINKX UN-ROLL UNIT

The blinkx Un-roll unit allows the viewer toengage with a brand continuously throughout a video. The experience begins with a branded cur-tain that draws back to reveal the video. As the video plays, touch-points such as overlay ads andlogos appear at contextually relevant moments within the video, made possible by blinkx’s Ad-

Hoc technology. The video ends with a clearcall-to-action and the viewer has the option tocontinue to the advertiser’s Web site.

OVERLAY ADVERTISING

Like user-initiated ads, overlays appear in the video player. In contrast, however, overlay adsare display advertisements in themselves, con-taining graphical elements and a orm o mes-saging or call-to-action that is always visible.



ways the video can be searched and retrieved.

Like traditional text-based technologies, blinkx generates textual in ormation about standardtitles, categories, and user-created tags, but inaddition, blinkx actually listens to, watches, andreads video content.

This means blinkx has the power to analyze and

process not only textual content, but also audioand visual video components, using speech rec-ognition and visual analysis technologies. Theseprocesses greatly increase the number o wordsassociated with a given video, thus driving moretra c to it. blinkx’s enhanced sources o de-scriptive data also enrich SEO content by allow-ing Web developers easy access to video content,no in rastructure involvement necessary.

ENTITY EXTRACTION

blinkx is able to generate a massive amount o textual in ormation about a given piece o video,so it’s critical to be able to re ne searchable in or-mation into the most relevant descriptive unitsor entities, that describe the video’s most basiccomponents. blinkx’s technology automaticallyand accurately assesses context and pulls out key

words, so publishers attract the most relevantpossible audience to their video.

Video entities such as names, statistics, and lo-cations are extracted rom in ormation associ-ated with a video object, like comments, titles,tags, and audio. A ter identi ying these entities, blinkx tags a publisher’s content with broad de-scriptions o the video’s content.



These descriptions automatically populate theconventional components o a Web page, like ti-tles, html, or optimized URLs, to make a page asaccurately searchable as possible. Additionally,these extracted entities can be used to providesuperior navigation via site structure, categori-zation, and taxonomical integration.

CONCEPTUAL UNDERSTANDING

For optimum SEO, it is essential to understandthe key concepts or themes associated with a vid-eo’s content, because they determine relevanceand ultimately drive navigation. In order to es-tablish these concepts, blinkx automatically ex-tracts and indexes textual in ormation about the video to urther its conceptual understanding.

Video assets and generated tags are then scruti-nized to identi y overall concepts, to yield moreaccurate, relevant results than are possible withkeyword-based search technologies.

By recognizing complex concepts within the video, rather than simply scanning in erior tag-

ging, blinkx determines relevance based on actu-al content, not subjective human interpretation. blinkx delivers advantageous ways or searchengines to drive relevant audiences to publish-er content by placing it in context, whether insearch results or associated links.

31



33



BLINKX’S MOVINGTHUMBNAIL GENERATION

blinkx’s patented Moving ThumbnailGeneration technology analyzes every incoming video le and creates a number o visualthumbnails – short, compressed video segmentsthat represent di erent points in time o a givenclip. Thumbnails are generated either arbitrarily

(e.g., every minute, every 10 seconds, at thestart and end, etc.) or, more typically, based onspeci cally identi ed events within the video(e.g., the utterance o key words, the appearanceo a amous ace, etc.). Later, when a video islisted as relevant to a user’s search, blinkx notonly returns the textual summary and title o therelevant video, but it also displays the MovingThumbnail that most closely demonstrates whya given video is relevant to the search.

For example, in the case o a longer orm video

that covers more than one topic, the Thumbnail will eature screenshots related to the searchquery. A user is there ore able to swi tly viewand assess how relevant a given video is to their.

STEP 2: DELIVERY OF CONTENT

In-browser streamed video is an inexpensivemethod o delivering content via popularstreaming technologies such as Real streaming

ormat, Windows Media, Flash Audio/Videoand Apple’s Quicktime. blinkx supports all o these ormats and, in addition, can deliver in- browser video using a number o lesser-knownstandards.Direct streaming o video, on the other hand, isan expensive process.

33

Under the Hood 2009

Documents

Transcript of Under the Hood 2009