In Video Advertising at the Lumières’

Why in-video context inference is so difficult

Google AdSense changed the world of online advertising, using standard Information Retrieval techniques to match ad inventory with web pages. This has opened new possibilities for advertisers to get a better bang for their buck and send the right message to the right audience. Can the same be done for video? Easier said than done.

Web pages lend themselves nicely for textual analysis. Not only can content be stemmed, counted for term frequency and enhanced semantically, cues in an HTML document such as headings, page title and metadata provide a much richer base for context inference.

Video, on the other hand, is a much tougher nut to crack. It is unstructured to begin with and has much fewer metadata elements that could be of any use in drawing semantic conclusions. Nevertheless, video is what web users are now consuming more than ever. The number of clips served daily by video sharing and premium content sites is staggering, and so is the ever-increasing time users spend on online entertainment.

It is, therefore, not surprising that more advertising dollars are now being diverted to online media on the expense of traditional TV campaigns. However, the fundamental question still begs – can this money be spent more effectively delivering not only the right message to the right audience, but also at the right time?

Time introduces the most challenging factor in video advertising. While concepts discussed on a web page are linear for reading but parallel to access (the whole page shows as one piece in the browser), scenes in a video are temporal and can only be consumed sequentially. Not only do we need to wait for content to be streamed to our client before we can scan it, skimming over the timeline is not quite the same as scrolling up and down a web page. To this, lets add lack of anchors and in-content links to realize that inter- and intra- navigation in a video space is very different from navigation in hypertext.

So how can advertisers tap into this elusive medium and deliver contextual messaging? Currently, with great difficulty and without much success. One can obviously use the surrounding textual information of the embedding page as well as look at the video title, description and tags to make some assumptions as to what the video is all about; but that would be like drawing similar conclusions based on a whole web site rather than at the page level – to use a web analogy. Nothing in and around the video can actually tell us much about the particular scenes or even define when those start and finish.

Better tools are, therefore, needed - if visual data is of not much help then how about audio? That, indeed, is the flavor of the month in contextual video advertising with companies using speech-to-text techniques to transform audio-visual data to temporal text signals to drive good old semantic analysis. Alas, verbal information only captures a small fraction of what actually goes on in a video. It is enough to consider narration over background visuals in a documentary or the stylishly rich details of a music video clip to see how so much information is lost in this translation.

There is no silver bullet to automate this just yet. No clever algorithm – trained, unsupervised, adaptive or other – can come close to an average person’s ability to easily describe what they see in a video. Any person, you say? Why not use plenty of those then? By the laws of large numbers, they’re bound to reasonably describe it collectively.

Stay tuned for the next post on crowd-sourcing context inference in video and why advertisers should care.

No tags