Experiments with AS3 Video Compositing

Ok, so it's already pretty cool that you can appear next to your favorite artist using the video record widget but I'm experimenting with other possible features.

Tyler helping Radiohead out

What if my surroundings were removed and I could appear right in the studio helping my favorite band out with my wicked guitar skills? Kind of Holodeck-style. That would be really cool. Professionals do this all the time, normally with the use of green screens or blue screens and a video editing program that can remove the color of the screen so that another image or video can show through. It's called Chroma Key.

Although it's not very hard to make a functional green screen, I can't expect music fans to erect one and light it properly. More likely, they won't want to even change out of their pajamas and will expect the performance to work in their messy bedroom. That's ok, I'll see what I can do.

Instead of subtracting similar colors from the video image, I'm experimenting with subtracting a known background from the video. This requires first getting a picture of the background and then in each frame removing any pixels that are similar to the background shot.

I tried various combinations of BitmapData.compare() and a DisplacementMapFilter but in the end the best results involved processing each pixel in each frame. Yikes... What I really need to do this efficiently is the ability to create a custom bitmap filter. Custom filters are possible but they're new in Flash 10 with Adobe Pixel Bender and well, were not going to require a Flash 10 player just for holodeck support.

Here's a work in progress. It's not smart enough or efficient enough yet and it cuts out part of your body if it happens to be the same color as the background but it's still fun. Oh, and you need a video camera.

I'll post updates as the code improves.

as3 · video compositing