Passing webcam data into WebGL via Flash

Note: To view the demos, you will need to have a WebGL capable browser plus the latest Flash Player (with webcam enabled in your Flash settings). I advise people to download Firefox 4.0 beta, enter "about:config" in the address bar, and change "webgl.enabled_for_all_sites" to true.

I tend to stay away from the HTML5 vs Flash "debate", principally because I think it's inane. Both platforms offer certain advantages over the other and some interesting results can be achieved by utilizing their strengths. Case in point, Flash's built-in webcam support and WebGL's hardware-accelerated 3D graphics are features that only exist in their respective platforms (for now). Bridging the technologies produces something unattainable by only using one, as the following quick examples illustrate.

Simple interactive cube.

Plane deformation
(Kaleidoscope shader by iq ported from Shader Toy).

(Kinderpainter by iq ported from Shader Toy).

Passing webcam data to Javascript involves a few steps. The BitmapData drawn from the Video object needs to be compressed to JPEG and encoded to Base64 for the data URI to read it. This can be a very taxing process in native ActionScript, but is nearly negligible when done in haXe or Alchemy. In this case, I'm using both. The JPEG compression is performed by metalbot's Alchemy JPEG encoder, while the Base64 encoding is handled by Bloodhound's haXe crypto library. ExternalInferface passes the resulting string to the WebGL's texture Image source with the following method:

  1. function fromFlash(event, value) {
  2.   texture.image.src = "data:image/jpg;base64,"+value;
  3. }

Not only is it possible for HTML5 and Flash to peacefully coexist, but they can also help each other achieve a result not possible on their own. Kumbaya.