With the Disney Research Lab’s as-yet-unnamed augmented-reality coloring-book app, a tablet’s camera watches children coloring a Disney character on paper. As seen in the video above, before they can say “bibbity-bobbity-boo,” the character springs to life in 3-D on the tablet’s screen.
There’s a lot of sophisticated science required to make this amazing thing happen.
For one thing, the app has to somehow guess the character’s texture to make a decent 3-D version. So it takes its best guess early on in the coloring process, mapping out the character’s probable colors in pixels. Then, using what the lab is calling “virtual springs,” the app slowly relaxes the original mapping of the pixels to match what the budding artist finally colors.
Another wrinkle is that coloring book pages don’t actually lay flat—pages slope upward, away from the coloring book’s binding. This makes scanning the artwork with any accuracy a challenge. The app compensates by using a process Disney calls “deformable surface tracking.” As Disney’s tests using focus groups in the video show, when the app is finally shipped, it might just seem like magic.
