The Dancing Bear Problem

There’s an old circus act where a bear shuffles around on its hind legs. The audience goes wild—not because it dances well, but because it dances at all.

That’s how I think about a lot of AI demos.

I’ve been running two kinds of tests with AI. The first is what I call the “dancing bear” test. You put in a strange prompt and you’re happy you get anything out. When I started my AI-assisted novel, this is where I was. I’d give it my characters, setting, and situation, and be delighted when it produced anything legible. Look! The bear can dance!

The second is the prima ballerina test. You ask it to do something you actually need done, at the quality you actually need. It’s no longer impressive that the bear dances at all—it needs to dance better than you can. With the novel, that meant good writing. Interesting writing. Deep character behavior. Complicated plots that twisted and turned. For that, I had to take over and do it myself.

I wanted to see if my students could use AI to cheat on sketchnote assignments. First test: “Sketchnote the book Switch by the Heath Brothers.” Pretty famous book, tons of existing sketchnotes online. The AI did fine. Then I tried an obscure academic paper—the kind I actually assign. Completely incompetent. It didn’t know enough about turning words into pictures and laying them out intelligently. It could handle the famous book because it had seen examples. The real work? No.

Same thing with coding. Put in “Make me Tetris where all the pieces are fruit” and it’ll turn something out. Exciting! But play it for a bit. Is it actually fun? That’s something a human can evaluate and fix. Try to launch it into production? The code falls apart. But for “I have a wacky idea and want to prototype it quick to see if I or my users are interested”? Magical.

Same tool. Dancing bear in one case, ballerina in the other.

So stop testing AI with wacky prompts. Test it with your actual work—the things you actually need done at the quality you actually need. That’s how you figure out where it’s a bear and where it’s a ballerina.​​​​​​​​​​​​​​​​