can testing ever replicate the live environment ?
This one time at UVA (I know, I know) — we landed a gig doing visuals at a fashion show.
Pretty simple concept : models walk down catwalk wearing clothes, we shoot them (capture video!) from the side, process the live stream, and project it onto the wall behind them.
We developed a staggeringly beautiful and accidental technique that analysed the visual edges in the image, and randomly drew a bunch of coloured triangles between them. The “accidental” part of the artistic process was leaving my laptop in a taxi on the final day of rehearsal, forcing us to use an older, simpler version of the look we’d built.
Video evidence survives, amazingly.
I wrote the edge detector with a maximum edge count of 1024 — it just seemed like the right size. This information will be important later.
We did a ton of rehearsals and tests in the shop, with a camera and a projector and people walking past them. Everything worked fine.
Come the show itself — the venue was all decked out, we ran a successful rehearsal, the audience assembled and took out their clipboards, the house lights went down, synth chords boomed from the speakers and the first model strutted her way down the catwalk.
She walked past the camera, the silhouette appeared, and it looked great.
At which point everyone in the audience took pictures with their cameras.
The projection surface we’d rehearsed with was just plain black curtains.
This surface was reflective, shiny, black, crinkly, something else.
The light from the fifty flashbulbs hit the reflective, shiny, black, crinkly something else, and reflected back a gigantic explosion of interconnecting white lines into the camera, and thence into my edge detector with a maximum edge count of 1024.
We found out four things, in very short order :
- images of crinkly surfaces lit by lots of flashbulbs have more than 1024 detectable edges. A lot more.
- this will cause the program to crash in such a flagrant and explosive way that the machine itself will decide to restart.
- the resulting PC reboot screen, when projected across a bunch of hot models in amazing clothes in a hip East London warehouse across the road from a fish market, will look so damn edgy and future that everyone will clap.
- you will still get fired.
So no, testing can’t replicate the live environment, because the live environment is live. It’s real, messy, anything-can-happen live. The best you can do is to make lots of shows yourself, if you possibly can. And actually show them to actual people. With flashbulbs.
If they like it, you passed the test. Even if you get fired.