Illucidation is a series of beginner-friendly programming editors for designing interactive lighting for theater performances, wearable costumes, and other lighting projects that I developed during my thesis. This work builds on research I did at Microsoft (see ARcadia).
Interactive lighting systems controlled by tangible and gesture-based controls are popular in live performance. However, designing interactive lighting systems is complex and requires knowledge of complex programming and electronics, forcing artists to pay top dollar to engineers to develop these systems for them (if they can afford it).
The currently available DIY tools are expensive, lack the ability to customize the technology to be authentic towards users' goals, and are challenging to use.
Empower non-technical artists to more easily program complex lighting systems by designing low-cost and usable tools.
I led a focus group with 11 dancers to explore their interests in incorporating technology into their performances. First, I led a discussion about their current performance practices. The majority of the group was unfamiliar with interactive technology and design, so I brought a few demos of wearable and tangible sensors that triggered some type of output like lighting, sound, vibration, movement, etc. This helped spur further discussion and questions.
In March 2018, a dancer contacted me to help her develop wearable technology for a performance. She wanted LED costumes that animated and changed colors based on the choreography.
This collaboration helped me identify challenges novices face programming costume LEDs, as I could record what she was able to accomplish on her own, and when she needed my assistance.
As an investigation into the specific challenges artists face while using the currently available UIs and tools, I hosted a workshop for 12 artists to build interactive lighting projects for small artistic projects.
While the micro:bit simplified the electronics aspects of their projects, participants ended up with extremely long and difficult to debug programs when programming lighting animations. In addition, one participant said she wished she had a more "visual" way to program the lights instead of writing a lot of code.
As my fourth investigation method, I performed a usability analysis on currently available low-cost DIY tools for building interactive stagelighting systems as an expert. I have a background in Computer Science and am a musician.
I found that even for an expert, building a system that used physical sensors to trigger stagelighting animations was an extremely complex task. It took me over a week to design a working system.
Based on my work with the MakeCode team at Microsoft Research (see ARcadia), and work done by researchers in the field of Computer Science Education, I identified that a block-based programming language that served as higher-level abstractions for stagelighting effects would be useful and usable to beginners.
I designed a blocks-based programming API that enables users to interface with physical computing sensors on the micro:bit (a popular computer science education tool), and use sensors to trigger lighting patterns for DMX stage lights. Users do not need to worry about electronics or writing low-level code, and the event-driven nature of the blocks (i.e. "when button is pressed, turn this light blue") makes it easier for users to create custom interactive systems.
Using the proof of concept prototype, I hosted a small workshop with three artists and musicians to program interactive stagelights.
Conducting user tests with artists and musicians informed me that while Illucidation made it easy to use physical sensors and controls to trigger lighting changes, it was still difficult to programming lighting animations. Even when making simple lighting animations, users ended up with extremely long programs that were hard to debug. Participants expressed wanting a more "visual" way of planning out their animations.
Using the old tools to build interactive lighting would be impossible or unreasonably time consuming for artists and musicians without programming or electronics backgrounds. Overall, Illucidation enables designers to use visual programming paradigms and high-level block abstractions to build these systems with a minimum of two lines of code.
Code Against the Machine: Beginner-Friendly Physical Computing Technologies for Performance
Master's Thesis, University of Colorado Boulder 2019
Code Against the Machine: Design-based Research Towards More Equitable Computational Tools for Performing Artists
In Proceedings of the 2019 ACM Conference on International Computing Education Research