Illucidation

Illucidation

Illucidation is a series of beginner-friendly programming editors for designing interactive lighting for theater performances, wearable costumes, and other lighting projects that I developed during my thesis. This work builds on research I did at Microsoft (see ARcadia).


Dancer on trapeze with sparkling LED costume.
The costume's LEDs sparkle faster as the dancer spins.
Problem:

Interactive lighting systems controlled by tangible and gesture-based controls are popular in live performance. However, designing interactive lighting systems is complex and requires knowledge of complex programming and electronics, forcing artists to pay top dollar to engineers to develop these systems for them (if they can afford it).

The currently available DIY tools are expensive, lack the ability to customize the technology to be authentic towards users' goals, and are challenging to use.

Goal:

Empower non-technical artists to more easily program complex lighting systems by designing low-cost and usable tools.

User Research:
Focus group with dancers

I led a focus group with 11 dancers to explore their interests in incorporating technology into their performances. First, I led a discussion about their current performance practices. The majority of the group was unfamiliar with interactive technology and design, so I brought a few demos of wearable and tangible sensors that triggered some type of output like lighting, sound, vibration, movement, etc. This helped spur further discussion and questions.

Dancers exploring the interactive technology.
User Research (cont.):
Collaborating with an artist on a wearable technology performance

In March 2018, a dancer contacted me to help her develop wearable technology for a performance. She wanted LED costumes that animated and changed colors based on the choreography.

This collaboration helped me identify challenges novices face programming costume LEDs, as I could record what she was able to accomplish on her own, and when she needed my assistance.

Notes generated by the dancer and me.
Dancers wearing the light-up costumes during rehearsal.
User Research (cont.):
Creative lighting workshop for artists

As an investigation into the specific challenges artists face while using the currently available UIs and tools, I hosted a workshop for 12 artists to build interactive lighting projects for small artistic projects.

While the micro:bit simplified the electronics aspects of their projects, participants ended up with extremely long and difficult to debug programs when programming lighting animations. In addition, one participant said she wished she had a more "visual" way to program the lights instead of writing a lot of code.

A project made by a lighting workshop participant.
User Research (cont.):
Expert usability analysis of available DIY tools

As my fourth investigation method, I performed a usability analysis on currently available low-cost DIY tools for building interactive stagelighting systems as an expert. I have a background in Computer Science and am a musician.

I found that even for an expert, building a system that used physical sensors to trigger stagelighting animations was an extremely complex task. It took me over a week to design a working system.

Me using cardboard/aluminum foil foot pedals to change stagelights.
Design and Prototyping

Based on my work with the MakeCode team at Microsoft Research (see ARcadia), and work done by researchers in the field of Computer Science Education, I identified that a block-based programming language that served as higher-level abstractions for stagelighting effects would be useful and usable to beginners.

I designed a blocks-based programming API that enables users to interface with physical computing sensors on the micro:bit (a popular computer science education tool), and use sensors to trigger lighting patterns for DMX stage lights. Users do not need to worry about electronics or writing low-level code, and the event-driven nature of the blocks (i.e. "when button is pressed, turn this light blue") makes it easier for users to create custom interactive systems.

A program that uses my programming blocks. This code initializes a stage light and animates it when a button is pressed.

Evaluation:
User testing with the stage lighting tools

Using the proof of concept prototype, I hosted a small workshop with three artists and musicians to program interactive stagelights.

Testing Illucidation with a group of artists and musicians.
Materials set up for the user testing session.
A user's project. The lights change when her drum sticks touch.
Design Revisions

Conducting user tests with artists and musicians informed me that while Illucidation made it easy to use physical sensors and controls to trigger lighting changes, it was still difficult to programming lighting animations. Even when making simple lighting animations, users ended up with extremely long programs that were hard to debug. Participants expressed wanting a more "visual" way of planning out their animations.

Low-fidelity mockup of the visual layout editor.
Node / React lighting editor in action.
Programming blocks users could select to trigger their lighting patterns.
How a user would program a button press to trigger a lighting animation.
Conclusions and next steps

Using the old tools to build interactive lighting would be impossible or unreasonably time consuming for artists and musicians without programming or electronics backgrounds. Overall, Illucidation enables designers to use visual programming paradigms and high-level block abstractions to build these systems with a minimum of two lines of code.

Publications

Code Against the Machine: Beginner-Friendly Physical Computing Technologies for Performance
Annie Kelly
Master's Thesis, University of Colorado Boulder 2019

Code Against the Machine: Design-based Research Towards More Equitable Computational Tools for Performing Artists
Annie Kelly
In Proceedings of the 2019 ACM Conference on International Computing Education Research

Other Media

A blog post I wrote working with an early prototype of this tool.

A description of my related graduate research on CU Boulder's ATLAS Institute website.