Dances with robots (with Catie Cuan)
Spencer Greenberg speaks with Katie Kwan, a PhD in mechanical engineering, about her unique path combining dance and robotics. They explore how dance principles inform robot motion design, the future of human-robot interaction in VR and daily life, and the collaborative role of humans in AI-generated art.
Deep Dive Analysis
14 Topic Outline
Catie Cuan's Journey: From Dance to Robotics Research
Defining Choreographic Interfaces and Robots
The Foundational Importance of Dance and Movement
What Roboticists Can Learn from Dancers
What Dancers Can Learn from Roboticists
Practicalities of Programming Robot Motion
The 'Time to Compile' and 'The Loop' Art Projects
Understanding Why Robots Frequently Break Down
Using Virtual Reality for Robot Control and Empathy
Leveraging Simulation and VR for Robot Training
Reasons for Limited Robot Presence in Everyday Life
Emerging Applications: Consumer and Space Robotics
The Indispensable Role of Humans in Robot Systems and AI Art
Debate: AI-Generated Art vs. Human Creativity
8 Key Concepts
Choreographic Interface
A system or device that infers information about human motion from its outputs, such as an Apple Watch detecting a thumbs-up gesture based on wrist orientation, essentially 'choreographing' or interpreting movement.
Robot (Catie's Definition)
A system characterized by its ability to sense the environment, process sensory information with a computer, and use actuators to move or act physically based on that analysis.
Serial Manipulators
The most common type of robot, consisting of a series of connected joints (e.g., prismatic, revolute, spherical) arranged in a chain, allowing for complex movements.
Joint Space Programming
A method of programming robots by directly commanding the position or value of each individual joint, either simultaneously or in sequence, to achieve desired movements.
Cartesian Space Programming
A method of programming robots by specifying a desired XYZ position and orientation for the robot's end effector (e.g., hand), with the robot automatically calculating the necessary joint movements to reach that target efficiently.
Teleoperation
The remote control of a robot, often facilitated by interfaces like virtual reality, where a human operator's movements are scaled and translated to direct the robot's actions, creating a sense of embodying the robot.
Sim-to-Real
The process of training robots extensively in virtual simulation environments and then transferring the learned behaviors to physical robots in the real world, which presents challenges in accurately translating simulated physics and environmental conditions.
Human in the Loop
A system design where a human operator actively monitors or intervenes in a robot's operation, providing guidance or taking over control when the robot encounters confusion, errors, or requires assistance, as seen in delivery or surgical robots.
13 Questions Answered
It's a system, like the Apple Watch, that infers information about human motion from the data it collects, essentially 'choreographing' or interpreting movement.
A robot is something with sensing capabilities, a computer to analyze sensory data, and actuators to move or act physically in the environment based on that information.
Humans are one of the few species that dance, using rhythmic, repetitive motion to codify nonverbal group dynamics and social bonds across all historical societies; this makes movement inherently meaningful, a critical lens for understanding and planning robot motion.
Roboticists can learn from dancers' extensive vocabulary for describing and creating motion, exploring a wider, more experimental design space for robot movements, including 'ugly' or less explored types of motion.
Dancers can learn to apply their knowledge of motion to new platforms and bodies beyond the human form and traditional stage, expanding the definition of dance to include interactions with technology like robots and wearable devices.
Robots are programmed by commanding individual joints (joint space) or by specifying a desired end-effector position in space (Cartesian space), using trackpads, software interfaces, or even human body movements, which ultimately translate into low-level joint torques for motors.
Robots often break due to software errors (bugs, incorrect drivers), hardware malfunctions (overheating), or poor internet connectivity, which can hinder real-time updates and performance.
VR is used for teleoperation, where a human operator in a VR environment controls a robot's movements, often through position translation of controllers, and can receive haptic feedback, making it feel like embodying the robot.
Robots are trained in virtual simulations to practice tasks like picking up thousands of different objects, generating vast amounts of data and learning policies much faster and more cost-effectively than in the real world, though translating this 'sim-to-real' can be challenging.
Robots are expensive due to high-precision sensors and actuators, they struggle to achieve 99.9% robust performance in varied environments, and compelling, widespread use cases beyond industrial or specialized applications are still emerging.
Consumer applications include elder care robots (e.g., companion robots like Paru, assistance for standing/mobility, cooking robots), while space applications involve robots like the Canada arm for ISS maintenance or new technologies for space junk removal.
Yes, humans are frequently 'in the loop' for robot operation (e.g., remote operators for delivery robots, surgeons for medical robots) and for training machine learning models by providing feedback, tuning parameters, and defining objectives for AI-generated art.
The speaker argues that AI-generated art is a fallacy because human beings are always 'in the loop,' deciding training data, tuning hyperparameters, and curating the output, using AI as a tool rather than an autonomous creator of new artistic forms.
14 Actionable Insights
1. Prioritize Human-Centric Robot Motion
When designing robots that interact with people, prioritize motion that considers psychological factors like comfort, empowerment, and friendliness over mere efficiency, because motion profoundly impacts human perception and interaction.
2. Integrate Choreographic Language
Roboticists should learn from dancers’ extensive vocabulary for describing and creating motion (e.g., repetition, inversion, transformation, canon, space holds) to explore a wider, more experimental design space beyond optimal trajectories.
3. Dancers: Expand Skill Sets
Dancers should learn programming, sensor technology, and robot motion planning to apply their expertise in new contexts like robot personality design or interaction modes for wearables, moving beyond the traditional stage.
4. Embrace Dance for Social Bonds
Engage in dance, regardless of perceived skill or sanctioned spaces, because it is a foundational human activity that codifies social bonds and interbehavioral group dynamics.
5. Combine Disparate Interests
Pursue parallel interests, even if they seem unrelated, as combining them (e.g., dance and technology) can lead to unique, fulfilling career paths and innovative interdisciplinary work.
6. Implement Human-in-the-Loop Systems
Design AI and robotics systems with a “human in the loop” where human operators can intervene when the system is uncertain or stuck, and their input helps train and improve the algorithm over time. This balances automation with human oversight and expertise.
7. AI as Collaborative Tool
Recognize that AI, especially in creative fields, functions as a tool for human collaboration rather than an autonomous artist; humans define training data, tune parameters, and make artistic choices, fundamentally shaping the output.
8. Train Robots with VR/Simulation
Utilize virtual reality and simulation platforms to efficiently train robots by having them practice tasks with vast amounts of simulated data, which can then be transferred to real-world robots, allowing for rapid learning and testing.
9. Leverage VR for Empathy
Use virtual reality systems to simulate inhabiting different bodies (human or robot) to foster empathy and understanding of diverse experiences, as demonstrated by projects like the “Be Another Lab.”
10. Experiment with “Ugly” Motion
Roboticists should consider exploring “ugly,” chaotic, or disorienting types of motion, similar to how dancers do, to expand the design space beyond purely optimal or aesthetically pleasing movements.
11. Explore Creative Feedback Loops
Create interactive systems where human and robot actions continuously feed into each other, allowing for organic, emergent motion and a commentary on societal interactions with technology, as exemplified by “The Loop” project.
12. Develop Niche Consumer Robots
Consider developing consumer robots for specific, high-value niche applications like elder care (e.g., companion robots, mobility assistance, automated food preparation) where the benefits clearly outweigh the cost and complexity.
13. Anticipate Robot Failures
When performing or demonstrating with robots, be prepared for frequent breakdowns due to software bugs, hardware malfunctions, or connectivity issues, as these are common and can dramatically alter the intended meaning of a piece.
14. Understand Robust Robotics Cost
Recognize that achieving 99.9% robust and reliable performance for robots in diverse, real-world conditions is extremely challenging and expensive, which currently limits their widespread consumer adoption.
6 Key Quotes
Apple's doing choreography. That's essentially the type of practice they're entrenched in. And the Apple Watch is a choreographic interface because you're inferring things about people's motion from the outputs that you get from the device.
Catie Cuan
Movement is so unconsciously important to human beings. And in a context where the robot has to move around a person... the stakes of that motion are so heightened.
Catie Cuan
The term choreography, literally it's a portmanteau. It means dance writing. So it comes from like dance notation.
Catie Cuan
Human beings can learn motion, you know, almost immediately. And when you have to go through all of the teeny tiny tuning of all of these robot motions, it takes a long time.
Catie Cuan
Robots are hard... to do that at 99.9% performance across every single parking lot in every single weather condition with every single surface of asphalt, grass, sidewalk, whatever is really hard.
Catie Cuan
AI can't create VR from scratch if it doesn't know that VR doesn't exist.
Catie Cuan