panhandlefamily.com

Exploring Synesthesia Through LEGO and Python

Written on

For my inaugural piece in 2023, I aimed for something both unusual and relatable. I set myself the challenge of merging four primary subjects I often write about: technology, coding, accessibility, and LEGO. This proved to be quite a challenge, but I remembered an idea I had back in the spring of 2020—a concept I found intriguing but had never shared.

It's also time to confess that my previous article on the LEGO smart lock was inspired by this earlier project, with a significant portion of the code being reused. Reusability is a great asset! Now, let's not keep you in suspense—let's explore a topic that has intrigued me for almost twenty years.

In 2020, my goal was to showcase synesthesia in a simple yet playful manner. After a few hours of experimentation and coding, I succeeded.

How To Create A Smart Lock Using Python And EV3dev

This section provides high-level instructions for constructing a functional smart lock using LEGO’s EV3 brick along with straightforward Python code.

What is synesthesia?

A brief introduction to this perceptual phenomenon will enhance this narrative. My first encounter with synesthesia occurred while watching the NBC series Heroes. One character, Emma Coolidge—who is also deaf—experiences a type of heightened synesthesia while playing the cello (or possibly the double bass). Looking back, I believe this was when the importance of accessibility began to resonate with me. This scene remains one of the most impactful moments in television for me. I encourage you to watch it, along with another scene that illustrates this phenomenon.

> Synesthesia (American English) or synaesthesia (British English) is a perceptual phenomenon where stimulation of one sensory pathway leads to involuntary experiences in another. Individuals with a lifelong history of such experiences are referred to as synesthetes. — Wikipedia

An abundance of literature exists on this topic, so if you share my fascination, I highly recommend delving deeper after reading this article.

Prerequisites for the synesthesia demonstration

To create this demonstration, you will need a few essential components; however, it's significantly simpler than the smart lock project, making it more approachable if the latter felt daunting.

  • An EV3 brick. This is crucial; it must be the version from set #31313 or the official educational edition set #45544. Only one smart brick is necessary for this prototype. You can purchase it separately, but availability may be limited.
  • Ideally, a light/color sensor is required. This will detect various colors from your chosen source and enable us to translate them into sound.
  • Basic Python knowledge. I may write a companion piece introducing Python basics soon, but for now, resources like Codecademy and the many links in this FreeCodeCamp article can help.
  • A computer. This could be a simple device like a Raspberry Pi or something more premium like a Mac. A Windows machine will work perfectly as well. I used my 2019 Intel MacBook Air for this project.
  • A microSD or microSDHC card (2 GB or larger, but not exceeding 32 GB due to the EV3 brick's limitations).
  • Regarding software, you will need two things: VSCode and EV3dev. Additionally, install the ev3dev-browser plugin for seamless project uploads to the brick.

As you can see, the requirements are quite minimal, and the beauty of this project is that you don't have to construct anything out of LEGO bricks apart from the smart brick and the light/color sensor, making it straightforward.

The synesthesia “engine”

While it's common to associate specific colors with particular sounds, my demonstration takes a different approach. The project is simple and requires minimal coding effort. The synesthesia "engine" is essentially just a set of if-else statements, but as Apple has shown us, marketing is key—so if I label it an "engine," you can bet it sounds impressive!

from ev3dev2.sensor import INPUT_1 from ev3dev2.sensor.lego import ColorSensor from ev3dev2.sound import Sound

spkr = Sound() xylophoneSensor = ColorSensor(INPUT_1)

spkr.speak('Starting synesthesia engine.') timeRemaining = 100

while time > 0:

# xylophone sensor operates on input port 1

if xylophoneSensor.color_name == 'Blue':

spkr.play_file('/home/robot/synesthesia/xylophone/a1.wav')

time = time - 1

elif xylophoneSensor.color_name == 'Green':

spkr.play_file('/home/robot/synesthesia/xylophone/b1.wav')

time = time - 1

elif xylophoneSensor.color_name == 'Yellow':

spkr.play_file('/home/robot/synesthesia/xylophone/c1.wav')

time = time - 1

elif xylophoneSensor.color_name == 'Red':

spkr.play_file('/home/robot/synesthesia/xylophone/d1.wav')

time = time - 1

elif xylophoneSensor.color_name == 'Brown':

spkr.play_file('/home/robot/synesthesia/xylophone/e1.wav')

time = time - 1

elif xylophoneSensor.color_name == 'Black':

spkr.play_file('/home/robot/synesthesia/xylophone/f1.wav')

time = time - 1

elif xylophoneSensor.color_name == 'White':

spkr.play_file('/home/robot/synesthesia/xylophone/g1.wav')

time = time - 1

Upon examining the code, you’ll notice the operations are quite straightforward. As the program initiates, the smart brick announces that the engine has started, at which point it awaits color input. For each color detected by the sensor, a different xylophone note is played. The instrument can vary, provided you have the necessary .wav files. It’s worth mentioning that the sensor can also detect the absence of color, but I found that this added complexity, so I limited it to seven notes.

You can view a demonstration of this code on my Instagram or TikTok. I would appreciate it if you checked it out and followed me there as I share interesting LEGO and EV3 programming content not found in my articles.

In fact, you can significantly enhance this demo by adding one or two more light/color sensors. This allows for color combinations to produce full octaves and even polyphonic sounds. If you want to modulate the sound, consider using an ultrasonic or infrared sensor, or a touch sensor to create additional effects or switch between instruments. Ultimately, your imagination is the only limit.

Synesthesia is an incredibly engaging method to introduce someone to diversity and accessibility, showcasing just how differently we can experience the same world.

Attila Vago — Software Engineer improving the world one line of code at a time. A lifelong enthusiast, I write about coding, web accessibility, LEGO, and more. Enjoy craft beer! Read my Hello story here! Subscribe and/or become a member for more stories about LEGO, technology, coding, and accessibility! For those who don't read regularly, I also explore random topics and writing.

Share the page:

Twitter Facebook Reddit LinkIn

-----------------------

Recent Post:

Navigating the June Earnings Dilemma: Insights and Reflections

Reflecting on the unexpected decline in earnings this June and exploring possible reasons behind it.

Exploring Investment Opportunities in AtomBeam Technologies

A deep dive into AtomBeam Technologies, evaluating its potential as an investment in data compaction and machine communication.

Insights on Spiritual Growth and Transformation

Discover transformative insights through tarot readings, focusing on personal growth, spiritual connection, and overcoming past challenges.

Embracing Life Beyond Hustle Culture: A Call for Balance

Explore the pitfalls of hustle culture and the importance of living fully before retirement.

Embracing the Art of Shelving: Why Writers Should Let Go

Discover why shelving a book can be a powerful step for writers, allowing for growth and new opportunities in their craft.

Empowering Businesses Through Data: Insights from Seshu Madabushi

Discover how Seshu Madabushi of mKonnekt leverages data to enhance business decisions and drive growth.

The HPC Revolution X9: A High-Performance Electric Bike

Discover the HPC Revolution X9, a remarkable electric bike that combines speed, comfort, and advanced technology for an unparalleled riding experience.

Understanding the Shadow: Embracing the Unseen Within

Explore the concept of the Shadow in psychology and how embracing it can lead to personal growth and creativity.