Recently, I've been watching some interviews of video game designer Jonathan Blow, and found his approach to teaching game mechanics in The Witness fascinating. The game teaches its mechanics intuitively, with no invasive tutorial and no words. This leads me to question if the same idea could be applied to musical instruments. Is it possible to design a musical instrument be naturally intuitive to a new player?
In practice video games and musical instruments have the same core problem: the player needs to learn how to interact with the game, or instrument. To see how that can be done, let's begin by exploring how The Witness approaches the problem. The core mechanic of the game is very simple: to draw a line from a starting point to a finishing point.
To teach this, the game begins with the player trapped in a small room. The player is naturally drawn towards the most visually distinct feature, the yellow puzzle panel, and a subtle visual cue encourages them to click on the circular end of the line. They happen to notice that they can complete the line. And so, the door opens.
What's going on here is simple. Players will naturally want to explore their environment, and by limiting what they can do, the designer can create an environment where the correct solution becomes obvious. It is impossible to leave the room without drawing a line, starting from the circular 'knob' and linking it to the semicircular line end.
The game continues to teach its mechanics in this way, by creating sequences of increasingly difficult puzzles. Within this framework, the player can experiment with the system, hypothesise about the rule, and eventually come to a conclusion. Do you see the rule below?
Some call this 'teaching without teaching'. However, it really isn't; it's teaching using a different medium of communication. Instead of words, teaching is performed through environmental cues, and by creating a limited, curated environment that guides people towards the correct solution. Thus, I call it 'environmental teaching' myself.
So that's how The Witness teaches its mechanics intuitively, but what about making musical instruments that are naturally intuitive to payers? From our above exploration, can a musical instrument create an environment that effectively guides a player towards good technique, and playing interesting music?
I think it's worth starting our exploration of the question with the kazoo. Inherently a kazoo can only do one thing, modify the sound of a player's voice using a membrane. As singing comes naturally to most people, it is intuitive to play. Someone can easily hear a melody, and hum along, playing it on the kazoo.
Outside of this simple case, the question of intuitive musical instruments becomes much more complex. Instruments like the xylophone and keyboard are simple to interact with. Their design makes it easy for a new player to achieve a good sound. Someone can easily press keys or hit bars, and there is a pretty good chance they will be able to find some notes forming a recognizable tune.
Yet they do little to actually guide the learner. They must explore the scope of the instrument by themselves. For some, this will come naturally, through experimenting they will find things that sound good to them, and progress from there. For others, having a whole instrument, and no idea what to do can be too intimidating. And in either case, there is a lot of opportunity to develop bad technique.
The fundamental problem is that musical instruments are open ended, they are tools used by a musician to produce sound, and usually have a great deal of flexibility. However this cannot provide the curated environment you need to guide someone. The more flexible an instrument is, the less guidance it can offer to a player.
The recorder and ocarina, for instance are intuitive in some regards. Both instruments are visually approachable, and have simple fingering systems which are intuitive. But the need to separate notes with the tongue, and control pitch with blowing pressure is unintuitive for most people. It's easy to develop bad technique, and have no awareness of that.
Ultimately, acoustic instruments are limited in the guidance they can offer to learners, but electronic instruments offer a lot of potential, as we will explore next.
I can see a great deal of potential to create electronic instruments that help to guide their players, as electronic instruments are not constrained by a physical system, in principle they can be made to do anything. Plus, they can be played on headphones, so there is never a fear of sounding bad.
At a surface level, electronic instruments can adapt their features and sensitivity to the player. For example, an electronic keyboard could be more tolerant of player errors by decreasing, or even disabling key volume dynamics. That could simplify a player's job by allowing a learner to focus on the fingerings and rhythm, while still producing a reasonable sound. Then pressure sensitivity and volume dynamics may be introduced later.
There are many things that can be done to allow a player to fix their own errors intuitively. Pitch for example is easy to learn with accompaniment. So, an electronic instrument without fixed pitch, the instrument could play a sympathetic note which is in tune, or reduce the pitch sensitivity at first until the player learns to control it. Similarly, an instrument could include a metronome, or otherwise highlight errors in rhythm.
A step above this would be to present a kind of 'game' or 'tutorial' integrated into an instrument, that presents a learner with a set of intuitive exercises. Such things have been done, but I feel that more thought needs to be given to making them open to player exploration, and less patronising. How could you design a 'tutorial mode' in an instrument more like an open world game?
I think that a feature like this could teach the basics, but teaching musicality well would be a hard problem. The game would need to evaluate the player on artistic merit, and doing so is arguably impossible as music is subjective, thus 'good' and 'bad' are very hard to define. I have discussed this before in 'Teaching music with video games'.
Electronic instruments have a lot of potential, and nowadays, electronic instruments don't even have to be a dedicated physical device. They can also be created as applications for computers, or phones/tablets. With regards to an interactive tutorial, it has benefits as a screen could easily present a simplified instrument interface, which adds features with time.
I feel it also worth noting that the same guidance we have been exploring here, can also be applied to a traditional student-teacher situation. For example, a teacher could begin teaching a student piano using one finger technique at first, quickly allowing the student to play some music they know.
As they get better, the limitations of the technique will become apparent, and playing using multiple fingers can be introduced. Teaching in this way means that the student is always aware of why they are doing what they are. This is essentially leading the learner into a trap, and making it obvious that it is a trap. From this point of view, it can be made apparent why the preferred technique is preferred.
I don't know if it would be possible to create an environment whereby a student could be guided to discover techniques like playing with multiple fingers, and the thumb tuck by themselves. Playing with multiple fingers can feel extremely awkward the first few times it is attempted, and someone may not know to push through this.
As I noted before intonation can be taught by playing with accompaniment, and it's interesting to note that a standard violin actually does this for the player. The instrument has sympathetic resonances between the strings, and sounds notably different when some notes are played in tune. In addition to this, if a student can play along with a teacher, and the teacher is in tune, their errors will be apparent as the clashing sounds will 'beat'.
I think that it is important for a teacher acting in this role to be willing to allow their student to struggle, to provide hints, but let them discover things for themselves.
I believe that it is possible to teach music and instruments in a more intuitive way using electronic instruments, or teacher guidance. Teaching using this natural intuition is useful, as it can help learners better understand the value of good technique, and can make them feel more in control, as they are exploring by themselves, with guidance.
I also find it interesting that, in my own experience, the realisations I've had during my research into the behaviour of the ocarina have come from similar situations to those artificially created by The Witness mentioned above. The nature of the environment makes a certain possibility obvious.
However, finding these situations is often arduous. For example, realising how air temperature affects an ocarina's pitch involved playing in many situations, and rejecting some mainstream beliefs. What we are doing with intuitive teaching, is letting learners find things for themselves, but providing hints in the right direction.