Get the latest Education e-news
 
  • Audio Design For Interactive Narrative VR Experiences

    [11.15.18]
    - Larry Chang

  • 2. Using Sound to Tell Story

    When it comes to using audio to tell interactive narratives in VR, there is one concept to keep in mind: Indirect Control. It is a technique to use design to guide the player to certain expected actions without letting them realize the fact that they are being guided.

    A well designed audio experience is a great opportunity for indirect control since audio has several key features that can be used to guide the player. First, it sets time and space of the experience for the player. In The Price of Freedom, for example, we have a radio in a scene that plays the music from the 60s, and a TV that broadcasts John F. Kennedy's inauguration speech so that the players can clearly get it that they are in a specific time period. In this case, the 60s in the United States.

    Also, apart from the instructional dialogues that directly guide the player what to do, we also have reel to reel tapes with dialogue that can not only guide the player to focus on specific area of the space but also provides fragments of the story as reinforcement. The tapes, along with other elements like letters, documents, and pictures remind the player of the story over and over again. So that even if they miss some of the information, they are still able to have an overall understanding of the narrative.

    The audio also sets emotional tone for the experience. Interactive music is a great way for us to affect the player's emotional states while they are immersed in an interactive environment. We designed a music piece into several scales with different intensity of the emotion and then put them in sequence for the player to trigger in the scene.

    Take the fire scene in The Price of Freedom for example. We designed the emotion of this scene into 4 levels of intensity. In the beginning, the player just read through a binder and discovered what terrible things they have done. We want the player to feel the emotion from feeling numb to grief gradually, just like the character would feel in the story. The first layer of intensity starts when the player finishes reading the binder, It is a pretty simple tune with a shade of gloominess. There is only one synthesizer simulating a heartbeat. As soon as the player lights the binder on fire, the second layer will be triggered. At this point, the emotion of sadness grows along with the fire. A piano melody added on to the second layer to indicates that the player's conscience has been awakened.

    Then, as soon as the player finds a way to exit the chamber they start in, the third layer of intensity is triggered. The sad emotion becomes more intense as a hint that everything is about to change in the character's life. As the scene fades out we designed a tail for the ending of the music cue so that it sounds like a linear musical experience. Then the story goes on...

    Speaking of music, some people may ask what the difference between writing music for VR and non-VR games may be. In my personal opinion, music in VR should be more subtle and be more intimate to the player. This is because it is the player's personal and direct experience. With the first person's perspective, music in VR is all about the player's connection to the virtual world. They are the protagonist of the show and the music is built around them.

    In The Price of Freedom, we have designed signature music combining both non-diegetic and diegetic forms for different themes of the narrative. We have developed theme music for Ben Miller's daughter, Cathy Miller. It shows up the very first time in the experience with a diegetic form as a tune played on a music box which her father gave it to her as a birthday present. After that, whenever the plot or information of the story is related to Cathy, the player will be reminded of the same theme music.

    Here is an example of the combination of two theme signature music:

    Project Mk Ultra Theme

    Cathy's Theme

    Combination (Ending)

    Final Suite

Comments

comments powered by Disqus