Structure of Music and the Design of Instruction

by Gregg Gorcsan
San Diego State University
Educational Technology
EdTech 653 Multimedia and Hypermedia
May 5, 1998

What is it about music that engages the listener? Is it the rhythmic pulse? Is it the musical scale, a particular musical phrase? Is it the time between the notes? Why do we have music? The late Frank Zappa wrote, "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid."

In order to design instruction, we must understand how the mind processes information. Instructional message design borrows heavily from behavioral and cognitive science principles. Reigeluth (1983) in Fleming and Levie, Instructional Message Design defines instructional design as "...the process of deciding what methods of instruction are best for bringing about desired changes in student knowledge and skills for a specific course content and a specific student population. "The "design" works, if change occurs. In their book, instructional methods become instructional messages.

The field of artificial intelligence may hold research that can begin to formulate a strategy for the use of music as an instructional message. The Media Lab at MIT is the center of research for applications of computing to various disciplines. Many researchers are working together developing novel ways to apply computing technology. In her book, Hamlet on the Holodeck, Janet Murray refers to the projects at the Media Lab and mentions the work of Marvin Minsky, the computer scientist. His work in artificial intelligence (AI) provides a working model of human consciousness as frame-based computational structures (Murray, 1997). Minsky, theorizes that human memory is like a set of frames with "slots."Each slot stores an instance of an abstraction. For example, John Coltrane or Miles Davis as instances of jazz musicians, are stored in the "jazz musician" frame. Specific attributes, such as instruments played are stored in the slots. "Soprano saxophone" may be stored in the "instruments played" slot for Coltrane, and in the "Trumpet" slot for Miles Davis. Murray discusses the specification of frames as formulaic structures, and applies Minsky's representation for building cyberdrama narratives (Murray, 1997). Frames can provide a structure for characters and their attributes, multiple objects, or various events in a story. Minsky speculates, in the Neuropsychology of Music edited by Manfred Clynes, how music and vision "build things in our mind," and "how listening to music engages the previously acquired knowledge of the listener (Minsky, 1981)." Music has structure. The American Heritage College Dictionary defines a sonata as "a composition for one to four instruments, one of which is usually a keyboard, typically consisting of three or four independent movements varying in key, mood, and tempo." A sonata form has three sections, the exposition, development, and recapitulation. Minksy uses the sonata to conceptualize how we may process musical structures created by composers. He presents the sonata as a teaching machine compared to a teacher's instructional strategy. The teacher must first get the students' attention by speaking softly or dramatically (exposition), then presents the basic lesson (development), and finally repeats, maybe many times, and summarizes the lesson (recapitulation). A sonata form begins with a simple statement of the theme, gradually develops the musical structure, and then recapitulates. Einstein once said, "Things should be made as simple as possible, but not any simpler."The composer creates simple expository forms, and develops them by varying key, mood and tempo, and then recapitulates. Minsky states, "A thing has meaning only after we have learned some ways to represent and process what it means, or to understand its parts and how they are put together (Minsky, 1981)."

Component Display Theory (CDT) developed by M. David Merrill modified by B.S.Allen provides a structural and functional microstrategy for display in interactive multimedia design (Allen, 1988). Specific descriptors are the functional components of CDT. The descriptors afford three levels of information as primary presentation forms.

CDT Primary Presentation Forms:

• Expository Generality - present the main idea
• Inquisitory Generality - ask them about the main idea
• Expository instance - present an example of the main idea
• Inquisitory instance - ask them about an example
• Expository Fact - present a fact
• Inquisitory Fact - ask them about a fact

Instructional designers use these components as "chunks" of information on the display to design "instructional messages" that hopefully are transmitted to the long-term memory. Musicians also use component design as musical structures, such as three notes forming a triplet, or measures that contain the pulse or beat indicated by a wholenote, halfnote, or quarternote. The most recognizable four notes, as Minsky points out, are from Beethoven's Fifth Symphony. Simple, effective, and memorable. So memorable, that other composers do not use this four-note pattern in order to prevent distraction (Minsky, 1981). Can these music structures be identified, measured, and possibly applied to the design of instruction?

Manfred Clynes, a pianist-physiologist, describes a method to record dynamic forms of emotional expression in Sentics, The Touch of the Emotions. His research details the discovery and measurement of sentic patterns. He proposes that specific temporal sensory patterns exist, and each pattern corresponds to a specific emotional state, and possibly relates to musical phrases. For example, he identifies the pattern for love as a long, smooth curve. Anger reveals a sharp downward curve. The forms are not only responses to emotions, but also to sound, pitch, and even visual stimuli. Minsky theorizes that since these sentic shapes occur in time, the 1 second period of the sentic form may correlate with musical phrases (Minsky, 1981). Clynes investigated the muscular movement common to instrumental performance of music, and believes that sentic signals engage the listeners' emotions. He identifies a unique inner pulse for a group of classical composers, including Mozart. In the book's preface, John A. Osmundsen writes, "His [Clynes] next elegant leap was to recognize that the qualities he was studying in music could be thought of as corresponding to the qualities inherent in human emotions. And, as he will explain, he was able to show that several different emotions have characteristic expressive 'shapes'(no doubt reflecting their generative brain wave shapes), just as different musical compositions do." Much of this is speculation and more corroborating research is needed, but it is possible that sentic messages exist and we respond to the through "innate sentic detectors" in the brain. Minsky concludes that sentic encryption may explain why we respond to certain signals and states relating to music, and why we like music. He writes, "If you find all this far-fetched, so do I. But before rejecting it entirely, recall the question, Why do we have music, and let it occupy our lives with no apparent reason? When no idea seems right, the right one must seem wrong (Minsky, 1981)."Is there a strategy for the appropriate use of music in multimedia design?

The design of multimedia requires representation, structure, and understanding on the part of the instructional designer. In the book Bringing Design to Software edited by Terry Winograd, contributors John Seely Brown and Paul Duguid in their article titled Keeping it Simple, argue for simplicity, evolution, and innovation in design by breaking new ground, moving out of the borders, and crossing boundaries (Winograd, 1996). They conclude:

The future of design in information technologies lies not in developing means of increasingly full re-presentation, but in allowing increasing amounts to be underrepresented; not by increasing what is said, but by helping people to leave more unsaid; not in refining abstractions, but by making use of their inevitable impurity; not by making more explicit, but by leaving as much as possible implicit, and in the process keeping things simple (Brown & Duguid, 1996).

Miles Davis is an example of a boundary breaker. He was often criticized by fellow musicians for moving from traditional jazz into new forms they considered his music incoherent. Davis had played the old forms when they were called unintelligible. He complained, "Don't tell me the way it was. Hell, I was there ... no one wanted to hear us when we were playing jazz." (Brown & Duguid, 1996). Research and experimentation lead to innovation and a new approach to the use of music in education.

The research of Frances Rauscher, cellist-psychologist, describes the effect of music on intelligence. In two similar studies, Rauscher shows a causal relationship between music and spatial intelligence. In the first study, 19 preschool children were tested after eight months of keyboard and singing lessons, and compared to a control group with no music lessons. Standardized intelligence tests measured the outcomes of five tasks taken from the Performance subtest of the Wechsler Preschool and Primary Scale of Intelligence, the Stanford-Binet Intelligence Scale, and the WPPSI-R test. The spatial reasoning abilities of the music lesson group exceeded the other group. Spatial reasoning is the accurate perception of the visual world, and the recognition of objects by size and shape. In theory, the ability to use a spatial skill is necessary in higher brain functions, such as music, complex mathematics, and chess. These spatial skills are thought to be what scientists and engineers use to solve complex problems. Many renown scientists such as Albert Einstein and Richard Feynman, played musical instruments for relaxation and inspiration. The authors conclude that musical training has a positive effect on spatial reasoning and may be used to enhance education in the public school system.

The second study by Rauscher, Shaw, Levine, Ky and Wright (1994) investigated the "Mozart Effect" in a larger and older population of students, and may have implications for music in multimedia design. A group of seventy-nine college students completed the experiments. The study investigated the hypothesis that listening to music enhances spatial task performance. The independent variable was a mixed listening condition compared to silence and the Mozart Sonata for Two Pianos in D Major, K. 48, applied prior to the task, not during the task, as music might be in a multimedia lesson. The conditions were ten-minute segments of an audio-tape of minimalist music, an audio-taped spoken story, and audio-tape of British style dance (trance) music, and finally Mozart's Sonata. The results showed an overall, but temporary improvement in spatial reasoning skills measured by a Paper Folding and Cutting item test from the Stanford-Binet Intelligence Scale. The authors conclude by pointing out that the Paper Folding and Cutting task is not only spatial, but also temporal, and relates to a spatial-temporal pattern development model predicted by Leng and Shaw (1991). Could their also be a relationship to the temporal sensory pattern research of Manfred Clynes and the work of Mozart? A quotation from Mozart about composition may add some insight, "When I am, as it were, completely myself, entirely alone, and of good cheer...my ideas flow best and most abundantly. Whence and how they come, I know not; nor can I force them. Those ideas that please me I retain in memory, and am accustomed, as I have been told, to hum them to myself. If I continue this way, it soon occurs to me how I may turn this or that morsel to account, so as to make a good dish of it, that is to say, agreeably to the rules of counterpoint, to the peculiarities of the various instruments, etc. all this fires my soul, and provided I am not disturbed, my subject enlarges itself, becomes methodized and defined, and the whole, though it be long, stands almost complete and finished in my mind, so that I can survey it, like a fine picture or a beautiful statue, at a glance. Nor do I hear in my imagination the parts successively, but I hear them, as it were, all at once (Munves, 1995)." Apparently, something wonderful is happening in the neural net of Amadeus. What is it? How does it work? Can this process be analyzed, quantified and applied to learning and instructional design?

A study investigating the use of music in multimedia design by Hardy and Jost (1996) contains an extensive review of reviews of literature related to music and instructional design. The general conclusion is that the body of research is certainly multidisciplinary in scope, but contradictory and inconclusive overall. They present a few guidelines. Music can and only appears to be used for its objective properties, such as tempo, tonality, and texture to elicit various feelings. Tempo seems to be the property chosen by instructional designers. The use of music as an instructional message has not been investigated according to the authors. Some communications researchers believe that although music can attract children to the learning messages, ultimately, the music distracts and competes with the educational content. The authors discuss some guidelines for design dividing music into two categories, literal and abstract (Hardy and Jost, 1996). For example, literal music is the sound of birds related to the content of the lesson. Abstract music is not related to the content and just serves as an emotional element.

Hardy and Jost also survey the use of music during instruction in the context of setting (learning environment), student population such as special education, and finally subject matter, such as language teaching. Some of these studies show a positive correlation between music, affective response and cognitive achievement especially in science classes. Literature reviews in the education field relating to music research and instruction were also reviewed. Educational research results are also contradictory.

Hardy and Jost conducted a study of applied instructional design strategies using music to enhance instruction in a multimedia learning program. The authors choose a triple-coding strategy to investigate the effect of music as the third part of the standard text and illustration method. This three part study used triple-encoding, the objective properties of abstract music (tempo, tonality, and texture), and linking the particular music treatment to the instructional strategy. The independent variable of the study was Music-Instructional Strategy-Integration (MISI). The researchers manipulated the tempo, tonality, and texture to elicit feelings of arousal, surprise, and pleasantness during multimedia instruction. The instructional design strategy involved affective orientation (arousal) and cognitive dissonance (surprise and pleasantness upon reconciliation). The music condition had two patterns that served as congruent and incongruent affective patterns that also moderated the treatments to enhance or retard achievement (Hardy and Jost, 1996). Fifteen different classical composers were selected by an unknown means, and selections from their music accompanied the lessons. The study does not explain the rationale for selecting composers or passages. The results showed no significant learning outcomes from the MISI variable because there was no difference between high MISI, low MISI, or no music conditions. The study did not support the theory that music distracts from learning. The students all learned even during the incongruent pattern MISI. The students did report enjoying the music and learning science through multimedia.

I see two problems with this study and one possibility for future research. First, as the authors point out, the instructional design presented just information in a PowerPoint slideshow, and possibly so psychologically saturating as an experience for the students that no more neural pathways were available to process the music. Secondly, the new experience of computer as teacher may have caused the students pay more attention to learn the lessons. The authors suggest further research using literal music.

In summary, what lessons and possibilities can be inferred from the topics discussed in this report? Literature reviews surveyed show very little research in the use of music in instructional multimedia. The research of Clynes, far-fetched as it may seem, needs corroboration, replication, and validation. A web-search revealed on-going research, and a located a sentics theory-based software program called SuperConductor. Rauscher and associates appear to have the strongest scientific study to date, and clearly show that Mozart's music does enhance spatial reasoning. Her research basis is a neurophysiological model of Leng and Shaw (1991) who suggest that enhancement of cortical firing patterns is possible. I would like to see a research project using a similar Hardy and Jost design, linking CDT Primary Presentation Forms to the MISI conditions using Mozart passages only in a multimedia format to look for a concurrent Mozart effect and related sentic patterns

.

In conclusion, the music of Mozart has a mystical, spiritual, quality. Jim Svejda, the opinionated music expert and host of the radio program The Record Shelf, introduces the Mozart section of his book, The Record Shelf Guide to the Classical Repertoire by telling a personal story, "Once, when filling out an application for a summer job, on the line next to 'other' under the heading of Religion, I wrote Mozart. The personnel officer was not amused, but then, I hadn't intended it as a joke. For there was a time when I was convinced that Mozart was at least a divinely inspired as Moses, Christ, the Buddha, Lao-Tsu, or Mohammed, and I suppose I still am. For in no other works of the human imagination can the divine spirit be heard more distinctly than in the miraculous music this often vulgar, unpleasant, and difficult man produced during his pathetically brief thirty-five years (Svejda, 1990)."

References

Allen, B.S. (1988). Designing Interactive Multi-Media.Workshop materials. San Diego, CA: San Diego State University.

Brown, J. S., & Duguid, P. (1996) Keeping it simple. In T. Winograd (Ed.) Bringing Design to Software. (pp. 143-144). New York, NY: ACM Press.

Clynes, M. (1978). The Touch of the Emotions. Garden City, NY: Doubleday & Company.

Fleming, M. & Levie, W. H. (1993). Instructional Message Design. Englewood Cliffs, NJ: Educational Technology Publications.

Hardy, R. D., & Jost, K. L. (1996). The Use of Music in the Instructional Design of Multimedia. Indianapolis, IN: Association for Educational Communications and Technology. (Eric Document Reproduction Service No.ED 397 797)

Leng, X., & Shaw, G. L. (1991). Toward a neural theory of higher brain function using music as a window. Concepts on Neuroscience, 2,229-258.

Minsky, M. (1981). Music, Mind, and Meaning. [on-line]. Available: http://www.ai.mit.edu/people/minsky/papers/MusicMindMeaning.txt

Munves, R. P. (1995). Mozart for your mind. In Mozart for your mind [ CD: liner notes]

Murray, J. H. (1997). Hamlet on the Holodeck. New York, NY: Simon & Schuster Inc.

Reigeluth, C. M. (1983). Instructional design: What is it and why is it? In C. M. Reigeluth (Ed.), Instructional design theories and modesl: An overviwe of their current status (pp.3-16). Hillsdale, NJ: Lawrence Erlbaum Associates.

Rauscher, F. H. (1993). Music and spatial task performance. Nature 365 (October), 611.

Rauscher, F. H., Shaw, G. L., Levine, L. J., Ky., K. N., & Wright, E. L.(1994). Music and Spatial Task Performance: A Causal Relationship. Paper presented at the 102nd Annual Convention of the American Psychological Association, Los Angeles, CA August 12-16, 1994.

Svejda, J. (1990). Record shelf guide to the classical repertoire. Rocklin, CA: Prima Publishing.