What I hadn’t appreciated until now was not only the extent of Brice’s career in the commercial side of broadcast technology, but also the depth of his musical background, which, no doubt, has contributed to his design work at Phaedrus Audio. - Bob Thomas (July 2016).
As an example of this, imagine here is a short scene of a man walking along a country lane. If this is accompanied by breezy, light music, the audience's reaction is very different from if the same scene was accompanied by dark, sinister, mysterious music.
Two important terms in film and television music derive from the distinction between source music; music which appears to come from a source in the picture (a radio turned or an in-vision singer) and music which is "laid-under" the dialogue, effects and so-on. This second type of music is known as underscore and, although composers sometimes do compose source music, the more normal role for the film composer is the invention of the underscore.
The composer and the director (and others including the music editor on a large production) decide each piece of music, its duration and exact timing in a process called the spotting session; each musical piece being referred to as a cue. (This is equally true for source music and underscore.)
In addition to the timing of each cue in relation to the film as a whole, the director and composer have to decide where a particular musical event is required to coincide exactly with an on-screen visual event, this being known as a hit or sometimes a dead-cue.
The synchronisation requirement, so that a musical event and a visual event are perceived as synchronous by the audience, is ≤2 film frames or ± 41mS. The usual rule of thumb is that the music cue must be within 1⁄10th of a second of the on-screen action.
The judgement as to where a musical event "hits" a visual event on the screen is one of the great skills of the film and television composer. To some extent this is dictated by the genre of film; comedy often "hitting" visual events more often than serious drama. Indeed for serious drama, more than one author has referred to the "counterpoint" that needs to exist between the music and the on-screen visuals.
Cartoons are the great example of the use of musical hits, where stealthy footfalls are accompanied by pizzicato celli and eye blinks by two-note, falling xylophone phrases. The great Carl Stalling of Warner Brothers is largely responsible for inventing these marvellous musical ideas. That Stalling's style would be inappropriate in an action picture or a thriller is obvious; indeed the tendency to over-hit visual action in a musical cue is disparagingly referred to as cartooning or Mickey Mousing!
It is not always necessary to compose music for picture so that music synchronises with the on-screen action. Music which is not recorded "to picture" is said to be wild or to be a wild track. That's not to say the music is completely feral! It may often be of excellent quality and chosen, or even specially composed, with great care. But it is "untamed" by the visual content.
The whole industry of library music or production music is based on providing generic music for certain screen genres and atmospheres and is necessarily composed with no reference at all to the video to which it is eventually married.
It is claimed that the development of the music video, which often has screen action only loosely coupled to the song for which it was shot, has greatly increased the audience's tolerance of the acceptable match of music and on-screen action¹. And the development of using popular songs for the underscore of films, rather than expressly composed music, (a development which is really the inverse of a music video) has further decoupled music and action. In many circumstances, it is only necessary that the song matches or enhances the general "feeling"of the film's protagonists (or audience) during a scene or sequence, rather than complementing the screen action.
Whilst this is an entirely valid technique, no further attempt is made to describe the use of music in this way here. Instead we will concentrate on music composed specifically to visual media.
Almost universally, a composer engaged specifically to compose for a film or TV programme will work to a video file copy of the fine-cut or locked-picture of a sequence of the film (clip) often with burned-in timecode (as illustrated left). Timecode is the count of: hours; minutes; seconds; and frames which identify every frame of a movie film or television clip.
The fine-cut is the edited version of the film, derived in turn from the working rough-cut. Music editors and composers prefer not to begin their work until the locked-picture has been edited and approved by the director. Because the music is composed to the visuals as reference, changing the picture after the music has been composed involves extra work and costs.
This video copy of the fine-cut will be a copy of the final video edit or be generated from a telecine dub of the 24 frames a second (FPS) film. If the score is purely electronic and sequenced, the composer will often require nothing more than this video copy of the locked-picture.
In the days of videotape, this copy was a physical tape. The videotape had one of the stereo audio tracks devoted to a quasi-audio signal called longitudinal time code (LTC, see Appendix): the other track being reserved for the dialogue track (because knowing where the dialogue falls is another issue the composer must appreciate and allow for). The composer synchronised the MIDI sequencer to longitudinal timecode coming from the video tape. Sequencers of the period provided for this type of synchronisation using MIDI Timecode (see Appendix).
This is rarely the case nowadays. Most modern sequencers allow for a video window in the DAW so that the video track (provided as a digital file) and the audio are held in synchronism within the sequencer itself. The dialogue track is retained, but the need for inter-machine synchronisation has passed.
When a film or television score calls for real musicians, this demands other forms synchronisation. Two methods are typically used, the click-track and streamers and punches. These are covered below.
Reel nomenclature
Film editors still break down a full-scale film into smaller reels of about 20 minutes duration. For that reason individual music cues are sometimes annotated in relation to these reel numbers so that the first music cue on reel one is referred to as 1M1, the second cue being referred to as 1M2. On the second reel, the music cues are referred to as 2M1 and 2M2 and so on.
By the time the Master Cue Sheet is assembled, the composer will have given each cue a verbal title but on the spotting notes, each cue is simply referred to by this reel and cue number. A short section of a typical Spotting Sheet for feature film is illustrated (right).
A click track is an electronically generated metronome which is played to the recording musicians so that they are synchronised to the moving picture, the tempo having been carefully chosen so that it complements the on-screen action. How this tempo is chosen (and tweaked for film use) is covered in this section.
A click-track was originally derived by punching holes in the side of the film where the optical sound track appears. As white light reached the photo-cell, a loud click was produced when the signal was reproduced by headphones or a loudspeaker. If a hole was punched every 24 frames, the click appeared once every second. In musical terms this would be referred to as 60 beats per minute (BPM). Every 12 frames and the click track would be at 120 BPM and so on. Smaller divisions of tempo were derived from sub divisions of frame length. The are four sprocket holes adjacent to each frame of 35mm film as illustrated (right).
There are therefore eight, well-defined positions in each film frame where the click-track "punch" can be made; against each sprocket hole and between each sprocket hole in each frame. These positions being referred to as zero through seven. For a tempo slightly lower than 60 BPM, the film was punched at 24 frames plus one eighth of a frame: this click track tempo being referred to as 24-1 or 241⁄8.
Prior to digital calculators and computerised tools, whole, very specialised books (Click books) were published with tables relating musical tempo to duration and click track markings. One of the first was published in Hollywood by the music editor Carroll Knudson. Calculated on an early computer, the book was a heroic achievement and composers "swore by" the Knudson book².
Illustrated (left) is a more modern incarnation of a Click book by Alexander Brinkman which was published in the first edition of On the Track: A Guide to Contemporary Film Scoring by Fred Karlin and Rayburn Wright. (A modern version of Hollywood Click-books is provided with the Phædrus Audio Cue and Hit Calculator.) The workflow using a Click-book is illustrated in the following example.
How to use a Click-book to calculate the correct tempo
At the spotting session, the duration and nature of cue 2M1 was agreed between the director and composer. Only a single hit (dead cue) was required at 34 seconds, 0 frames.
The composer invented a suitable music cue with approximate tempo of ♩ = 80 (18 film frames per beat).
The music editor (or the composer) looks up the Knudson table for 18 frames/second beat (80 BPM) and discovers the closest beats to 34 seconds are the 46th and 47th beat which fall respectively at 33:75 seconds and 34:50 seconds. The first a quarter-second early: the second, a half-second late. So 80 BPM exactly won't work.
The editor therefore flips back one page to the table for the click at 17-7 frames (♩ = 80.56 BPM) and discovers that - at this tempo - the 46th beat falls at 33:52 seconds and the 47th at 34:27. So this time, the problem's exchanged: the first is a half-second early and the second, a quarter-second late. So 80.56 BPM won't work either.
So, the editor flips one page further back to discover that at a tempo of 17-6 frames/second (♩ = 81.13 BPM), the 47th beat falls at 34.03 seconds. Perfect!
When the performers come to play the cue, they are provided with a 17-6 frames/second (♩ = 81.13 BPM) click. The difference in tempo between 80 BPM and 81.13 BPM is negligible from a musical point of view, yet the music now perfectly complements the on-screen action.
You can replicate this workflow using with the Phædrus Audio Cue and Hit Calculator.
Click calculations
The modern, electronic metronome (calibrated in BPM), synchronised to timecode, has now replaced the laborious process of punching holes in film. But, should you need to make the translation between elapsed time, BPM and frame-numbers, the first step is to remember that the terms: 12-1; 12-2; 12-3; and so on, really mean 12 and one-eighths, two-eighths, three-eighths. These fractional expressions make life much easier when calculating. So, if N is the number of frames, we can say,
N - 0 = N.0 N - 1 = N.125 N - 2 = N.25 N - 3 = N.375 N - 4 = N.5 N - 5 = N.625 N - 6 = N.75 N - 7 = N.875 To translate to Seconds per Click (S), S = N/24 And for BPM, 60/S = BPM
The click track is a very powerful technique and, as a means of keeping a relatively large ensemble in strict time, it is peerless. A click track is really indispensible when playing a long, fast cue with lots of hits. The "click" is by far, the most widely used technique in developing music for film and it remains the key tool in the most modern workflows with the composer-performer (see below).
But the click-track's inevitable mechanical genesis tends to result in a rather mechanical performance when the music calls for a more "fluid" (rubato) style.
Where the composer and/or conductor wish to keep the synchronisation between picture and music somewhat more fluid on artistic grounds, a simple stopwatch approach can be used. But this technique sacrifices accuracy to the point that the music is little more than a wild-track of a controlled duration. It makes synchronising to hits very difficult.
A combination of benefits results from the technique known as streamers and punches. Sometimes referred to as picture cueing, the technique was developed in the golden days of Hollywood by the great composer and conductor Alfred Newman3. The technique ensures synchronisation on the hit, but otherwise leaves the conductor a degree of freedom in terms of tempo and pacing.
Streamers and punches are still widely used in the post-dubbing of sound-effects, a process know as Foley (after the legendary sound effects director Jack Foley), and in Additional Dialogue Recording or Automatic Dialogue Replacement (ADR) in which dialogue captured during production is gradually replaced with studio recorded alternatives.
The term picture cueing should alert us to the fact that the click-track approach (also invented by Newman) was originally developed so that music could be recorded away from reference to the moving picture. Once recorded, the music editor would take the carefully timed audio and marry it to the film. Synchronisation via a click-track is assured by maths, mechanical devices and methodical performances.
As the emotional gamut expressed in movie visuals and scripts developed, Newman pioneered, the now familiar sight of, the conductor before the orchestra facing a projected print of the movie, so that he might communicate both the synchronisation requirements and the emotional requirements demanded of the orchestra by the composer.
In the days of film, each streamer and punch was derived, like a click-track, by punching a hole in a film frame. But in this case, instead of the punch being in the non-visible soundtrack portion of the frame, the punch was made in the visible portion of the frame. Coinciding with the exact frame where a hit was required, the film was punched so that a good deal of the entire visible portion was removed. The preceding 48 frames (equivalent to exactly 3 feet of film) were then scored with a sharp object so that the emulsion was removed in a long diagonal line. When projected, this technique resulted in a bright, near-vertical line moving from left to right across the frame over a period of 2 seconds and finished with a bright, white flash as shown in the animation (above right).
Sometimes the scored line was coloured with ink to distinguish different types of streamers. There seems to have been little standardisation on streamer colours but generally, a green streamer indicted the beginning of a cue, a yellow or white streamer a musical "sync" point or a change of scene, and a red streamer for the end of a cue.
Nowadays, of course, most streamers and punches are produced electronically. An example of a short clip with streamers and punches is given below.
Using this technique, the conductor (or nowadays the composer) can control the music with relative freedom and has, by means of the streamer, a two-second warning of a hit, the exact position of the latter indicated by the bright flash of the punch. In the score (if there is one), a streamer is indicated by a line terminaing in a bar, and a punch as a cross in a circle. Both are illustrated in this example4.
Two pops and three-pops
A Two pop (2-pop) is a 1 kHz tone that is one frame long and placed 2 seconds before the start of a video sequence or programme. Rather similar to the old film clapperboard, the 2-pop is a simple and effective method of ensuring synchronization between sound and picture in a video or film. If used, for the composer, it is synchronous with the first frame of the (green) start streamer.
A 2-pop is typically placed at the end of a visual countdown. Only the first frame of the "2" is shown, and the remainder of the 2 seconds prior to the program is black. This provides a unique point of reference where the frame-long image and frame-long sound should align. It is often used for cueing upcoming television programmes in master-control or TV continuity.
The way a 2-pop relates to a streamer and punch is illustrated below.
Three-pop
A variation of the 2-pop, called (rather confusingly) a three-pop is a cue used in post-production dialogue replacement. A three-pop consists of three, one-frame bursts of 1kHz, once a second for three seconds. It acts as a countdown guide for the actor to synchronise the re-recorded dialogue with the on-screen production track recorded at the time of filming. It sounds like this:
Since the 1990s, digital sequencers programs have become the staple of the film and music production industry. MOTU’s Digital Performer (right) is a particular favourite for music to picture work, but Cubase, Logic Pro X, Pro Tools, Reaper and Ableton Live also all have their adherents. All the programs feature MIDI and audio recording which may be straightforwardly synchronised to a video file.
The programs feature intuitive tape-style recording layouts and are often coupled with scoring capabilities, digital processing effects and comprehensive, integrated sample-based sound libraries including: pianos, guitars, basses, church organs, electric organs, orchestral and individual string instruments, brass and woodwind choirs, synths, ethnic instruments, various choirs (boys, childrens, church, gospel etc), individual voices, percussion and sound effects. In short, pretty much all the tools required for composing and realising underscore in one box!
This technology has fostered an entirely new role as the jobs of composer and performers for film music have collapsed into the one-man (or woman) composer-performer.
Even for the (nowadays presitigous, or "old-school") composer intending to work ultimately with a real orchestra, the sequencer and sampler is used to create mock-ups of the ultimate orchestral performance and the sequencer data is used to generate the various orchestral scores and parts.
In a modern workflow, the composer-performer will set-up the sequencer with a synchronous video "window". Watching the on-screen action, she will decide a general tempo and time signature which feels right for the sequence and set these as parameters on the sequencer; usually in dialog boxes adjacent to the tape-like transport controls.
Hits in the on-screen action will be set by shuttling the video to identify the exact instant a particular hit is required. Once identified - in a wonder of integration - these time-based cue-points will automatically appear on the sequencer timeline. By "tweaking" the tempo slightly up and down, these cues may usually be brought to coincidence with a musical beat. (Note that this is the direct analogy of flicking forward or back a few pages in a Click-book to ensure coincidence of beat and elapsed time. See the box How to use a Click-book to calculate the correct tempo above.)
If hits are required to appear on down-beats (which is usually much more convenient and natural musically), the time signature of one or several bars is adjusted so as to conform the cue-point to a musical downbeat. So, for example, if a hit is conformed nicely to the beat, but that beat is the third beat of a 4/4 bar, a bar of 2/4 will be inserted prior to the cue-point so as to bring the hit onto a downbeat.
You'll see that the spreadsheet has various fields marked in YELLOW. These are free fields and you can change the data in these fields. Parameters which may be adjusted in the Cue calculator page are: BPM; frame-rate; time-signature; initial timecode value; and sequencer resolution/quarter-note. The Cue calculator spreadsheet calculates timecode on the basis of tempo, frame-rate and time-signature. You can also program the timecode offset of the first cue point.
The second page of the spreadsheet (the Hit Calculator) is intended as an iterative tool. It allows the timecode values of each hit to be programmed (in the upper table). Once the frames-per-second value is inserted, BPM may be adjusted and the time-sgnature, to maximise the number of hits to precise musical beats. These are annotated in the lower table and are coloured green when a hit is within a main beat or half-beat of the music.
Once a rough tempo and time-signature has been decided, adjust the value of the tempo to maximise the number of green indications in the last column of the lower table, this will ensure that the maximum number of hits coïncide with musical beats.
The third sheet is a modern version of a Click book with the advantage that the BPM (rather than frame-time) may be input as any value. As with the Hollywood Click books by Knudson and Brinkman, the subsequent time of all clicks (in minutes and decimal seconds) are calculated up to the 599th click.
At the top of the page, the BPM may be derived from the frame and eighth number. Obviously, this is a legacy feature.
Read the SoS review of Music Engineering by Richard BriceThe last chapter of Richard Brice's Music Electronics condenses Brice's knowledge of video techniques gleaned from a lifetime working in television, into a chapter which covers everything from the theory of scanned images and persistence of vision to digital picture compression techniques and practical issues for the musician such as synchronisation and of writing, performing and recording music for film and television.
Free Download
The last chapter of Richard Brice's Music Electronics is available free from Phaedrus Audio. Just click on the icon below for an automatic email to be generated. When we receive the email, we will send you an .pdf file by return.
1. How the MTV generation turned movies into long music videos. L. Rosewarne, The Guardian 6 January 2014
2. Despite being antiquated, examples of the Knudson book (the title is Project Tempo) sometimes appear in the pages of on-line auction sites and command very high sums (€500 in 2019).
3. Born in New Haven, Connecticut in 1900, Newman was considered a child prodigy for his piano playing by the age of eight. Via a period as a conductor on Broadway, Newman started scoring films in the 1930s and became one of the most important, most influential and most respected figure in the history of film music. He received an unprecedented 44 Oscar nominations, and his 9 Academy Awards are more than any other musical director or composer has ever received.
4. The Art of Writing Music. John Cacavas Alfred Music Co., Inc. Los Angeles 1993.
Address all mail to sales@phaedrus-audio.com
