Jill Walker Rettberg
University of Bergen
PB 7800
5020 Bergen Norway
Jill.walker.rettberg@uib.no
Tel: +47 55588431
Keywords: multimodality, emoji, gestures, social media, video, lipsyncing
A teenager smiles on the screen of my mobile phone. His hands move fast across the screen, shaping one hand sign after another to the lyrics of Justin Bieber’s “The Beauty and the Beat”. The sound is from a cover version of 15 seconds of Bieber’s song, and the boy’s lips move in perfect synchrony with the words that are sung. He is a proficient lipsyncer and has made many popular videos on musical.ly, an app where young people share lipsyncing videos and other videos. “We gonna party like it’s thirty-twelve tonight,” I hear, as the teen holds his arm up in a joyful fist, pumping it a couple of times to signify “party”, then rapidly moves his fingers to show the digits three- zero-one-two. The boy’s fingers flow together as he makes a sign for “tonight” by laying his hand under his cheek as though to rest. The song continues, though the version the teen lipsyncs to is not sung by Bieber, but is a 15 second cover of the song created by another musical.ly user.
We gonna party like it’s thirty-twelve tonight
I wanna show you all the finer things in life
So just forget about the world, we’re young tonight I’m coming for ya, I’m coming for ya.
As the song flows on, the lipsyncer has trouble finding precise translations for each word of the lyrics. He sweeps his hair back, holds his hand above his eyes (perhaps signifying “show you”) and gestures broadly (perhaps to mean “all these things”), turning back to the camera with his thumb and finger forming a circle (signifying “OK, good, fine”) for the line “I wanna show you all the finer things in life,” then just moves with the beat, looking a little uncertain, until he jumps up and starts dancing, leaving the camera propped up and still recording. Another boy, previously hidden behind our lipsyncer, is now visible, and looks at the lipsyncing dancer in surprise, then butts him with his hip, until both fall down in laughter and the video loops back to the beginning again.
This short video has almost 50,000 likes (hearts), and is one of the 12 million videos uploaded to musical.ly every day. There are many fascinating aspects of muscial.ly and its use that each deserve a research paper to be written about them, but in this paper, what I am most interested in is the hand signs that lipsyncers like the teen described here make. The hand signs amount to a fairly complex gestural language that is understood and used by musers (as users of musical.ly are called) and that is used to perform the lyrics of songs visually. I propose that the gestural language that accompanies lipsyncing videos on musical.ly is similar to emoji in that it is a system of pictograms, that is, visual representations of objects that signify a closely related concept or object (Chandler & Munday, 2011). The use of pictograms, whether in the form of gestures or icons, is becoming a common feature of informal, first person communication on the internet, and is used to complement writing, selfies and videos. It seems to be more common in asynchronous communication, that is, when there is a lag or a wait before the speaker can see the response of the person or people they are addressing. So we are more likely to see emoji used in an written SMS or a visual snap than in a Skype video conversation, even though the response may be almost- instantaneous, because the speaker cannot see their conversation partner’s response as they are forming their words, whether in writing or video.
To explore this proposition, I will analyse the use of hand signs in lipsyncing videos on musical.ly. I look at how gestures have been used in communication previous, and how they have been codified. I describe one sequence of hand signs in detail, and give examples of other signs, using both musical.ly videos and YouTube tutorials as a source. I then consider the multimodal nature of the lipsyncing videos, and discuss the hand signs as a form of performance. Finally, I open up the discussion to talk about musical.ly as a social media platform, and the variety of videos that exist on the platform. But first, I should explain what the musical.ly app is.
What is musical.ly?
Musical.ly is an app, launched in 2014, where users can share 15 second lipsyncing videos and other short videos. Users can either select soundtracks from the extensive music database available on musical.ly, use songs they have on their phone, record live sound, or use sound from other musers’ videos and skits. They can add visual filters to their videos like on other video- and image-sharing services, as well as applying less common time-based effects, such as a time loop, where you select a second of the video that is repeated a few times.
The app was launched in Shanghai in October 2014 by Alex Zhu and Luyu Yang, and has rapidly grown to become one of the most popular apps globally for tweens and young teenagers. On an average day in July 2016, 12 million videos were uploaded to the app, which has grown from 10 million users in 2015 to 90 million users in mid-July 2016 (Rys, 2016). The rapid growth of the app is all the more interesting for its very specific demographics: it is extremely popular with tweens and young teenagers, but many adults have not heard of it – unless they have young people in their life who have shown it to them.
Communicative Gestures
Gestures are fundamental to human communication. People all over the world gesticulate as they speak. Early homo sapiens used gestures to communicate before we developed spoken language, possibly with quite complex syntax (Corballis, 2002). While sign languages for the deaf take time to learn, most humans can manage some basic communication without words, for instance when trying to buy an item in a shop in a country where we do not know the language.
Some of our gestures are encoded signs with clear meanings. When wave our hands as we speak, or slump when we are feeling dejected, this is not necessarily conscious. Other gestures are agreed upon signs, where the gesture is a signifier referring to a specific signified that is commonly understood within a certain culture. Semaphore (which can be used with hands as well as flags) and sign languages for the deaf are examples of specialized gestural languages, known only to some people. We also have vernacular signs that are generally understood by all: for instance, nodding the head means “yes” in many (though not all) cultures, and shrugging the shoulders means “I don’t know”. In many countries, thumbs up means good and showing the middle finger is an insult.
Other gestures are indexical, to use the linguist Charles S. Pierce’s term, and they have a natural connection to their meaning: a hand held to the ear with outstretched thumb and little finger mimics holding an old telephone to the ear, and signifies “call me” or some sense of telephone. Moving the right index finger against the flat palm of the left hand signifies writing, or, if in a restaurant and directed to a waiter, it means please bring the bill. Indexical signs like these are equivalent to pictograms in writing: they visually mimic that which they represent, much as emoji do. Some emoji represent gestural signs: thumbs up, hands together in prayer, a fist bump.
Technology affects gestural communication. Writing obviously prioritises verbal language, and with print, writing became heavily standardized. As writing has become conversational, as in email, online chats, SMS messages and social media, we have increasingly found ways to represent non-verbal and gestural modes in or alongside writing, through emoticons from the 1970s, which only used standard typographical characters, and more recently, with emojis, which offer a whole new set of standardized icons that can be used alongside alphabetic characters.
Gestures have been studied for their role in supporting verbal communication since, at least, the ancient Romans, when Quintilian and others described how orators used gestures and bodies to strengthen their message. Cicero coined the term chironomia in his De Oratore (55 b.c.) for the study of non-verbal communication through hand and arm gestures that accompany speech. (Verhulsdonck & Morie, 2009).
In the 17th and 18th century scholars studied and analysed gestures of ordinary people and of actors and produced dictionaries and systems for understanding these gestural languages, describing the field as chironomia or chirologia (Bulwer, 1644; Gilbert Austin, 1806). Another systematic analysis of conversational gestures from about the same time is Andrea de Jorio’s book on Neapolitan gestures. Some of these gestures are used in musical.ly videos, such as the first gesture shown Figure 1, a finger held up to the lips to signify silence, or the admiror in Figure 2.
Figure 1: These 19th century Neapolitan gestures are from a 19th century book (1832, p. 427) and signify as follows: 1. silence, 2. no, 3. beauty, 4. hunger, 5. to mock, 6. weariness, 7. stupid, 8. squint, 9. to deceive, 10. cunning.
Figure 2: A chart from John Bulwer’s 1644 book Chirologia, showing different hand gestures to be used in oration, with their meanings.
Giorgio Agamben argues in his “Notes on Gesture” (Agamben, 2000) that photography killed gestures by freezing them in time. “By the end of the nineteenth century,” Agamben writes, “the Western bourgeoisie had definitely lost its gestures” (49), and in silent movies and a few other genres, “humanity tried for the last time to evoke what was slipping through its fingers forever” (53). Agamben saw photography as killing gestures by locking them into still slices of time: “images are the reification and obliteration of a gesture (it is the imago as death mask or as symbol)” (54). Cinema, on the other hand, has its center in the gesture, Agamben writes. While this is a seductive line of argument, I must point out that this freezing of gestures was achieved by the detailed drawings of gestures in books on chironomics (as shown in Figure 1 and Figure 2) well before photography was invented. The urge to freeze gestures and categorise them seems more related to the Enlightenment desire for dictionaries and encyclopedias than to the 19th century technology of photography, although perhaps an earlier technology, that of print, can be blamed for the Enlightenment. Elizabeth Eisenstein has argued that print technology is a prerequisite for the kind of large-scale collection, categorizing and distribution that was so important in the Enlightenment era (Eisenstein, 1979), and which we see in the books on chironomics.
Agamben sees cinema as a technology that rekindles gestures in our culture. Silent cinema in particular required large and expressive gestures, as it had to do without sound, and verbal language was limited to short captions. Early internet communication had no room for gestures, being almost entirely text-based. Now, when video is gaining importance, it makes sense that gestures are also regaining prominence. It is interesting, though, that gestures, at least as seen on musical.ly, are codified, deliberate representations rather than the largely unintended or at least unplanned, natural gestures of face to face communication.
Hand signs on musical.ly
While many of the gestures shown in books on chironomics or chirologia involve the whole body, and represent emotional states, hand signs on musical.ly usually only use a single hand (as the other hand holds the camera) and are almost always indexical. Like emoji, these hand signs are visual representations of an object, a concept, an action or an emotion. We might call them pictograms, as they generally have a visual likeness to the action, emotion or object they signify.
Figure 3 shows a typical set of hand motions as used in a musical.ly lipsyncing video posted by a young teenager in [country removed for review] in August 2016. These signs accompany the lyrics “I’m way too good for you”, and the lipsyncer uses signs to represent certain words from the lyrics. First, she shows two fingers up to signify “too”, then quickly shifts to a thumbs up for “good”, back to the two fingers up again to signify “to” then points at the camera for “you”. The whole sequence takes less than a second.
Figure 3: Stills from a musical.ly video showing hand signs to the lyrics “I’m way too good for you.” These images will be further anonymised before publication by tracing over them so only a drawing is shown.
In the second part of the video, shown in Figure 4, the muser doesn’t use hand gestures for every word, as in Figure 3. The lyrics for this part of the video are a little more abstract: “You take my love for granted / I just don’t understand it.” In the first line, the muser chooses to sign “you” (index finger pointing at camera) and “love” (hand held in the shape of half a heart). The last line is shown in just one sign: the finger points to the side of the head.
Figure 4: These stills show the hand motions used in the next part of the Musical.ly video. The lyrics for this sequence are: “You take my love for granted / I just don’t understand it,” and they are from the song “Too Good” by Drake and Rhianna, although the audio that this muser is lipsyncing to was created by another musical.ly user.
Hand motions on Musical.ly are almost always direct interpretations of words in the lyrics. Sometimes, as with the use of two fingers for “too” and “to”, the signs refer to homonyms of the word used–although we could also interpret the two-fingered sign as signifying a “2” as often used in texting, rather than signifying “too” and “to”: “I’m way 2 good 2 you.”
In many ways, performing a Musical.ly lipsyncing video is similar to the performance of a musician reading sheet music to play a tune. The muser listens to the music and the words, practices singing along, and chooses which words to represent visually through hand motions. Although most musers I have seen use the same set of signs (see Table 1), there is some leeway for personal interpretation. In Figure 3 and Figure 4 this is evident in the selection of which words to sign and which not to sign, but also in the conclusion of the video, which is without words. After the part of the video shown in Figure 4, the muser puts her hand in front of the camera, making the screen go black. Then the video cuts to a slow motion sequence of her first looking down, then smiling, her hand moving in towards her chin as though she is going to lean on it. She appears to be about to blow a kiss to the viewer (or to her own image on the front-facing screen of her mobile phone camera) when the video ends, or rather, loops back to the beginning.
Table 1: Examples of hand signs used in musical.ly lipsyncing videos.
| Bed/sleep/rest | Tilt head, leaning against hand, palm away from head. |
| Cold | Wrap hand around self, shake camera in shivering motion |
| Come here | Beckon with index finger |
| Cry/tear | Touch finger to cheek, pull down as though a tear is running down cheek |
| Die/death | Make a gun from thumb and two pointed fingers and point it at your head, or, hold hand horizontal and slash across throat |
| Drink | Hold hand to mouth and move as though drinking from cup or bottle |
| Drive | Hand on imagined steering wheel, move back and forwards |
| Face | Hand moves down against side of face, palm towards face |
| Half/middle | Hold hand flat and vertical, 90 ̇ to face, thumb towards face, little finger to camera, and (optionally) move up or down, moving camera in opposite direction |
| I/me | Thumb points to self |
| Lie | Fingers up, palm towards face, move hand down in front of face while wiggling fingers as though in intricate pattern |
| Look | Hand held horizonantally above eyes |
| Love | Make the shape of half a heart with your fingers and thumb |
| Money | Rub fingers together as though holding paper money between them |
| No | Shake head or hold index finger up and shake back and forwards |
| Numbers | Fingers held up, palm facing self |
| Peace | Peace sign (two fingers held up, palm facing camera) |
| Pray/hope/miracle | Same as “half/middle”, although here the reference is of two hands held palm to palm as though in prayer. |
| Run/leave/go | Move fingers as though running |
| Sing/talk/say | Hold hand in front of mouth and move thumb against fingers to mimic a mouth opening and closing to speak. |
| Stressed out | Hands against top of head, upset expression on face |
| Take | Begin with outstretched open-palmed hand, move towards self while closing hand |
| Think/wonder | Tap one or more fingers lightly against side of head |
| Time | Hold forearm horizontal and look at wrist as though looking at a watch |
| You | Index finger points to camera |
Table 1 shows hand signs that are commonly used on musical.ly, and which I have learned from from watching hundreds of musical.ly videos, from conversations with users and from tutorials on YouTube.
Most of the signs are, as mentioned, directly representational: they refer to a specific word in the lyrics. There are also some signs that are non- representational, or that function more as a dance, or to underscore the beat of the song. For instance, when moving the camera (which is done almost as a form of dance move) many users hold their left hand at the bottom of the frame, palm up and little finger slightly raised as they move the hand from right to left. “When you move the camera, you move your hand like this,” a twelve-year-old user explained to me, holding her left hand horizontally, palm up, little finger slightly raised, and moving the hand from right to left as she held the camera in her right hand. Similarly, Nigerias Blessing explains how you can move the camera in a square in time with the beat of a song if you do not have words you want to sign.
The hand signs are usually performed with the left hand, as the camera is held in the right hand. In her tutorial on musical.ly hand motions, muser Nigerias Blessing explains the importance of camera motions (Nigerias Blessing, 2015). Camera motions certainly enhance the hand signs, and the ability to use camera motions effectively is a marker of real skill and prestige. In her video, Nigerias Blessing explains how the camera often should move in the opposite direction of the hand sign. But some hand signs, in Nigerias Blessing’s view, need to be combined with appropriate camera motions. For instance, the sign for “drop” involves holding your hand with fingers down, tight together, as though holding something, then opening the fingers as though dropping that thing. Blessing suggests holding the hand quite high, and keeping it high throughout the sign, while moving the camera down. She also embellishes the “shoot” sign by shaking her camera a little bit, mimicing the rat-tat-tat jolts of a series of bullets being shot. For hitting, she makes a fist, aims it at the camera, and moves the fist and camera towards each other as though they are about to hit.
Nigerias Blessing’s is the most thorough tutorial I have found on musical.ly hand signs. Other popular tutorials tend to be a lot less specific. Nigerias Blessing’s tutorial has been watched almost 200,000 times on YouTube, although she is not a musical.ly superstar, with less than 2000 followers on muscial.ly. Baby Ariel is one of the most popular users on musical.ly, with over 11 million followers, and at 15, she has established herself as a professional social media influencer based on her success on musical.ly. A lot, though not all, of her YouTube videos, give users tips on how to make good musical.ly videos. Baby Ariel’s advice is less specific than Nigerias Blessing’s advice is: “I shake my camera a lot. I don’t know how I do it – I…shake it [looks into her mobile camera while gently shaking it] … like, really softly, but it looks like, bah! [Looks into camera again, moving camera sharply] – and it turns out good!” (Baby Ariel, 2015). In another video, Baby Ariel teaches her mother to make a Musical.ly video (Baby Ariel, 2016). Here, she explains in detail how to move the camera either in the opposite direction of the hand, or following the hand.
Camera movements are, as these tutorials show, an important part of the Musical.ly aesthetic, both as direct complements to hand motions and as a form of dance move that follows and emphasizes the beat of the music. Camera movements appear not to signify directly on their own, as many of the hand motions do, instead they “look good”, as the tutorial makers often say, often visibly searching for more specific words as they speak but ending up with “good.”
This ability to make it “look good” clearly involves a fair bit of skill and experience. It is not a skill that can be learnt simply from watching other videos or tutorials, or by your friend showing you. You need to practice a lot in order to succeed. BabyArial’s video of her mother learning to use the movements demonstrates this by showing how inept her mother’s first attempts are, and by showing that although the mother improves with practice, she still lags far behind her daughter’s skill level.
The skill required in understanding hand signs on musical.ly is not only on the part of the performer (the lipsyncer) but also on the part of the viewer of the lipsyncing video, who has the pleasure of interpreting the hand signs. In this sense, the hand signs can be seen as a puzzle, or more directly, a rebus, as journalist Clive Thompson has argued (2013).
Figure 5: A rebus that can be read as “Can you see well?” [check about permission to reprint, found this in Preston 1982 – might need to find a different example of a rebus]
In their paper on emojis (2015), Luke Stark and Kate Crawford connect Thompson’s comparison between emojis and the rebus to linguist Michael J. Preston’s comment that “Just as a pun is conventionally met with a groan, the rebus is often acknowledged by a statement of disdain, unless, of course, one knows a rebus or two and can respond in kind” (Preston, 1982, p. 119). This is often the reception lipsyncing videos receive when spread beyond specialised sites like musical.ly, such as Snapchat. Luckily for lipsyncers, those who would disdain the puzzles of hand signs do not tend to spend much time on muscial.ly, though no doubt, as lipsyncing becomes more mainstream it will be mocked more, following the pattern we already know from blogs, selfies and other forms of self-representational digital culture (Burns, 2015).
Multimodality: Words, Images and Gestures
With touch screens, we have become increasingly accustomed to using gestures as codes or signs that have very clear meanings. We shake our phones to undo a mistake. We swipe right on Tinder to dismiss a photo of someone who does not please us (David & Cambre, 2016) and swipe down when watching a live story in Snapchat to go back to the first screen. Cultural practice is evolving to incorporate these new modalities, both in vernacular communication and creation and in artistic genres, although we still lack a sophisticated rhetorical understanding of gestures of interaction (Verhulsdonck & Morie, 2009). In her analysis of kinetic poetry and other forms of electronic literature that use the affordances of digital media, Alexandra Saemner proposes that we consider combinations of gestures and words or images pluricode couplings, which “involve two different semiotic systems, a text and a icon, within the same active support of the sign” (Saemmer, 2013). In the works of electronic literature that Saemmer discusses the gestures can either be movements the reader must make in order to access the poem (touching the screen, clicking a link, dragging a word) or movements made by the words and letters on the screen. Semoticians have used the term multimodal for some time to describe a text (text is used here in the broad, Barthesian sense) that uses several modalities, for instance, an ad in a magazine that uses both text and image, or a movie, where moving images are combined with sound.
In a muscial.ly lipsyncing video, I propose that we think of the hand signs as an independent modality. Thus, we can identify at least four distinct modalities: the music, the lyrics (the words sung being a linguistic modality), the moving image, and the hand signs.
The image in a magazine ad illustrates the text, and expands the meanings and connotations of the ad as a whole. The relationship between the lyrics and the hand signs in a musical.ly lipsyncing video is even closer than this – rather than expanding the meaning, the two modes of words and signs repeat each other. In Alexandra Saemmer’s words, there is a coupling between word and hand motion. The word “love” and the hand held in the shape of half a heart against the chest are a pluricode coupling, that together, means love. In a sense, then, the hand motions are redundant. They simply repeat the meaning that is already conveyed in the lyrics.
Lipsyncing as Performance
If we instead think of the hand signs as an enactment or a performance of the lyrics, the apparent redundancy makes more sense.
Music audiences have always responded to music we like in a bodily way. We dance, we sing along, we invent. When I first saw musical.ly videos I immediately recognized them as similar to the way my friends and I would use music in the 1980s, when we were young teenagers. We would use cassette recorders to tape pop songs from the radio, then listen to them over and over again, stopping and starting the tape so we could write down all the words. We would sing along, alone or in groups. We invented dances to them that we performed in small groups at end of year school concerts for parents, or in gym class. Sometimes I would stand in front of the mirror, pretending to sing the way my idols sang. Later, I listened to my favorite songs again and again on my WalkMan on the bus to school, mouthing the words silently so as not to disturb my fellow passengers. Some songs had specific dances that we had learned from the music video, although we were rarely able to watch music videos as MTV was only available through satellite television and hardly anybody had that. I am sure that my friends and I, at twelve or thirteen, would have embraced Musical.ly had we had access to it.
Young people have listened to and performed the popular music they love for generations, and as Kyra Gaunt has described, pre-adolescent girls have their own tradiitona of music and performance that can influence adult musical culture, as girl’s game-songs have influence hip hop music (Gaunt, 2012). What has changed with apps like musical.ly is the scale at which they are able to share their performances (potentially to the world, not just their class), the scale at which they can access other versions of performances to the same music, and the fixity of the media they use, which allows them to record and edit their performances far more easily than my generation could. It is not surprising that a global platform for sharing the kind of performance of popular music that tweens and teens for so long have engaged in would be hugely popular.
A lipsyncing video with hand movements can be viewed as a performance of a set text, much as a musician plays a tune from sheet music or an actor acts from a script. Of course interpretation and individual choices are involved, but even though two performances of the text may be very differnet, they can generally be recognised as performances of that text. The Musical.ly app encourages this by allowing users to see many performances of the same song. By tapping the small circle in the lower right hand corner when watching a video, we arrive at a page that shows information about the song that was performed, with thumbnail videos of other performances of the same video (see Figure 6).
Figure 6: The page for an individual song includes moving thumbnail videos of its most popular or most recent lipsyns beneath. The top three are musical.ly celebrities @lisaandlena, @babyariel and @mackenzieziegler, who all have several million followers.
This availability and even celebration of multiple interpretations of a set text is an established pattern of the internet today. Knitting was revolutionised by platforms like Ravelry.com that allowed knitters to share their projects and connect them the patterns they are based on. This created a database where you can often see several hundred different ways in which a single pattern has been worked up, looking at how other knitters have adjusted the pattern to their liking by using different colours, different yarns or changing the pattern. Before the internet, in the age of mass media, knitters usually only had access to a few local friends who knitted, and to the official patterns published by yarn companies and sold in yarn shops. With the internet, creativity blossomed, not least because of the possibility of seeing hundreds of different ways that a single pattern could be knitted. Memes are another creative area where we see many interpretations of an original, and where databases, such as Knowyourmeme.com, allow users to browse through catalogues of variations on a shared original. These modes of creativity encourage a thinking where the original pattern or meme is not something to be revered, but something to be developed and made one’s own.
On Musical.ly, creativity is similarly encouraged by the invitation to make an original your own. When you watch somebody else’s video, you have the option to “start duet now!” which means making your own recording of the same song, which Musical.ly will edit together with the first one you saw to create a duet. Or you can click the circle in the lower-right corner to see information about the song, other examples of videos made for it, and a tempting, bright yellow button labelled “shoot now!” inviting you to create your own version.
The Young Teaching the Old
Musical.ly is a young person’s app. Swiping through the latest videos from users in my city [name removed for blind review] I see that almost all the videos are from children or young teenagers, mostly at that age around 12 or 13 where people begin to experiment with makeup and hairgel and emerging sexuality. Some are clearly younger children of around 10, and a few appear to be older teens, perhaps in their twenties.
I was introduced to Musical.ly by a young family member, much as Baby Ariel taught her mother to use the app in her YouTube tutorial. Swiping through videos on Musical.ly, this is clearly not uncommon – among the many young teens, there are a few parents and grandparents, usually appearing with young people. As an article in Business Insider stated, “Unless you live with a teenager, you’ve probably never heard of Musical.ly. If you do, then you’ve probably already appeared in one of your kid’s music videos” (Carson, 2016).
GangstaGrandma is a popular user [check] who posts videos that are clearly made in collaboration with her grandchildren, who appear to range in age from about 8 to the mid-teens. Several of her videos show her dancing with children in the background, and of course, her username explicitly introduces her based not on her own general interests but on her relationship as a grandmother to children and teens.
Interestingly, parents and grandparents on Musical.ly seem to use hand motions and camera movements less than young users. This supports my argument that the hand and camera movements are a way of signifying cultural capital and status as an experienced, knowledgable creator of Musical.ly videos. Adult users rarely if ever invest the time required to achieve the level of skill that their children often have.
Instead of following the in-app conventions and adhering to its system of cultural capital, parents on Musical.ly often break the rules. This is no doubt in part because they lack the knowledge to follow the rules, but there may also be an element of mockery or derision in their subversion of the rules of the app. As an adult, I found my own first attempts at lipsyncing quite disheartening. My young teacher made perfect videos. To begin with, I thought my less impressive videos were due to my wrinkles – my teacher’s unwrinkled, fresh complexion, further aided by the adept use of filters, made her look stunning, whereas I looked at my own face and mostly saw old. There was more though. My teacher looked alternately demurely and coyly at the camera in a way that fits a stereotype of sweet young thing, but that looks foolish when a midde-aged woman attempts it. Or perhaps I simply worried that such a look would be at odds with my professional self-confidence, which I have carefully developed over decades. Then there was the skill with which my teacher used hand and camera movements. It took me a while to even perceive that these motions were an essential part of the performance. When I realized, and started asking for explanations so I could learn how to do them, I quickly realized how difficult it was to make them appear easy and effective. My attempts simply looked silly, so I responded by exaggerating so that I could come across as comic instead of simply inept.
Very few of my friends are on Musical.ly. By connecting with Facebook, I found a handful (all parents of tweens and teens) who had apparently joined but never posted a video. Only one of my Facebook friends, also a parent, had enthusiastically thrown himself into the game, and his videos were strikingly different to the ones my own 12-year-old teacher was showing me, and to the videos that pop up when swiping through the featured or local posts. He used a black and white filter, propped the camera up so his whole body was visible, and danced in perfect 80s style to The Clash’s punk hit “Shall I Stay or Shall I Go”? Of course his performance appealed to me. Music from my own teens, a style of dancing I could recognize. The black and white filter lent it a sense of nostalgia that perhaps was heightened for me by the sense of our own long gone youth. This was the way in which my generation, or my subculture within my generation, sought to find our identity through music. I felt a sudden sadness that we had not had a Musical.ly app back then. Oh, I would have loved it.
You can browse musical.ly videos by swiping through videos posted by people you follow, or looking at the featured videos (selected by musical.ly staff) or you can see a feed of all videos posted in your own city. Swiping through the local videos I found a different kind of parent video: a father and a young girl of about 10 or 11 working together to create a funny video to the song “Smoothie” (check). The camera is set up so that there is a table with a half-filled glass between us and the girl, so that the girl looks as though she is inside the glass. The dad stands next to her. The lyrics say “You want a smoothie? Get a smoothie! But if you want a frappe, get a frappe!” The girl pops up and down, as though she is in the glass, while her dad shrugs. It is quite an amusing video, at least to an adult viewer. A week later, another Facebook friend of mine let me follow her private account. She had posted a series of videos with her nine or ten year old daughter as the star. Her daughter held the camera, and had apparently cast her mother as a side-kick – although the account was in the mother’s name.
Parents and grandparents aren’t the only group musers are including in their videos. In other videos, we see tweens teaching much younger children to use the app. Sometimes, the younger child simply gazes, thrilled, at the screen as the older child tries to show them how to move their hands. Other times, the child is given a clear role, as in one video showing a small child attempting to lipsync “You mess with one of us, you mess with all of us”, a spoken line from an Alvin and the Chipmunks song. The child is alone in the first part of the clip, but as the words “all of us” are spoken, four older children, look as though they are around 10-13 years old, pop up behind the sofa the preschooler is sitting in. Many of the most popular lipsyncs to this line involve groups of young children, although other, made by older teens or adults, tend to play on breaking with that expectation to create a comic juxtaposition – for instance moving from a closeup of a face to a wider shot showing the person reflected in mirrors in a mise en abyme, or cutting to a wide angle shot that shows a host of small lego figures behind the lipsyncer.
Not Just Lipsyncing
Looking at videos from my city, or watching YouTube tutorials about how to create a Musical.ly video, lipsyncing videos appear to be by far the most common content on the app. However, reading about the company in business magazines, or reading their self-description in the FAQ on their support website (Musically.zendesk.com) or simply swiping through the featured videos in the app, that is, the videos the company itself has selected to be featured, it is clear that the company has much broader ambitions. Many of the featured videos are skits and jokes, not dissimilar to those seen on Vine. In the FAQ for musical.ly on Zendesk, the question “What is musical.ly?” is answered with no mention of music or lipsyncing:
Musical.ly is a free service and social media platform for creating and sharing short videos. Everyday, millions of people choose musical.ly as an outlet for creative expression and communication. More than just a video creation tool, musical.ly is a fully realized platform for connecting individuals to a vibrant community of content creators. (Accessed 7 August 2016)
Clearly, the company aims to develop Musical.ly as a more general social media platform rather than sticking to lipsyncing.
What, then, will happen to the hand signs? Will they continue to develop in the same way as non-standard punctuation and emoticons developed into emoji? Will they become part of a face to face vernacular communication, or will they remain tied to screen-based communication only? Will this gestural language, if we can call it that, become more standardized? Will we see a range of distinct dialects, as we have seen emoji are used differently in different cultures?
If you search musical.ly for tags like “angry”, “depressed” or “mad” you’ll find a different kind of lipsyncing than the kind I have described this far. Young people act out monologues from films or spoken word parts of songs, dramatically living themselves into a role. Other times, songs appear very deliberately chosen to tell a story. One muser I stumbled across, wearing low pig tails, careful makeup and dark-framed glasses, had posted a series of lipsyncing videos about depression and mental illness. Several of her videos used music made by another muser, who regularly records herself singing 15 second snippets of popular songs, such as Gnarls Barkley’s “Crazy”. A couple of dozen other musers have also created their own lipsyncs to this tune. Some of these seem deeply heartfelt. One, for instance, simple shows half a girl’s face, the one eye that is visible carefully made up, the rest of her face hidden in her bedclothes. She doesn’t move her lips. Towards the end of the clip, she pulls the bedclothes over her head. The black and white filter adds a sense of almost nostalgic aesthetics to it. These videos seem to show young people using excerpts from popular culture to express themselves, much as we have seen people use memes from popular media to express their own stories on Tumblr (Seko & Lewis, 2016, p. 14). Musical.ly also has its own homegrown memes, although the term meme doesn’t seem to be used. Instead, musers talk about challenges. For instance, popular musers @holly?? posted a short skit in response to a song that was popular in the summer of 2016about needing two phones, one for calls and one to play Pokemon Go. In her skit, she begins in the same way as the song, then breaks out “I only have one phone! And nobody calls me” (CHECK). Many other musers have lipsynced this skit, and in early August 2016 it was one of the top hashtags listed in the musical.ly app. In future research, it would be interesting to explore similarities and differences between the re-enactment or performance of popular culture we see on musical.ly and the reblogging and curating of popular culture we see on Tumblr.
Conclusion
Until now, we have thought of emoji and other forms of non-standard punctuation that are used to enhance writing in dgital media as being a way to add aspects of non-verbal speech that are necessary to conversation to a text-only medium. Some have argued that emoji, being machine-readable, also function as a way of shaping conversations and making them more easily analysed as big data. Emojis can also be seen as part of the capitalistic basis of today’s internet: Luke Stark and Kate Crawford see emojis in the context of affective labour, writing that “emojis create new avenues for digital feeling, while also remaining ultimately in the service of the market” (Stark & Crawford, 2015).
However, if we read hand signs on musical.ly as analogous to emoji in texts, we need to rethink the way we understand not only hand signs, but also emoji. Emoji are not simply about writing. They are about digital communicaton, and about self-representation and conversation in social media. Emoji often replace conversational gestures or facial expressions. On musical.ly, the skill in lipsyncing consists not only in miming the words of the lyrics, but in performing their meaning using hand signs, dance and facial expression. Both emoji and hand signs are remediations (Bolter & Grusin, 1999) of the gestural communication that is so fundamental to human conversation. As video and other visual forms of communication become more common online, we will continue to see how the human need for gestures leaks into digital communication.
Funding Acknowledgement
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Bibliography
Agamben, G. (2000). Notes on Gesture. In V. Binetti & C. Casarino (Trans.), Means Without End: Notes on Politics (pp. 49–62). Minneapolis: University of Minnesota Press.
Baby Ariel. (2015). Musical.ly Tutorial [YouTube video]. Retrieved from https://www.youtube.com/watch?v=EQq0FLlMuQE
Baby Ariel. (2016). Teaching my Mom How to Make a musical.ly. Retrieved from https://www.youtube.com/watch?v=X0FjhRqW8iU
Bolter, J. D., & Grusin, R. (1999). Remediation: Understanding New Media. Cambridge, MA: MIT Press.
Bulwer, J. (1644). Chirologia, or, The naturall language of the hand composed of the speaking motions, and discoursing gestures thereof : whereunto is added Chironomia, or, The art of manuall rhetoricke, consisting of the naturall expressions, digested by art in the hand, as the chiefest instrument of eloquence, by historicall manifesto’s exemplified out of the authentique registers of common life and civill conversation : with types, or chyrograms, a long-wish’d for illustration of this argument. London: Tho. Harper. Retrieved from https://archive.org/details/gu_chirologianat00gent
Burns, A. (2015). Self(ie)-Discipline: Social Regulation as Enacted Through the Discussion of Photographic Practice. International Journal of Communication, 9, 1716–1733.
Carson, B. (2016, May 28). How a failed education startup turned into Musical.ly, the most popular app you’ve probably never heard of. Retrieved August 6, 2016, from http://nordic.businessinsider.com/what-is-musically-2016-5/
Chandler, D., & Munday, R. (2011). Pictogram. In A Dictionary of Media and Communication. Oxford: Oxford UP. Retrieved from http://www.oxfordreference.com/view/10.1093/acref/9780199568758.001.0001/acref-9780199568758-e-2045
Corballis, M. C. (2002). From Hand to Mouth: The Origins of Language. Princeton: Princeton UP.
David, G., & Cambre, C. (2016). Screened Intimacies: Tinder and the Swipe Logic. Social Media + Society, 2(2), 2056305116641976. http://doi.org/10.1177/2056305116641976
Eisenstein, E. (1979). The Printing Press as an Agent of Change: Communications and Cultural Transformations in Early-Modern Europe (Vol. 1). Cambridge: Cambridge UP.
Gaunt, K. D. (2012). Girls’ Game-Songs and Hip-Hop: Music Between the Sexes. Parcours Anthropologiques, (8), 97–128. http://doi.org/10.4000/pa.116
Gilbert Austin. (1806). Chironomia; or, A treatise on rhetorical delivery. Printed for T. Cadell and W. Davies. Retrieved from http://archive.org/details/chironomiaoratr00austgoog
Jorio, A. de. (1832). La mimica degli antichi investigata nel gestire napoletano. Dalla stamperia e cartiera del Fibreno.
Nigerias Blessing. (2015). Musical.ly Tutorial 30+ HAND MOTIONS & EXPLANATIONS [YouTube video]. Retrieved from https://www.youtube.com/watch?v=2ICGnFAHePQ
Preston, M. J. (1982). The English Literal Rebus and the Graphic Riddle Tradition. Western Folklore, 41(2), 104–138. http://doi.org/10.2307/1499785
Rys, D. (2016, June 29). Fresh Off a Big Funding Round, Musical.ly Signs Its First Major Label Deal with Warner Music. Retrieved August 10, 2016, from http://www.billboard.com/articles/business/7423281/warner-music-group-deal-musical-ly
Saemmer, A. (2013). Some reflections on the iconicity of digital texts. Language & Communication, 33(1), 1–7. http://doi.org/10.1016/j.langcom.2012.10.001
Seko, Y., & Lewis, S. P. (2016). The self—harmed, visualized, and reblogged: Remaking of self-injury narratives on Tumblr. New Media & Society, 1461444816660783. http://doi.org/10.1177/1461444816660783
Stark, L., & Crawford, K. (2015). The Conservatism of Emoji: Work, Affect, and Communication. Social Media + Society, 1(2), 2056305115604853. http://doi.org/10.1177/2056305115604853
Thompson, C. (2013). The Prehistory of Emoji. Womanzine. Retrieved from https://issuu.com/lindseyweber5/docs/emoji_by_womanzine
Verhulsdonck, G., & Morie, J. F. (2009). Virtual Chironomia: Developing Standards for Non-verbal Communication in Virtual Worlds. Journal For Virtual Worlds Research, 2(3). Retrieved from https://journals.tdl.org/jvwr/index.php/jvwr/article/view/657