A key technology for sperm whale (Physeter macrocephalus) monitoring is the identification of sperm whale communication signals, known as codas. In this paper we present the first automatic coda detector and annotator. The main innovation in our detector is graph-based clustering, which utilizes the expected similarity between the clicks that make up the coda. Results show detection and accurate annotation at low signal-to-noise ratios, separation between codas and echolocation clicks, and discrimination between codas from simultaneously emitting whales. Using this automatic annotator, insights into the characterization of sperm whale communication are presented. The results include new types of coda signals, analysis of the distribution of coda types among different whales and for different years, and evidence for synchronization between communicating whales in terms of coda type and coda transmission time. These results indicate a high degree of complexity in the communication system of this cetacean species. Source code and data for our system is publicly available. A key technology for sperm whale ( Physeter macrocephalus ) monitoring is the identification of sperm whale communication signals, known as codas . In this paper we present the first automatic coda detector and annotator. The main innovation in our detector is graph-based clustering, which utilizes the expected similarity between the clicks that make up the coda. Results show detection and accurate annotation at low signal-to-noise ratios, separation between codas and echolocation clicks, and discrimination between codas from simultaneously emitting whales. Using this automatic annotator, insights into the characterization of sperm whale communication are presented. The results include new types of coda signals, analysis of the distribution of coda types among different whales and for different years, and evidence for synchronization between communicating whales in terms of coda type and coda transmission time. These results indicate a high degree of complexity in the communication system of this cetacean species. For example, Project CETI (Cetacean Translation Initiative), founded in 2020, is an interdisciplinary research initiative that uses advanced machine learning and cutting-edge robotics to better understand sperm whale communication 1 . The backbone of this effort is custom-built passive bioacoustic arrays covering a 20 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times$$\end{document} 20 kilometer area where these whale families reside (collecting over 30 TB/month), in conjunction with robotic acoustic and video tags on the whales, underwater gliders, towed hydrophone arrays and aerial drones to augment the rich contextual communication data. This initiative has led to a theoretical model using unsupervised machine translation to understand cetacean communication 2 . In addition, the method in 3 modeled codas as (variable-length) Markov chains, revealing new patterns of inter-clan sperm whale social learning. The work in 4 examines vowel-like spectral properties of codas and that in 5 elucidated a sperm whale phonetic alphabet. The method in 6 trains a transformer on click timings (inter-click intervals), which is able to predict sperm whale codas in an exchange based on long-term dependencies, as well as future diving behavior. In this case, given the importance of culture in conservation 9 , 10 , and that the cultural population structure of sperm whale clans is defined by acoustic repertoire 11 , 12 , an automatic detection and annotation of sperm whales codas will allow for the definition of clans present in previously unrecorded waters using passive acoustics, and thus provide not only a tool for understanding this species’ communication system, but also for their applied conservation and management. Current approaches Sperm whales are among the most acoustically active toothed whales, making them attractive species for passive acoustic monitoring (PAM). The two most important features for distinguishing between coda and echolocation clicks are the decay rate between pulses within the two click types, which is probably the result of different signaling pathways in the whale’s nose 29 , and their rhythmic structure. The echolocation clicks are produced in long series with a periodic pattern with characteristic inter-click intervals (ICIs) which are a function of the whale’s search range 19 . In contrast, codas are sequences of clicks with stereotyped rhythm and tempo 21 , which are thought to serve communication during The results show no significant difference between the near-field and far-field datasets. Although the accuracy level in detecting the coda clicks is promising, we note that misidentification of a single click may lead to misclassification of the coda type. We leave the improvement of this result to future work. Communication characterization: findings from the coda annotator The ability of the developed automatic detector and annotator to provide large datasets of coda was used to investigate the characteristics of coda signals towards the understanding of sperm whale communication. The results of the automatic annotation were demonstrated to explore the characteristics of the codas. In particular, the distribution of coda types, the delays between successive codas and the ICI variation for different coda types and between different focal and non-focal whale pairs. An analogy was made for the resemblance of coda exchange to a telecommunications session, where the synchronization between the focal whale and its interlocutor in terms of delays between successive codas was compared with the baud rate of the communication, and the synchronization in terms of ICI variation was attributed to capacity outage.

As humans, many animal species that communicate via vocalization show a wide range of accents and dialects driven by environmental and social factors. “Dialect or the speech of the people is capable of expressing whatever the people are,” said Sterling K Brown, an American actor. A Southerner in the USA, a Sicilian in Italy or a Bavarian in Germany would be easily recognizable as such, but also often stresses his or her dialect to signal where they come from geographically and culturally. … many animal species from multiple taxa have evolved forms of vocal communication that are subject to geographic, genetic, environmental, behavioural and social variations. This study analyzed whistle variation in two geographically separate groups of the Mediterranean common bottlenose dolphin ( Tursiops truncatus ), one around Sardinia and the other near Croatia on the opposite side of the Italian peninsula. The authors found that variations in frequency were determined mostly by acoustic characteristics of the surroundings, while variations in tempi were associated only with locality and group size. They concluded that longer and more complex tonal sounds enabled individual recognition and cohesion in social groups, while more basis variations such as frequency evolved for optimum communication in water. The cognitive capabilities of dolphins further allow them to modify their whistles to address individuals by name for enhanced collaboration. A recent study found that male dolphins can indeed learn the signature whistles of up to around 15 of their closest allies (King et al , 2021 ). This indicates that dolphins have a concept of team membership, which partly explains how they maintain tight-knit societies. It also indicates that dolphins can convey information or instructions, rather than just group identities, for which there is growing evidence for porpoises and whales as well. Deciphering animals’ languages This raises the next logical question: whether some of these species exhibit language in a strict sense: exchanging minimal units of information along with rules akin to grammar for putting them together. Project CETI, a five-year multidisciplinary project (Cetacean Translation Initiative, www.projectceti.org ), has been set up to answer this question with the help of machine learning and language processing. The project focuses on sperm whales as they communicate for up to 40 min on end in a series of complex clicks while engaging in a variety of tasks. The aim is to capture millions of whale codas and analyze them in the context of where the whales are and what they were doing at the time. Machine learning will be used to identify possible rules of the whales’ language, by making intelligent guesses at such rules and then see if the observed “conversations” confirm them or require adjustment. This approach has been used for analyzing animal vocalization for some years now, and there is growing evidence that it can categorize sounds more accurately than traditional statistical methods such as the one applied in a 2014 study on goats (Favaro et al , 2014 ). The objective was to assess how reliable an artificial neural network (ANN) was in classifying calls according to individual identity, group membership, and maturation in the domestic goat Capra hircus . As the authors note, analysis of animal vocal signals had traditionally been performed using subjective methods, such as signal classification by multiple listeners. This was limited in scale and unable to divine clear relationships between vocalizations and categories such as social groups or different locations. All this has become possible over the past few years through application of more advanced computational techniques, especially machine learning, that have been shown to give accurate indications of group membership, body size, sex, and even age, by sifting the sounds into clusters aligned with those variables. In that 2014 study, the algorithms performed better than those established statistical techniques and set the stage for further investigation of ANN application to animal vocalization, thus laying the foundation for CETI. Green hylia in particular sang at lower frequencies at higher elevations and under reduced canopy cover, perhaps because of this increased range in the lighter air or foliage. … many other animal groups reliant on aural communication, such as insects, frogs and cetaceans, are also forced to adapt to noisy human activities, especially underwater from shipping and in urban areas. The effects of anthropogenic noise on animal vocalization have also been well studied, particularly in the case of songbirds. However, many other animal groups reliant on aural communication, such as insects, frogs, and cetaceans, are also forced to adapt to noisy human activities, especially underwater from shipping and in urban areas. One study noted that many animals are known to be capable of changing their vocalizations almost immediately in response to noise, but that other evolutionary mechanisms of adaptation have been largely unexplored (Moseley et al , 2018 ). The authors hypothesized that selection for signals less masked by noise was also an important mechanism of adaptation to anthropogenic sounds over a longer period. There is still a lot more to be discovered about animal vocalization and dialect variation and not just for the curiosity of some ecologists. Studying the dialects and accents between different social groups could also help to better understand and eventually decipher the much more complex communication—if not language—between cetaceans for example. References Favaro L, Briefer EF, McElligott AG (2014) Artificial neural network approach for revealing individuality, group membership and age information in goat kid contact calls. Acta Acust United Acust 100: 782–789 Google Scholar King SL, Connor RC, Krützen M, Allen S (2021) Cooperation-based concept formation in male bottlenose dolphins.

ng desires is the enchanting notion that we might one day converse with other species. In the years since Gero’s insight, and partly because of it, the potential to bridge this communications gap has grown less fanciful. On Monday, a team of scientists announced that they have embarked on a five-year odyssey to build on Gero’s work with a cutting-edge research project to try to decipher what sperm whales are saying to one another. ( Why Killer Whales Go Through Menopause But Elephants Don’t) Such an attempt would have seemed folly even just a few years ago. But this effort won’t rely solely on Gero. The team includes experts in linguistics, robotics, machine learning, and camera engineering. They will lean heavily on advances in artificial intelligence, which can now translate one human language to another without help from a Rosetta Stone, or key. The quest, dubbed Project CETI (Cetacean Translation Initiative), is likely the largest interspecies communication effort in history. Already, these scientists have been at work building specialized video and audio recording devices. They aim to capture millions of whale codas and analyze them. The hope is to expose the underlying architecture of whale chatter: What units make up whale communication? Is there grammar, syntax, or anything analogous to words and sentences? These experts will track how whales behave when making, or hearing, clicks. And using breakthroughs in natural language processing—the branch of artificial intelligence that helps Alexa and Siri respond to voice commands—researchers will attempt to interpret this information. Listen in on sperm whales Sperm whale vocalizations, among the loudest animal sounds on the planet, have a Morse code-like structure that shares the hallmarks of a highly-evolved language. Audio Timeline Slider: Use the left or right arrow keys to decrease or increase the slider value. Play audio 15 15 Audio Unmuted. Mute Volume Slider: Use the left or right arrow keys to decrease or The team includes experts in linguistics, robotics, machine learning, and camera engineering. They will lean heavily on advances in artificial intelligence, which can now translate one human language to another without help from a Rosetta Stone, or key. The quest, dubbed Project CETI (Cetacean Translation Initiative), is likely the largest interspecies communication effort in history. Already, these scientists have been at work building specialized video and audio recording devices. They aim to capture millions of whale codas and analyze them. The hope is to expose the underlying architecture of whale chatter: What units make up whale communication? In 2017, while a fellow at Harvard University’s Radcliffe Institute, Gruber, a diver, became fascinated with sperm whales , the largest toothed whales, after reading a book about free divers who study them. One day while listening to whale codas on his laptop, another Radcliffe fellow, Shafi Goldwasser, happened by. “‘Those are really interesting—they sound like Morse code,’” Gruber recalls Goldwasser saying. She had been hosting lectures for a group of Radcliffe Fellows on machine learning, a subfield of artificial intelligence that employs algorithms to find and predict patterns in data. In addition to Gero and Gruber’s Radcliffe computer colleagues, there is whale biologist Roger Payne, a MacArthur Award winner, who had popularized the mesmerizing songs of humpbacks in the 1960s and 1970s, helping to ignite the “Save the Whales” movement. There is Robert Wood, a Harvard roboticist who, with Gruber, constructed the jellyfish handler and whose lab has built self-folding origami and an insect-sized flying drone. And there is Daniela Rus, another MacArthur recipient and director of Increasingly, animal communication discoveries are assisted by AI. Through machine learning, researchers in 2016 decoded call differences between Egyptian fruit bats squabbling over food and those fighting over resting spots. Rats and mice communicate far above the range of human hearing. By transforming those sounds into sonograms and running the images through artificial neural networks loosely inspired by human brain circuitry, scientists in 2019 linked different sounds to different behaviors, such as fleeing danger or trying to attract a mate. Researchers dubbed their algorithm “DeepSqueak .” Sperm whales spend only about 10 minutes of every hour near the surface, so researchers are building an array of audio and video recorders to capture what they say to each other deep beneath the surface. Artificial intelligence will then search for patterns in the chatter. These insights are now possible because breakthroughs in machine learning have come at a lightning clip in the last decade as algorithms get more sophisticated and computer processing power explodes. But humpbacks are usually just being aggressive. Artificial intelligence can weed out our biases and more accurately find meaning in communication and behavior, Fournet says. For the CETI researchers, much of the value will be in the journey of discovery itself. The Apollo mission put people on the moon, but along the way humans invented calculators, Velcro, and transistors, and they helped launch the digital age that makes this project possible. Even if CETI never cracks the sperm whale code, researchers are bound to make significant advancements in machine learning, animal communication, and our understanding of one of the world’s most mysterious creatures. It’s the idea that we want to know what they’re saying—that we care.” The National Geographic Society, committed to illuminating and protecting the wonder of our world, funded Explorers David Gruber, Shane Gero, and Robert Wood. Learn more about the Society’s support of ocean Explorers. Related Topics SPERM WHALE ANIMAL COMMUNICATION LANGUAGE ARTIFICIAL INTELLIGENCE You May Also Like ANIMALS Scientists filmed a whale birth—and found something amazing SCIENCE What are animals saying?