Artificial intelligence used to recognise primate behaviours in the wild

An adult chimpanzee in a clearing of the forest using a hammer stone and anvil to crack open a nut

Scientists at the University of Oxford have developed new artificial intelligence (AI) models to recognise behaviours of chimpanzees in the wild. The method will allow researchers and wildlife conservationists to significantly cut back on time and resources spent analysing animal behaviour in video footage, according to the new paper published in Science Advances: Automated audiovisual behavior recognition in wild primates. The international collaboration also featured scientists from other institutions in the UK (University of Exeter), Japan (Chubu Gakuin University, Japan Monkey Centre) and the US (University of Rochester, California Institute of Technology).

The new computer model was trained using videos from two populations of wild chimpanzees in West Africa from Bossou in Guinea, and Cantanhez National Park in Guinea-Bissau, to capture several behaviours: nut-cracking, eating, and buttress drumming. The tool is the first of its kind to automatically recognise behaviour in wild primates using both audio and video, and builds on previous work which developed an automatic method for tracking and identifying individuals using face recognition.

“Our models can be applied to thousands of hours of video recordings of chimpanzees in their natural habitat from camera traps or archival footage,” says Max Bain, researcher and DPhil student at Oxford University’s Department of Engineering. “We use methods from deep learning with networks that are able to ingest both the audio and the visual stream of information from a video, crucial in the wild where an animal behaviour might be heard but not seen (e.g. cracking nuts behind a tree) or inaudible actions”.

With the widespread deployment of camera trap databases and use of long-term video archives in field research, capturing the sheer volume of animal behaviours using human researchers is becoming increasingly difficult to scale. “For species such as chimpanzees which have remarkable behavioural complexity, cross-site comparisons of video datasets using AI presents an exciting opportunity to capture subtle variation between groups and the evolution of behaviour over time at a scale and depth not previously possible”, says Daniel Schofield, researcher and DPhil student at Oxford University's Primate Models Lab, Institute of Human Sciences.

Using artificial intelligence models to identify chimpanzee behaviours in the wild: chimpanzees performing various behaviours including nut-cracking and eating which are recognised and labelled

The method is novel by combining individual identification from previous approaches with audio-visual behaviour recognition to gain richer insight into the complexity of animals’ lives. “Our method is not restricted to chimpanzees, and can be trained to recognise any behaviour” says Bain. We hope that other teams and researchers can apply our cutting edge methods to other species”.

“This work will allow answering novel questions”, says Susana Carvalho, Head of the Primate Models Lab in Oxford and co-author of the study. “For example, how individuals develop their technological skills over a lifetime, are certain behaviours in decline in some populations or if there are drumming signatures that are stable in a group over decades, are all questions that are now starting to be within the realm of reality because of this ‘marriage’ between wild chimpanzee researchers (holding incredible treasures of chimpanzee behavioural knowledge) and the computational scientists that have been developing AI tools for a long time, mostly to apply to human settings/data but now are expanding to include non-human animals”.

Monitoring vast amounts of data from field research of wild species is a crucial component of conservation efforts tackling the biodiversity crisis. The use of behaviour recognition using AI has potential to capture novel behavioural indicators that can more accurately measure the viability of threatened populations. “Ultimately this can help us examine the social and ecological drivers of behaviour, as well as monitor how these communities are responding to environmental pressures caused by climate change and habitat degradation from human activities”, says Schofield. “As a computer scientist, it is extremely satisfying to see these methods applied to solve real, challenging biodiversity problems” adds Arsha Nagrani, a co-author who is now at Google Research.

You can learn more about the team's research in the Science Magazine news article by Rachel Fritts, 'Artificial intelligence helps scientists spy on chimp behavior in the wild', and through the below video shared by Science Advances.