Dairy farmers and vets enlisted the help of video gaming experts to enhance cow health in a unique new study.

The Innovate UK-funded SmARtview research project saw Agri-EPI Centre and UK Veterinary Group VetPartners join forces with experts at award-winning games developer Pocket Sized Hands and Abertay University’s School of Design & Informatics.

The project developed an Artificial Intelligence (AI)-based system that can recognise a cow by its skin patterns and use a ‘HoloLens’ Augmented Reality (AR) headset to display health and productivity data about an animal as a user views it.

Agri-EPI’s innovative South West Dairy Development Centre is the project test-bed.

Agri-EPI Head of Dairy Duncan Forbes explained:

“By using the type of technology that is becoming extremely advanced in the video gaming world, we hope to be able to take the care and treatment of dairy cattle into a new era.

The beauty of this project is that it combines farmers’ and vet’s experienced ‘eyes’ with real-time data: technology is being used to enhance, rather than replace, human skills. In addition, SmARtview could provide a solution to one of the key challenges of on-farm technology – while different devices like robotic milkers, sensors and collars can already collect plentiful data, its lack of integration limits insight to drive decision-making.”

Typically, a vet or farmer faces the time-consuming process of examining a cow then going to the farm office to review a variety of paper of PC-based records before determining what kind of intervention may be needed. SmARtview would make the whole process much more efficient and effective.

The new system will be trialled by vets and allied professionals from VetPartners’ practices around the UK. Rachel Dean, VetPartners’ Director of Clinical Research and Excellence in Practice said:

“The farming industry is changing and VetPartners wants to support farmers through this change by offering the best capabilities in terms of health, production and welfare. SmARtview has huge potential for vets and allied professionals, such as foot trimmers and veterinary technicians, who work on farms of all types and sizes. It is a unique and very innovative project which could transform the way they practice.”

Abertay is a world leader in applying games technology to the business, education and healthcare sectors and the University is ranked number one in Europe for videogames education.

Pocket Sized Hands and Abertay University’s School of Design and Informatics are bringing world-leading expertise to the project. Abertay’s Professor Ruth Falconer explained that the first challenge is to develop the AI technology capable of recognising an individual animal. She said:

“Our aim is to put new technology to best use in order to solve problems in the real world. This project brings together our research areas of AI, User Experience and games technology. We aim to develop ‘marker-less’ technology which can achieve the difficult task of recognising a cow by the patterning of its skin and shape in an environment where it is likely to be dirty and the light and weather conditions change frequently.”

When the researchers have cracked the challenge of cow-recognition, Pocket Sized Hands (PSH) will lead the prototype AR development and lend their expertise to the User Experience (UX) aspect of the technology. Their skills with ensure the augmented reality display of data is presented to the user in an engaging way.

PSH Chief Executive Gary McCartan explained:

“It’s really exciting to be applying technology we normally use in game development to the world of farming. AR combines the real world with digital assets – information that is usually tied to a screen can now be viewed in the real world. Use of the technology is in its early stages. For example, current headsets are quite bulky which would not be practical in a farm situation. We will be looking at ways to allow cow data to be easily accessed and understood.”

The initiation of this project was supported by video games cluster InGAME.