Colour Imaging Pipeline for Orchard Aerial Surveys

Outfield Technologies is an agri-tech startup based in Cambridge that is developing applications for drone and imagery recognition systems in managing fruit crops.

 

Outfield Technologies is building scalable crop management systems that will allow growers to meet challenges such as rising labour costs and reduced access to pesticides. Outfield currently generates revenue through mapping orchard variability and by providing other drone surveying services for the wider agriculture industry.

 

The Challenge

 

Commercial apple production is worth £150 million per year in the UK and £1.5 billion in the EU, but predicting the value of a single orchard’s harvest is a tough challenge. Growers currently have no reliable method for estimating the quality or amount of fruit being produced until the fruit is picked from the tree and sent for packing. This lack of reliable forecasting data leads to inefficiencies and wastage throughout the growing season, as growers struggle to predict their requirements for labour, logistics, storage, and to track their performance against production contracts.

Outfield is trying to address this challenge by using drones and imagery recognition technologies to provide growers with accurate data on their orchards.

But volume of the crop is only one predictor of harvest value – the other is fruit quality. Tracking the colour of fruit in the orchard allows growers to assess if the harvested fruit will satisfy the strict requirements from retailers for high quality and therefore high value fruit.

Tracking colour also allows the grower to monitor the crop’s development, produce better predictions for harvest time and detect disease. Mapping the variation of fruit colouration across the orchard allows growers and agronomists to identify key underlying trends in the crop. By tracking these metrics during the season, the grower can make interventions in the orchard and reduce the number of rejected apples during and after harvest. This reduces food waste and improves farm efficiency.

The objective of this EIRA R&D project was to develop an imaging solution for assessing the colour of fruit in a commercial orchard from a drone survey, in order to predict the quality of the orchard yield. The project investigated the whole colour imaging pipeline for drone surveys of orchard crops. The project team developed methods for calibrating individual cameras as well as discounting the influence of varying illumination on the colour appearance of the fruit. They also measured the true colour appearance of relevant fruits of different grades in order to relate them to the colorimetric imaging pipeline being developed. The resulting solution will improve Outfield’s current surveying systems and could lead to developing future crop management services.

The outcomes from this project will allow Outfield to offer fruit quality assessment as a feature to apple grower customers worldwide. This will increase the competitiveness of the Outfield system and its utility to growers.

 

The Approach

 

The project made use of University of East Anglia’s specialist facilities at the Colour and Imaging Lab. This Colour and Imaging Lab is at the leading edge of colour research and is active in a range of different research areas such as computational colour constancy, physics-based vision, image reproduction, colour graphics, image indexing, colour scanning, colour science, and computer vision.

Dr Michal Mackiewicz from UEA was the lead academic working on this project. He received his MSc in Telecommunications at the University of Science and Technology (AGH) in Krakow in 2003. That year he joined the School of Computing Sciences (CMP) at the University of East Anglia and began a PhD titled “Wireless capsule endoscopy video analysis”. He has worked as a research assistant on a research project sponsored by Buhler Sortex and later on the EPSRC grant titled “Illuminating Colour Constancy: From Physics to Photography”. In 2014, he became an Independent Research Fellow and in 2016 a Lecturer. Since 2020, he has been an Associate Professor in Computer Science at CMP, UEA. Michal has been involved in researching areas of colour science, physics-based vision, remote sensing, medical imaging and machine learning.

For the R&D project with Outfield Technologies, the academic team sought to deliver 3 distinct work packages:

  • Camera colour calibration – RAW and JPEG images of relevant colour targets were captured using Outfield drones in the controlled lab environment at the Colour & Imaging Lab, UEA. The obtained colour calibration was compared to the in-built calibration and found to be significantly more accurate. The calibration method developed allows for accurate colour calibration of Outfield images both in the lab using Colour & Imaging lab facilities as well as importantly, in the field by the drone operator.
  • White balance (WB) algorithm development – here we measured the errors of the drone camera built-in AutoWB algorithm. These errors were found to be significant and seriously affecting the colour of apples. Consequently, we have developed a bespoke algorithm aimed at orchard survey images. The algorithm exploits the physical constraints of the orchard environments and reduces typical illuminant estimate errors by a factor of eight.
  • Apple colour measurement – in this work package, the project team measured the colour characteristics of a number of apple varieties in a lab setting using previously calibrated drone cameras.

 

The Outcome

 

Project results have driven improvements to Outfield’s  surveying processes. Outfield now have a better understanding of camera colour processing and several areas of follow-on research have been identified by the project. Moving forward, Outfield plan to target the commercial launch of fruit colour assessment for southern hemisphere growers this (2021) season.

Jim McDougall, Founder and Commercial Director of Outfield Technologies said:

“It’s a real privilege to be working with the team at UEA on this project. Outfield has developed a world leading system to gather and analyse fruit level data from orchards, and now there is so much potential to provide growers and the industry with further insights from this data. We work with growers every day, now on 4 continents, so we know the importance of understanding fruit quality. This project plans to take a big step forward in providing growers with colour assessments for their crops, forecasting quality and allowing for improved crop management. This project aims to push the limits of technology to provide very valuable data for growers internationally, and potentially can also reduce waste and improve efficiency in orchards to make feeding the world more sustainable.”

Oli Hilbourne, Founder and Operations Director of Outfield Technologies said:

“Through this EIRA project, Outfield has been able to access the amazing expertise and facilities at UEA’s Colour and Imaging Lab. It’s been fantastic for us to have that support in developing our technology and pushing the bounds of precision farming. Thanks to everyone at EIRA who helped make this project possible.”

You can find out more about this project by watching Dr Michal Mackiewicz presentation in our Innovation Month AI Webinar here:

Oli Hilbourne from Outfield Technologies also leant his expertise to our virtual EIRA Showcase, where he delivered a presentation on the project here: