The FGVC Plant Pathology 2020 challenge dataset to classify foliar diseases ofapples

Ranjita ThapaPlant Pathology and Plant-Microbe Biology Section, Cornell University

Geneva, NY, 14456, USA

Noah SnavelyCornell Tech

2 W Loop Rd, New York, NY 10044, USA

Serge BelongieCornell Tech

2 W Loop Rd, New York, NY 10044, USA

Awais KhanPlant Pathology and Plant-Microbe Biology Section, Cornell University

Geneva, NY, 14456, USAawais.khan@cornell.edu

Abstract

Apple is one of the most economically important fruitcrops in temperate regions of the world. A large numberof diseases and pests pose a significant threat to apple pro-duction in the U.S and globally. Diseases and pests not onlyresult in tree loss but also affect cosmetic appearance of thefruit, leading to decrease in consumer appeal. Early andaccurate disease detection is critical for disease manage-ment in orchards. Traditional disease and pest detectionin apple orchards relies on human scouts, and is labor in-tensive and time consuming. Recently, the growth of com-puter vision has provided new opportunities for accurateidentification of disease infections in many other crops. Inthis study we collected 3,651 high-quality, real-life RGB im-ages with symptoms of multiple foliar diseases of apples,under variable illumination, angle, surface, and noise con-ditions. These images were expert-annotated to create apilot dataset for apple scab, cedar apple rust, and healthyleaves. This dataset was made available to the Kaggle com-munity for ‘Plant Pathology Challenge’ as part of Fine-Grained Visual Categorization (FGVC) workshop at CVPR2020 (Computer Vision and Pattern Recognition). The topperforming model so far reported an AUC (Area Under theROC Curve) value of 0.99. This pilot study shows that com-puter vision and machine learning could be promising toolsfor disease detection and classification in apples.

1. Introduction

Apple (Malus x domestica) is one of the most impor-tant temperate fruit crops in term of its value (in the US,$15 billion annually) and production. Producing apples iscostly, due to the high cost of land, labor and orchard man-agement, and high risk, due to reliance on a few cultivars,which increases the risk of pests and diseases. Many bac-terial, fungal and viral pathogens and insects pose a con-stant threat to apple orchards in the growing season. Pre-cise and early identification of diseases and pests is neededfor timely management to prevent crop loss. The mostcommon destructive diseases of apples are fire blight, ap-ple scab, powdery mildew, cedar apple rust, Alternaria leafspot, frogeye leaf spot, sooty blotch, flyspeck, and summerrot [8, 11] common insects are mites, aphids, codling moth,brown marmorated stink bug, and European Sawfly (Fig-ure 1). Severe leaf infections cause chlorotic and necroticlesions and can result in leaf fall and even death of the tree.The damage caused by fruit disease infection reduces theircosmetic appearance and quality, ultimately leading to lowmarketability and great economic losses [11]. Disease de-tection is most commonly done by manual scouting. Dur-ing the growing season, human scouts visually inspect treesfor insects, pests, and diseases at 1-2 week intervals. Dueto large orchard sizes, high number of clients and a highcost/time analysis, scouts usually make random samplingswithin an orchard block and look for specific susceptiblecultivars, or specific regions of the orchard (e.g., edges etc.).

1

Figure 1. Sample images from the dataset showing symptoms of a) cedar apple rust, b) apple scab, and effect of differences in lighting,capturing method, age of the leaf and disease symptoms as well as host resistance response.

Figure 2. Images of disease symptoms on apple leaves capturedunder different light conditions (a) Indirect sunlight on leaf, (b)Direct sunlight on leaf, and (c) Strong reflection on leaf.

Unfortunately, few experienced scouts are available, so theratio of number of orchards to scouts is high. Also, therandom sampling strategy undertaken by scouts in a largeorchard is prone to error [6, 7, 4].

1.1. Problem context

Many disease symptoms in apples have similar appear-ances, which leads to complexities and difficulty in identifi-cation of the actual causal pathogen [2]. At the same time,the visual symptoms of a single disease can vary acrossvarieties due to their varying morphology e.g. leaf color,leaf shape, leaf pubescence etc. Factors like temperature,

humidity, and the physiological developmental stage of aplant also play crucial role on disease infection and insectdevelopment [3, 12]. The shape and form of symptomsalso change over time as the disease progresses and leafor fruit tissue ages. In recent years, computer vision andmachine learning have been extensively used as promisingtools across fields of plant study. Digital imaging and ma-chine learning can speed up plant disease diagnosis [10],but when using computer vision as a tool for precise dis-ease identification, all variables must be accounted for inthe digital image database [1]. For example, image captureconditions must include multiple positions and angles of in-fected tissue in the trees, many shades of lighting, capturesensor types, season and weather results, in addition to eachof the different diseases on fruits and leaves of various ages(Figure 2).

2. Apple foliar disease symptoms

Apple scab, caused by the fungal pathogen called ‘Ven-turia inaequalis’, is one of the most harmful fungal diseasesof apples in temperate regions of world [9]. The typicalsymptoms of apple scab are visible fungal structures on theleaf and fruit surface. The initial infection appears as blackor olive-brown lesions bulging on the upper surface of leaf;later stages show chlorotic sporulating lesions on infectedleaves. Apple scab affects not only leaves, but fruits as well,

2

and can cause premature fruit and leaf fall and deformationof fruits. Infected fruits show dark colored, sharply bor-dered, brown, and corky lesions. Cedar apple rust is causedby a Basidiomycotina fungus called Gymnosporangium ya-madai miyabe. Early symptoms of the disease are small,light yellow spots on leaves. Later, these spots expand andturn bright orange. The infected leaves get swollen, en-larged, and curled at the edges and in severe cases, pre-mature dropping of leaves occurs. Severe outbreaks of rustpathogen for two to three years can severely injure or kill thetrees of susceptible apple varieties [5]. In natural scenario,the most common condition is the appearance of multiplediseases in same plants which make it difficult to visuallyidentify the causal pathogens.

3. Apple foliar disease dataset

In this study, we used a large, high-quality, real-life dis-ease pilot dataset of multiple apple foliar diseases capturedduring the 2019 growing season from commercially growncultivars in an unsprayed apple orchard at Cornell AgriTech,Geneva, NY. Photos were taken using a Canon Rebel T5iDSLR and smartphones under various conditions. A ma-jority of the pictures are of apple scab, cedar apple rust,Alternaria leaf blotch, frogeye leaf spot, and healthy leaves.We have manually confirmed diseases and annotated imagesof apple scab, cedar apple rust, and healthy leaves. The fi-nal dataset contains a total of 3,651 RGB images of leaveswith 1,200 apple scab, 1,399 cedar apple rust, 187 complexdisease, and 865 healthy leaves, respectively. The leaveswith complex disease patterns comprised more than one dis-ease in the same leaf. To reflect real-world scenarios, theimages were acquired directly from apple orchards duringthe growing season under varying light/angle/surface/noiseconditions: (1) Imbalanced dataset of disease categories, (2)Different background of images, (3) Time of the day whenthe images were taken, (4) Different physiological age ofthe plants, (5) Co-occurrence of multiple diseases on thesame plant, and (6) Different focus of the images. Thedataset was randomly split into training and stratified testset of 80% and 20%, respectively, or 2,921 training imagesand 723 test images. This dataset has been made public forautomatic image-based disease classification and quantifi-cation to train and test computer vision models.

4. Results

The expert-annotated pilot dataset for apple scab, cedarapple rust, and healthy leaves was made available to theKaggle community for ‘Plant Pathology Challenge’ com-petition as a part of Fine-Grained Visual Categorization(FGVC) workshop at CVPR 2020 (Computer Vision andPattern Recognition). The competition was launched atKaggle on March 9, 2020 and was open until May 26, 2020.

Figure 3. Frequency distribution of entries with top auc (areaunder the ROC curve) values from each team.

Mean AUC (area under the ROC curve) values will be usedto select the three winners with top models. A total of 1,345teams have participated in the competition and have sub-mitted approximately 5,796 entries. Approximately 41% ofteams had top entries below 0.95 AUC, 28% teams had en-tries between 0.95 and 0.97 AUC, and 31% of teams had topentries above 0.97 AUC. More than 20 contending teams re-ported an AUC value greater than 0.985 (Figure 3).

5. ConclusionThis dataset will contribute towards pushing the state of

the art in automatic image classification for identificationand quantification of apple diseases from a large numberof symptom classes in various real scenarios. We will con-tinue adding more images captured at a diverse range of an-gles, lighting, and distances to our pilot dataset to build aneven larger, more comprehensive expert-annotated dataset.This will include manually capturing and annotating imagesof symptoms on apple leaves representing apple scab, fireblight, powdery mildew, cedar apple rust, alternaria leafblotch, frogeye leaf spot, and marssonina leaf blotch, aswell as insects, including apple aphids and mites, on leaves.We will also capture and annotate images of fruits with ap-ple scab, bitter rot, and brown rot.

6. AcknowledgmentsWe acknowledge financial support from Cornell Initia-

tive for Digital Agriculture (CIDA) and special thanks toZach Guillian for help with data collection.

References[1] Jayme GA Barbedo. Factors influencing the use of deep

learning for plant disease recognition. Biosystems engineer-ing, 172:84–91, 2018.

[2] Jayme Garcia Arnal Barbedo. An automatic method to de-tect and measure leaf disease symptoms using digital imageprocessing. Plant Disease, 98(12):1709–1716, 2014.

[3] Pengbo Dai, Xiaofei Liang, Yajing Wang, Mark L Gleason,Rong Zhang, and Guangyu Sun. High humidity and age-

3

dependent fruit susceptibility promote development of tri-chothecium black spot on apple. Plant disease, 103(2):259–267, 2019.

[4] Annie Deutsch and Christelle Guedot. Apple Maggot.University of Wisconsin–Extension, Cooperative Extension,2018.

[5] Lewis Ralph Jones and Elbert Thomas Bartholomew. Applerust and its control in Wisconsin, volume 253. AgriculturalExperiment Station of the University of Wisconsin, 1915.

[6] Gary JR Judd, Alan L Knight, and Ashraf M El-Sayed. De-velopment of kairomone-based lures and traps targeting spi-lonota ocellana (lepidoptera: Tortricidae) in apple orchardstreated with sex pheromones. The Canadian Entomologist,149(5):662–676, 2017.

[7] Xiaolong Li, Shubao Geng, Hanjie Chen, Chuleui Jung,Chunliang Wang, Hongtao Tu, and Jinyong Zhang. Masstrapping of apple leafminer, phyllonorycter ringoniella withsex pheromone traps in apple orchards. Journal of Asia-Pacific Entomology, 20(1):43–46, 2017.

[8] William E MacHardy. Current status of ipm in apple or-chards. Crop Protection, 19(8-10):801–806, 2000.

[9] William E MacHardy, David M Gadoury, and CesareGessler. Parasitic and biological fitness of venturia inae-qualis: relationship to disease management strategies. Plantdisease, 85(10):1036–1051, 2001.

[10] Anne-Katrin Mahlein. Plant disease detection by imagingsensors–parallels and specific demands for precision agricul-ture and plant phenotyping. Plant disease, 100(2):241–251,2016.

[11] Turner B Sutton, Herb S Aldwinckle, Arthur M Agnello, andJames F Walgenbach. Compendium of apple and pear dis-eases and pests. Am Phytopath Society, 2014.

[12] Thomas Wohner and Ofere Francis Emeriewen. Appleblotch disease (marssonina coronaria (ellis & davis) davis)–review and research prospects. European journal of plantpathology, 153(3):657–669, 2019.

4