Washington Tree Fruit Research Commission

Research Reports

Real-time detection of apples infested with codling moth using x-ray (2005)

FINAL PROJECT REPORT
WTFRC Project #PH-03-895
YEAR 0/0
Organization Project #58-5325-4-478
Title:Real-time detection of apples infested with codling moth using x-ray
PI:Ron Haff
Organization:USDA-ARS-WRRC 800 Buchanan St. Albany, CA 94710 (510)559-5868 ron@pw.usda.gov
 PDF version of report

Co-PIs

Jim Hansen USDA-ARS-YARL

 5230 Konnowac Pass Road

Wapato, WA 98951 USA

Phone: 509-454-6573 E-mail: jimbob@yarl.ars.usda.gov

Objectives

The principle objective was to demonstrate the potential of x-ray technology to identify apples that have been infested with coddling moth and the feasibility of installing x-ray equipment on the apple processing line for the identification and removal of infested fruit. A secondary objective was to also identify apples with other defects such as watercore and core rot.

Specific objectives to complete the project were as follows:

  1. Establish codling moth colonies at WRRC.
  2. Obtained apples and infest them with coddling moth larvae.
  3. Obtain digital x-ray images of the apples on a daily basis.
  4. Input x-ray images into a scrolling program on a PC that simulates the apple processing line, and test the ability of human operators to identify the infested fruit.
  5. Develop and test a computer algorithm to automatically identify the infested fruit.
  6. Modify the algorithm to identify core rot and watercore.
  7. Evaluate the feasibility and cost of installing the necessary x-ray equipment on an apple processing line for the removal of defective fruit.

     

Significant findings

1.  Digital x-ray images of apples containing codling moth show evidence of infestation six
      to ten days after infestation.
2.  Humans (so far) are much better at identifying infestations than computer algorithms for
     stationary and slowly moving images, but accuracy decreases rapidly with speed and
     human inspection of x-ray images is unfeasible at production line speeds.
3.  A technique has been developed to alleviate the problem of the edges of the x-ray images
     being washed out (a consequence of the shape of the apple).
4.  Reliable detection (>90%) of infestation with present algorithms occurs only on the
      twelfth day after infestation. Even if algorithms are improved, apples would require two
      inspections spaced approximately ten days apart for a commercial sorting system to be
      effective.
5.  Image capture, algorithm application, and rejection of detected infestations can currently 
    be accomplished at a  rate of between ten and twenty apples per second per channel.

Methods

Codling moth colonies have been established at WRRC with the help of personnel and material from USDA-ARS-YARL. 100 each of Red Delicious and Golden delicious apples were infested with codling moth “neonates” (small worms recently hatched from eggs). Three neonates were applied to each apple, as previous experiments indicated that the survival rate was less than 50%. The apples were placed in individual containers and held at room temperature. In addition to the 100 infested apples for each cultivar, the data set included 100 non-infested apples as a control set.

 

X-ray images of the apples were generated on two separate x-ray systems. The first was an x-ray image intensifier based system as depicted in Fig. 1. The x-ray source was an OEG50 high current source (up to 99 mA) with beryllium window, and the image intensifier had a nine inch diameter also with beryllium window. Apples were loaded manually for inspection one at a time, but the components could be incorporated into an automatic inspection system, as the time required for image capture is on the order of a few milliseconds. This system has the advantage of extremely high resolution (~50 micron) and low noise compared to other commercially available systems. The disadvantage of this system is that the components would add substantially to the cost of an automated system compared to a linescan unit as described below.

 

 

Fig. 1. Image intensifier based x-ray imaging system.

 

The second system used to generate images of the samples was a ScanTrac x-ray inspection system (Peco Controls InspX, Freemont, CA, model 200 S12). This system is a linescan unit designed for the inspection of packaged foods. The general principle of a linescan x-ray unit is shown in fig. 2. The x-ray beam is collimated onto a plane over a single row of detectors. Movement of the sample across the detector line allows the image to be constructed one row at a time. Fig. 3 shows the actual unit, which is designed to fit over existing conveyor lines, allowing implementation with minimal disruption to the existing processing line. Unlike the schematic of fig. 1, this unit is a side view system so that the conveyor system itself does not interfere with the imaging process. This system is capable of inspecting between 10 and 20 samples per second, depending on the size of the sample and complexity of the recognition algorithm. This throughput rate includes application of a recognition algorithm and rejection of undesirable product. The detectors have a resolution of approximately 0.8 mm. The system is also available with 0.4 mm detectors which give higher resolution at the cost of lower product throughput.

 

Fig. 2. General principle of a linescan x-ray system. The x-rays are collimated into a fan shaped beam over a row of detectors. Movement of the sample across the beam allows row by row construction of the image.

 

 

Fig. 3.  ScanTrac x-ray inspection system with side view orientation, capable of inspecting 10 – 20 samples per second.

 

For the results reported here, samples were imaged over a period off 22 days from the time of infestation. For the intensifier based system, samples were x-rayed on days 0, 3, 7, 10, 14, 17, and 21. For the linescan system, x-rays were taken on days 1, 6, 9, 13, 16, 20, and 22. All images were saved as .bmp files to be used in human and computer algorithm recognition tests.

 

Images were imported into a scrolling program to simulate humans inspecting images on a processing line. Results were tallied by age of infestation and scrolling speed. These tests were conducted only for the intensifier images, since the results suggested that repeating the experiment with the linescan images was not warranted. These results will be discussed further in the results section.

 

Two computer algorithms have been developed for automatic recognition of the infested apples. The first uses discriminant analysis based on features extracted from the images. The need for rapid processing limits the number and type of features that can be used. A routine has been developed to select the most effective set of three features from a much larger set of potential features. The features tested included gradients in pixel values, absolute pixel values over a region, and various statistics involving linear regions within the image. The process of selecting the most effective features from this larger set of potential features is outlined in fig. 4.

 

 

Fig. 4. Process for selecting a small set of the most effective features from a larger set for use in discriminant analysis.

 

The second algorithm is based on developing an image of an ideal apple, and classifying samples based on their similarity to the ideal image. The similarity is determined by performing a correlation calculation between the two images. The major motivation for using this technique is that, in theory, it would be effective for any type of disorder, i.e. watercore, core rot, bitter pit, etc. that produces an x-ray image that is different in some way from the ideal image. The difficulty lies in deriving an ideal image. So far, the approach has been to use a large number of “ideal” images from the control set, perform the correlation calculations with a set of images with known defects, and select the one that yields the best results.

 

Technical problems with extracting images from the linescan machine into a standard format have prevented testing the algorithms on those images or importing them into this report. These problems have been overcome (fig. 6), and a new set of images is being produced so that the algorithms can be tested and the results compared to those from the intensifier system. These results are not ready to be reported here, as the apples being used are now in their tenth day after infestation. These results will, however, be ready for presentation at the meeting of the WTFRC in July.

Results and discussion

Typical images from the intensifier system are shown in fig. 5. In this case, the first signs of infestation are visible after four days. In general, the majority of apples show signs after 6 to 8 days. After 20 days the apples are generally too rotten to continue x-rays. Fig. 6 shows an apple x-rayed on the linescan unit. Clearly, as expected, the lower resolution linescan image is of a lesser quality than those from the intensifier system. However, infestations beyond the tenth day can generally be seen on the linescan monitor. Now that image conversion problems have been solved, we will be able to make comparisons with recognition results from the intensifier system.

 

 

Fig. 5. X-ray images from the intensifier based x-ray system at various stages of infestation.

 

 

Fig. 6. Linescan image of an uninfested apple. Dark streaks across the image are artifacts from the detector system, and not features of the apple.

 

Results from the scrolling experiment are shown in table 1. For still images, 50% recognition was achieved at the tenth day of infestation for both cultivars. The actual presence of infestation (ground truth) when a moth emerged from the apple at the end of the trial. Even at speeds of 40 apples per minute (less than one per second) recognition results dropped significantly. Above a speed of three apples per second humans can not keep up with the scrolling images. The results clearly indicate that having people monitor the images on computer screens is not a viable option, and automatic detection is required. Human inspection of the still images is useful, however, as it gives an indication of when information is present in the images for an algorithm to detect.

 

Table 1 Days of infestation required for 90% recognition of infested apples scrolling on a computer screen vs. scrolling speed.
Speed   apples / min
Days of infestation for 90% recognition

 

Golden Delicious
Red Delicious
0
12
12
40
15
18
80
20
>21
160
> 21
-

 

Results of applying the two algorithms to the images from the intensifier system are shown in table 2. For both algorithms, decision boundaries can be adjusted to increase or decrease the sensitivity. Increasing the sensitivity yields higher recognition percentages, but also increases the amount of non-infested fruit classified as infested. For the results reported here, the sensitivities were adjusted so that false positives were kept below 10%.

 

 

                             Table 2. Recognition results for the two automatic recognition algorithms
                                with false positive (FP) results maintained at 10%.

 

Algorithm 1
Algorithm 2
% rec
% rec
Days after infestation
RD
GD
RD
GD
0 - 5
10
8
14
12
5 - 10
31
27
36
33
10 - 15
77
68
79
68
15 - 20
92
85
91
87
> 20
98
96
96
95

 

 

The results make two very important facts clear. First, present x-ray technology does not allow reliable identification of infestations less than 10 days old. This problem can be circumvented by x-raying the product twice with an appropriate (yet to be determined) time interval in between. The feasibility of doing this has to do with whether or not this can be done without undue disruption to the processing line. Second, it is clear that better recognition algorithms need to be developed. Towards this end, our lab is developing collaboration with a professor at the University of Hawaii with expertise in this area far beyond our own. We are also working on collaboration with another ARS lab that is involved in apple inspection with color sorting technology. They have done considerable work involving apple orientation on the processing line and identification of exterior defects such as bruising and corking that could be incorporated into an inspection unit. This project has been incorporated as a major objective in the most recent five year plan for our research unit, and so although the grant with WTFRC expires this year the research will continue. There are still some significant difficulties to overcome, but we are hopeful to have an operational system by the end of that five year period. Since contact with the apple industry is obviously a critical issue for this project, we would be happy to continue to make annual reports to the commission given the opportunity to do so.

 

Recent development
A new technique has been developed to improve the quality of x-ray images of many agricultural commodities, including apples. Because apples are generally spherical in nature, producing enough x-ray energy to penetrate the thickest parts of the sample tends to wash away the edges. This is true of many agricultural products, and also product stored in cans or jars. Many software corrections have been used in the past to compensate for this phenomenon, but software corrections can not recover information that is not present in the x-ray image. We have developed a technique to alleviate this problem. Figure 7 shows an apple x-rayed with and without this technique. Tests to determine to what extent this technique may improve the ability to detect internal defects are under way.

 

 

 

 

 

 

 

Fig. 7.  Apple x-rayed with and without technique to alleviate problem of washing away of edges.
                      

WA Tree Fruit Research Commission , 1719 Springwater Ave, Wenatchee WA 98801, 509-665-8271, Contact Us

The WTFRC Research Reports database is hosted on the servers of WSU-Tree Fruit Research & Extension Center