Patent for “Hyperspectral imaging system to monitor agricultural products during processing or manufacturing”

Industrial Products – Henry M. Dante, Samuel Timothy Henry, Seetharama C. Deevi, Altria Client Services LLC

Search Patent for “Hyperspectral imaging system to monitor agricultural products during processing or manufacturing”

Abstract for “Hyperspectral imaging system to monitor agricultural products during processing or manufacturing”

The method is for monitoring the manufacturing process of an agricultural product. This method uses hyperspectral imaging. It involves scanning at least one area along an agricultural product using at most one light source with different wavelengths. Then, generate hyperspectral pictures from that at least 1 region. Next, determine a spectral signature for the agricultural product by comparing it to a database of spectral fingerprints.

Background for “Hyperspectral imaging system to monitor agricultural products during processing or manufacturing”

“The disclosed method and system is for optimizing and monitoring the manufacturing process of agricultural products including tobacco via hyperspectral imaging. The disclosed method and system provide high sensitivity, high quality, high resolution and fast operation in a simple, yet highly effective, cost-effective, commercially viable manner.

“In one aspect, disclosed is a method to monitor the manufacturing of an agriculture product. The method uses hyperspectral imaging. It involves: scanning at minimum one region of an agricultural product using at most one light source with different wavelengths; creating hyperspectral pictures from that at least 1 region; determining the spectral fingerprint of the agricultural product; comparing the spectral fingerprint to a database of spectral fingerprints obtained at various points in the manufacturing process using a computer processor to determine where the sample is at.

“In one form, a method to monitor a manufacturing plant is provided. It works by obtaining hyperspectral signatures of the agricultural material being processed. This reduces or eliminates the need for human evaluation. First, a standard database must be created. It should include hyperspectral signatures from each stage of a process such as the tobacco aging and fermentation processes. This database is used to monitor the progress of each stage of the process against a reference. The process can be adjusted in real time by monitoring the processing parameters. This will ensure that the product is of high quality.

“From the hyperspectral images, one form is a spectral fingerprint. The spectral fingerprint can be correlated with desirable sensory attributes. It is possible to determine a wide range of biological, chemical, and physical properties. Some forms may be automated online using hyperspectral imaging or analysis.

“Another form of the method involves scanning multiple areas along the sample of agricultural products using at least one light source with different wavelengths and then generating hyperspectral pictures from these multiple regions.”

“In another form, this method also includes the determination of a physicochemical number for the sample.”

“In yet another form, this method includes the object to manufacture an agricultural product with desirable sensory characteristics. A further form of the method involves determining the spectral fingerprint which corresponds with the desired sensory attributes.

“In an even further form, the agricultural product of tobacco is described as a fermentation process and the manufacturing process for it is called a fermentation process. The method is used to determine the time it takes for the fermentation process to be completed on the sample of tobacco.

“In one form, the product is tobacco. The manufacturing process is called a tobacco aging. Another method is used to determine the amount of time it takes to finish the tobacco aging process.

“In another form, the method involves correlating one or more of the spectral fingerprints of the sample of an agricultural product with the desired sensory attributes.”

“In yet another form, at least one source of light is placed to minimize the angle in which each beam of light hits the sample. The source must include a light source from the following group: a tungsten source, mercury source, xenon source, or a combination thereof.

“In a further way, manufacturing cost is used by the computer processor.”

“In an even further form, the method also includes storing data concerning the spectral fingerprints for the plurality sample of agricultural products within a computer storage device; and storing at most a portion of the plurality sample of agricultural product.”

“In another aspect, there is a system to monitor the production of an agricultural product.”

“In another aspect, there is a method to determine the stage of processing of an agricultural product. This method uses hyperspectral imaging. It involves: (a) scanning multiple areas along a desired agricultural product using at minimum one light source of different wavelengths;(b) creating hyperspectral photos from the multiple regions; (c), forming a spectrum fingerprint for the sample using the hyperspectral pictures; and (d). Correlating the spectral footprint obtained in step (c), to a database of spectral signatures

“In one form, this method includes: (e) storing data concerning the spectral fingerprint in a computer storage device; and (f) repeating steps a), (b),(c), and d) using multiple samples.”

“In yet another aspect, there is a system that determines the processing stage for an agricultural product.”

“A further aspect is that there is a method for determining the stage in processing of a product. This method involves: determining whether a sample meets the desired attribute and, if so applying hyperspectral analysis and theoretic analyses to establish a relationship between P and said unique spectrum P. Creating a hyperspectral image analysis of the sample to characterize it according to said elements (at minimum x, y); and then mathematically solving from said characterizations whether the sample attains the values of said elements of P.

“A further aspect is that the invention provides a method of controlling a manufacturing process to produce an agricultural product. This method uses hyperspectral imaging. It involves obtaining a sample from an agricultural product that is going through a manufacturing process. The manufacturing process is conducted at one or multiple predetermined parameters. Hyperspectral imaging is used to generate hyperspectral pictures from at least one of the regions. A spectral fingerprint database with a plurality fingerprints from various points in the manufacturing process is then created. Finally, adjustments are made to at least one process parameter to optimize manufacturing.

“A method for creating a database to control a manufacturing process that produces an agricultural product is provided in a further aspect. This method uses hyperspectral imaging. It involves the following steps: (a) creating a dark image and an image reference for calibration;(b) analysing the reference image to determine calibration coefficients; (3c) obtaining hyperspectral images for an agricultural sample; (3d) removing dark values; (e) applying calibration codes to compensate for fluctuations in system operation conditions; (f). Repetition steps (c-(e); and (g) storing all hyperspectral samples hypercubes to create the database.

“In one form, the computer data is stored on a computer-readable medium.”

“Certain forms are executed by steps or procedures. Sub-steps or subprocedures can be performed in any of the following ways: manually, semi-automatically or fully automatic. These include operation of system units and system sub-units. Devices, assemblies, mechanisms, structures and components. And peripheral equipment, utilities. Accessories. Furthermore, depending on actual steps or procedures sub-steps and subprocedures, device units, system subunits and devices, devices assemblies, mechanism, components and elements, as well as peripheral equipment, utilities and accessories, are used to implement a form. The steps or procedures and sub-steps and subprocedures can be performed using hardware, hardware, or an integrated combination thereof.

Software used to implement certain forms can be operatively interfaced and integrated into an operating system. This includes software programs, routines, subroutines and software symbolic languages. It can also include software code, software codes, instructions or protocols, and software algorithms. Hardware used to implement certain forms can be operatively interfaced and integrated, connected, as well as functioning electrical, electromechanical and/or electronic system units, devices, assemblies and sub-assemblies. It may also include peripheral equipment, utilities and accessories. This could include one or more computers chips, integrated circuits or electronic circuits. You can implement certain forms by using a combination of the exemplary software and hardware.

“In some forms disclosed herein, steps and procedures can be performed by a processor such as a computing platform for execution of a plurality instructions. Optionally, the processor may include volatile memory to store instructions and/or information and/or non-volatile storage such as a magnetic hard-disk or removable media for storing data and instructions. Certain forms herein may include a network connection. Some forms herein may include a display device as well as a user input device such a keyboard, mouse, and touch screen.

“The following sections will describe various aspects, with specific examples. Those skilled in the art will appreciate that the scope and spirit of the apparatus, method and methods described herein is not limited to these forms. It is important to note that the figures in this document are not drawn at a particular scale or proportion. Many variations can be made from the illustrations. FIGS. FIGS. 1-8 are where like numerals can be used throughout to denote like elements.

“Each one of the following terms is written in singular grammatical forms:?a? ?an,? ?an,? As used herein, the terms?the’ and?the’ may also be used to refer to or encompass a plurality or entities or objects, unless explicitly stated or indicated herein or unless the context clearly indicates otherwise. The phrases “a device”,? and “an assembly” are examples. ?an assembly,? ?a mechanism,? ?a component,? ?A component? As used herein, it may also refer and include a plurality or devices, a multitude of assemblies, multiple mechanisms, a plurality if components, and a plurality if elements.

“Each one of the following terms: ?including,? ?has,? ?having,? ?comprises,? ?comprises,? and their linguistic or grammaral variants, derivatives and/or conjugates as used herein means?includes but not limited to.

“It is important to understand that the various forms described herein do not limit their application to details of the sequence or number of steps or procedures and sub-steps or other sub-procedures of operation or implementation forms. They also include details of type composition, construction, arrangement, number of system sub-units and devices, assemblies, subassemblies, mechanisms and structures. As those skilled in the art will appreciate, the apparatus, methods, and systems disclosed herein may be used in other forms or in other ways.

“It should also be understood that technical and scientific terms, phrases, and/or words used in this disclosure have the same or similar meanings as those commonly understood by an ordinary skilled person in the art, except where otherwise stated or defined herein. The present disclosure uses phrasing, terminology, and notation for description purposes only and should not be considered as limiting.

“Moreover, all technical terms and phrases, including scientific ones, that are introduced, described, and/or illustrated in the preceding sections, are equally applicable or similar in the illustrative descriptions, examples, and attached claims.”

“Steps and procedures, sub-steps and sub-procedures, equipment and materials system units, device assemblies, subassemblies mechanisms, components, elements and configurations and peripheral equipment, utilities and accessories, as well the operation and implementation of exemplary forms and specific configurations and additional and optional aspects of the methods and systems disclosed herein are better understood by reference to the accompanying drawings and illustrative descriptions. The following illustrative description, along with the accompanying drawings, use the same terminology and reference numbers (i.e. numbers, letters and/or symbol) to refer to the same system units, device sub-units and devices, assemblies, subassemblies and mechanisms, components and elements.

The system is described in this illustration for tobacco processing. However, it could also be used to process other agricultural products.

The forms described herein focus on domains that involve the manufacture or processing of tobacco, blend components, or samples. They are also focused on domains that include automatic monitoring of tobacco processing via hyperspectral imaging, analysis, and aging, among other things. The forms described herein can be used to cover other areas such as tea manufacturing and processing, fruit juice production, grapes for wine production, and a wide range of agricultural products.

“The systems and methods described in this document have many utility. In one form, tobacco samples can be monitored throughout a specific process such as aging and fermentation to determine their progress and adjust the process parameters to get the best result.

Hyperspectral imaging is where a sample’s field of view is scanned and photographed while it is being exposed to electromagnetic radiation. The hyperspectral scanning is a process that generates multiple spectral images. These images are collected one-at-a time, but in a very fast sequence of objects emitting electromagnetic radiation at many wavelengths and frequencies. Where the wavelengths or frequencies are associated to different portions or bands of an entire hyperspectrum, the objects emit. Hyperspectral analysis and imaging can be used in a very fast manner to provide extremely high-resolution spectral and spatial data as well as information about an imaged sample. This is fundamentally different from standard spectral analysis and imaging.

“In general, electromagnetic radiation in the form light, such that is used during hyperspectral imagery, is incident upon an object. The electromagnetic radiation is affected either by one or more physical, chemical, or biological species or components of the object. This can be done by any combination electromagnetic radiation absorption or diffusion, reflection or diffraction scattering, and/or transmission mechanisms. An object that contains organic chemical components or species, will usually exhibit some fluorescent or phosphorescent characteristics when illuminated with electromagnetic radiation. The object’s electromagnetic radiation in the form diffracted, scattered and/or transmitted electromagnetic radiation is directly and uniquely related both to its physical, chemical and/or biological characteristics and the chemical species and components that make up it. This unique signature pattern or spectral fingerprint can be used to identify and characterize the object.

A typical spectral imaging system includes an automated measurement system with corresponding analysis software. The automated measurement system consists of optics, mechanics and electronics. It is used to irradiate a scene or sample using an illuminating source. Next, it measures and collects light from the sample or scene. Finally, calibration techniques are applied to extract the desired results. Analysing software is software that displays and presents useful information about objects in the scene.

“The hyperspectral images of a scene, or samples, could be obtained using commercially available hyperspectral imaging camera from Surface Optics Corporation, San Diego, Calif. or custom built to meet the user’s needs.”

Hyperspectral imaging is a combination of imaging and spectroscopy. A spectra is a collection of data at one point in spectroscopy. Spectra are information about the chemical composition of a sample and its material properties. They consist of a continuum that corresponds to measurements at different wavelengths. Traditional cameras, on the other hand, collect data from thousands of points. Each point, or pixel, contains one value (black-and-white image), or three values (color image), which correspond to the colors red, green and blue. Hyperspectral cameras combine the spatial resolution of cameras and spectroscopy. These cameras create images that have thousands of pixels and an array of values that correspond to different wavelengths of light. In other words, each pixel’s data is a spectrum. The corresponding spectra and pixels create a multi-dimensional cube. An image cube contains a lot of information, which provides a detailed description of the sample.

“Each spectral picture is a three-dimensional data set of volume of pixels (volume of voxels) in which the two dimensions represent spatial coordinates (or position) of an object, and the third dimension the wavelength. The object’s emitted light can be used to represent the coordinates of each voxel of a spectral picture as (x,y,?). Any wavelength, (? Any particular wavelength, (? Each spectral photo, which includes a variety of wavelengths of imaged sunlight of the object, is analysed to create a two-dimensional map of one or several physicochemical property, such as the geometrical shape, form or configuration of the object, and/or the chemical composition of its components, in a scene.

“Spectral profiles are a way to view image cubes as a collection spectra. They provide a comprehensive and detailed description of the entire image cube. To summarize the material composition of a sample, they use a collection of characteristic spectra along with their relative occurrences in an image cube. The amount of characteristic spectra that are extracted depends on the variability of the material. Usually, there is a small number to several dozen. By matching the spectra of an image cube with a particular spectra, spectral profiles can be created. To create the spectral profile for an image cube, the number of spectra that match each characteristic spectra is counted and normalized. The spectral profile can be viewed as a fingerprint that is derived from the hyperspectral image of the tobacco sample.

Hyperspectral imaging is a technique that generates multiple images from objects emitting electromagnetic radiation. These wavelengths and frequencies are associated with selected parts or bands of the entire spectrum. Hyperspectral images of objects are created from electromagnetic radiation emitted by them. These wavelengths and frequencies correspond to one or more of the following wavelength ranges: the visible, which stretches the wavelength range between 400-700 nanometers; the infra red band, which stretches the wavelength spectrum of approximately 700-3000 nanometers; and the deep-infrared band, which stretches the wavelength range from 3-12 microns. The data and information from hyperspectral imaging could be used to identify and classify the object and/or material.

“High speed hyperspectral imaging systems are often needed for various types of repeatable and unrepeatable chemical or physical processes that take place within the sub-100-millisecond timescale. These processes cannot be studied with regular hyperspectral imagery techniques. Combustion reactions, impulse spectro-electrochemical experiments, and inelastic polymer deformations, are examples of such processes. Remote sensing objects in distant scenes using rapidly moving platforms such as satellites or airplanes is another example of an observable that can change quickly and requires high-speed hyperspectral imaging.

“Disclosed is a system and method for monitoring the production of agricultural products, such as tobacco via hyperspectral imaging. Certain forms of these methods, protocols, procedures, and equipment are highly accurate, precise, reproducible, and robust in evaluating agricultural products such as tobacco. These methods are highly sensitive, precise, and fast during automated on-line operation.

“Certain forms described herein are focused on measuring, analyzing and determining microscale properties, characteristics and features of agricultural products such as tobacco. They can be used to analyze and determine the parameters of individual tobacco leaf samples and, more specifically, the various physical, chemical, or biological properties, characteristics and features of individual or multiple tobacco leaves within a tobacco bale, lot, or sample. One form provides an automated on-line monitoring system that uses hyperspectral imaging and analysis.

“Certain forms described herein use what will now be called?hyperspectrally detected and classifiable code? A?hyperspectrally detected and classifiable (or code) is as used herein. A micro-scale property, characteristic, feature or parameter of a bulk agricultural product such as tobacco, that can be hyperspectrally detected by hyperspectral imaging. The resulting hyperspectral data, information, and hyperspectral data, may include hyperspectral “fingerprint” Hyperspectral?fingerprint? or?signature? Patterns can be used to classify at least a portion of the single or individual tobacco leaves contained in a particular tobacco sample. The classified portion of each individual or single tobacco leaf within a particular sample of tobacco is then usable for monitoring the processing of tobacco and can be used to suggest or adjust processes to achieve desired results.

A?hyperspectrally detectable, classifiable code? is defined as follows: It is used to refer to an individual agricultural product such as tobacco. However, it can also be used to describe a single or single tobacco leaf within a particular tobacco sample. As part of a process for (uniquely, unambiguously) monitoring the manufacturing or processing of an agricultural product such as tobacco, the hyperspectrally detectable/classifiable codes can be used.

“Primary examples of micro scale testing for generating hyperspectrally detectable and classifiable codes, include: physical (geometrical/morphological) shape or form and size dimensions of single or individual tobacco leaves; coloring of single or individual tobacco leaves; moisture (water) content of, or within, single or individual tobacco leaves; type, distribution, and compositional make-up, of (organic and inorganic) chemical species or components, of single or individual tobacco leaves; types, distribution, and compositional make-up, of possible unknown or foreign (physical, chemical, and/or biological) matter or species and aspects thereof on, and/or within, single or individual tobacco leaves; activity and/or reactivity of single or individual tobacco leaves in response to physical stimuli or effects, such as exposure to electromagnetic radiation; activity and/or reactivity of single or individual tobacco leaves in response to chemical stimuli or effects, such as exposure to aqueous liquids or to non-aqueous (organic based) liquids; and activity and/or reactivity of single or individual tobacco leaves in response to biological stimuli or effects, such as exposure to biological organisms; physical (geometrical/morphological) shape or form and size dimensions of single or individual tobacco leaves; coloring of single or individual tobacco leaves; moisture content of, or within, single or individual tobacco leaves; types, distribution, and compositional make-up, of (organic and inorganic) chemical species or components, of single or individual tobacco leaves; types, distribution and compositional make-up of possible unknown or foreign (physical, chemical, and/or biological) matter or species and aspects thereof on, and/or within, single or individual tobacco leaves; activity and/or reactivity of single or individual tobacco leaves in response to physical stimuli or effects, such as exposure to electromagnetic radiation; activity and/or reactivity of single or individual tobacco leaves in response to chemical stimuli or effects (such as exposure to aqueous (water based) liquids or to non-aqueous (organic based) liquids); and activity and/or reactivity of single or individual tobacco leaves in response to biological stimuli or effects, such as exposure to biological organisms.”

“Accordingly, there is a method to monitor the manufacturing or processing an agricultural product. The method uses hyperspectral imaging. It involves: scanning at minimum one region of an agricultural product using at most one light source with a single or multiple wavelengths; creating hyperspectral pictures from that at least one area; determining a spectrum fingerprint for the agricultural product; comparing the spectral fingerprint to a database of spectral fingerprints from various points in the manufacturing process using a computer processor to determine where the sample has reached

“The method relies on obtaining hyperspectral signatures of the agricultural material being processed in order to reduce or eliminate the need for human assessment during processing. First, a standard database must be created. It should include hyperspectral signatures from each stage of a process such as tobacco aging or fermentation. This database is used to monitor the progress of each stage or step. The process can be adjusted in real time by controlling the processing parameters. This will ensure that the final product is of high quality.

“The method could include scanning multiple areas along the sample of agricultural products using at least one light source with a single wavelength or multiple wavelengths; and creating hyperspectral images of the multiple regions. This may also include determining the code for the sample.

“The agricultural product could contain tobacco. A sample may contain the tobacco. To minimize the incidence of the light beams on the tobacco bale, at least one light source can be used. The computer processor may use the cost of processing samples to monitor and adjust the process.

The method could also include repeating the steps of scanning at most one region of an agricultural product using at minimum one light source of different wavelengths and creating hyperspectral pictures from that at least 1 region. This creates a spectral fingerprint of the agricultural product sample from the hyperspectral photos, which can be used to identify a variety of agricultural products during processing.

“The method may also include storing data on spectral fingerprints from multiple samples of agricultural products taken at different stages of processing within computer storage devices to create a process database.”

“Further provided are systems that monitor the processing or manufacturing of agricultural products, in accordance with the methods described above.”

“A method is also provided for determining the processing stage for an agricultural product. This method uses hyperspectral imaging. It involves scanning multiple areas along a desired agricultural product with at least one light source that emits different wavelengths, generating hyperspectral pictures from each region, and then forming a spectrum fingerprint for the sample using the hyperspectral photos. Finally, the spectral fingerprint is correlated to a database of spectral footprints that contains a variety of fingerprints taken at various points during processing.

The method could also include storing data regarding the spectral signature within a computer storage device, as well as repeating the steps of scanning multiple areas along a desired agricultural product using at most one light source with different wavelengths. A desirable agricultural product could be unprocessed, semi processed, or fully processed agricultural products, such as tobacco.

“While the invention has been described in great detail for processing tobacco, it is important to understand that the tobacco used herein is only used to illustrate the methods described and not to limit their application. Referring to FIGS. Referring to FIGS. 1 and 2, the method and system for monitoring the processing or manufacturing of an agricultural product 100 is disclosed. It can determine the sensory attributes (102), which can then be used to evaluate the processing stage 104. The system 100 uses the spectral fingerprints (106 and 124) obtained using hyperspectral imaging system 110. Each spectral fingerprint (106 or 124) gives an indication of the chemical and physical characteristics of the sample of tobacco 108 or 122, or any other agricultural raw material. The sensory attributes and processing stages of different tobacco samples are determined by their chemical and physical characteristics 108 and 122. A database of the spectral fingerprints (106) of different tobacco can be created 118. This system is based on statistical prediction and/or neural networks and artificial intelligence techniques 120. It allows for the development of a system to monitor the processing or manufacturing of agricultural products 100. An optimization scheme can optimize the algorithm for cost reduction by including different tobacco samples along with their processing costs.

Referring to FIG. “Referring to FIG. A neural network 120 or artificial intelligence algorithm 120 is used to build an intelligent system 118. This provides a map of the hyperspectral signature and the processing stage, as well as the subjective sensory attributes 116 derived from a sensory panel 120. To optimize the processing scheme for sensory characteristics and cost effectiveness, the individual costs of tobacco can be used as independent parameters. The first stage 101 has ended. A composite hyperspectral signature is created. This can be used to correlate with satisfactory sensory attributes.

Referring to FIG. 2. In a second phase, spectral fingerprints of tobacco samples 124 are obtained at various levels of processing. These spectral fingerprints 124 of tobacco samples 122 are used to calculate the cost of each sample and to input system 100. Intelligent system 114 will use the expert system 101 to determine if additional processing is necessary. It will also assess whether process parameters need adjustment in order to achieve optimal cost and acceptable sensory attributes. The input parameters, spectral fingerprints (124), and cost information for each sample are used.

“Accordingly, the method is for monitoring the manufacturing process of an agricultural product. This method uses hyperspectral imaging. It involves scanning at least one area of an agricultural product with at least one light source, at different wavelengths, and then generating hyperspectral pictures from that at least 1 region. Then, determining a spectrum fingerprint for the agricultural product by comparing the spectral fingerprint from step (c) to a database of spectral fingerprints, which contains a variety of fingerprints taken at various points in the manufacturing process. A computer processor is used to determine the point at which the sample has reached. One or more features of the spectral fingerprint can be identified that correspond to desirable sensory attributes. One or more of the spectral features of an agricultural product sample may be correlated with the desired sensory attributes.

“The agricultural product could contain tobacco. The tobacco can be in the form a bale or lot, or as a sample. One light source can be placed so that the sample is imaged with the least amount of light possible.

“In some cases, the computer processor may consider the cost of samples or their processing.”

“In some cases, the agricultural product is tobacco. The manufacturing process is a fermentation process. The method used to ferment the sample of tobacco determines how long it takes.

“In some cases, the agricultural product is tobacco. The manufacturing process is called a tobacco-aging process. The method used to determine the amount of time it takes to age the tobacco sample will affect the length of the process.

“As you may see, the method described herein can be computer implemented.” Data about the spectral fingerprints for a plurality of agricultural products are stored in some forms within computer storage.

“A method of determining the processing stage for an agricultural product using hyperspectral imaging is also provided.” This method involves scanning multiple areas along a desired agricultural product with at least one light source at different wavelengths, generating hyperspectral pictures from these multiple regions, and then correlating the spectral footprint obtained in step (c), to a database of spectral signatures that contains a variety of fingerprints taken at different points of processing using a computer processor to determine the processing stage.

“Another aspect is a method for determining the processing stage of a product. This method involves determining whether a sample meets the desired attributes for the product, and then applying hyperspectral analysis and theoretic analyses to establish a relationship between P and said unique spectrums. The unique spectra include at least two spectral element x, y, and their respective values. Finally, hyperspectral analysis is used to characterize the sample using said spectral components (at least x, y) of said unique spetra P. After that, mathematically solving from these characterizations to determine if the samples meet the values of P.

Referring to FIG. Schematically, a system 10 is used to monitor the manufacturing process of an agricultural product using hyperspectral imaging. The system 10 contains at least one light source 12, which provides a beam light. The at least one lightsource 12 can be mounted on an arm 14, which allows for the positioning of at least one source 12 near the agricultural product (not illustrated). This light source may also be placed on platform 50. Arm 14 may be mounted to the frame 16 of cabinet 20, and can be fixed or moved as described below. FIG. 3 shows that a second light source may be added to the frame 16 or to an arm (not illustrated).

“In one form, at least one light source 12 is required to provide a beam of light with different wavelengths. This could be a tungsten, light source halogen, or xenon source. A mercury light source is another option. Another form of the at least one source of light or the second source of light 18 includes an ultraviolet source to provide a chemical signature for the agricultural product P. Another form is the at least one source 12 which includes a xenon source. The second source 18 contains a mercury source while the third source (not shown), comprises an ultraviolet source.

“In one form, at least one light source (12 and/or 18) may be placed to minimize the angle for the incidence of a beam light with the agricultural product.

Cabinet 20 may have walls added (not shown) to separate ambient light from system 10. This enclosure will provide system 10 with a dark-room environment.

“Hyperspectral imaging camera 24 is used to obtain the hyperspectral image of a scene, or sample.”

“Test results were based on scanning and counting individual samples. Each sample consisted of dozens of scans and each sample was classified using spectral bands features, spectral fingers (SFP), major components of major spectral spectrum, purity and quality, relative quantity of each component, SFP, major spectral representative compounds, purity and quality, major spectral components, morphological and morphological characteristics, optionally crystallization.

“A variety of agricultural products such as tobacco at different stages of processing are scanned. To reduce the impact of variability in sample results, it is important to scan a large number of samples for tobacco products. It has been shown that this variability can have a reduced impact on the quality of the samples if the N number is between 5 and 25. If you carefully select representative samples, it is possible to include all variations that may occur in the processing of a product. This technique can be applied to tobacco. Toxic samples may be scanned with xenon, mercury, tungsten and/or other halogen light sources. An optional ultraviolet light source could also be used for chemical signature classification.

The hyperspectral camera system creates a hyperspectral image cube during scanning. The image cube may, for example, be approximately a 696 by 520 pixels array. This would mean that a picture or frame with 361,920 pixels would be possible. The skilled artist will be able to see that each pixel could contain approximately 128, 256 or 500 spectra points at different wavelengths.

Referring to FIG. “Referring now to FIG. Step 210 is where a dark image (or reference image) is obtained in order to calibrate the system. The reference image is analysed and calibration coefficients obtained in step 220. A hyperspectral image is taken of a sample of tobacco. Step 240 is where the calibration information is used to remove dark values and normalize the sample image. To compensate for variations in operating conditions (e.g. light intensity, ambient conditions etc.), calibration coefficients are used in step 250. Step 260 is where the steps 230 to 250 are repeated for each sample. The data thus obtained is then added in step 277 to the database of spectrum hypercubes (whole dataset). The algorithm used to create the database, as shown in FIG. 4. is only an illustration and should not be considered as limiting. These same steps could also be used to create a spectral database that can be applied in other agricultural products, such as tea, fruits, grapes, or other products. The final result is 350. This is a spectral library that contains all the samples and could be used for monitoring and assessing the processing of the agricultural product. It also includes the unique spectra and spectral fingerprints from step 340.

Referring to FIG. 5 shows a method of analyzing data in order to create a 300-sample spectral library. Step 310 is used to create the spectral library 300. The dataset is preprocessed in step 320. Step 330 identifies unique spectra that are indicative of the dataset. Step 340 is where spectral distributions (spectral fingerprints), are calculated for each sample using the unique spectra from step 330.

“In certain forms, after imaging, several data processing procedures are performed to reduce noise and increase consistency, enhance feature extraction spectra, and reduce computation times. Below is a summary of the data processing routines.

“Spectral binning can be used to produce consistent signals, reduce file sizes, and decrease processing times. Spectral binning involves the addition of adjacent wavelengths in order to create a down-sampled spectrum. The signal-to-noise ratio is higher for down-sampled spectra, which have less resolution. The sampling rate that produces the signal with minimal compression and loss of fidelity was selected. The median filtering is used to reduce noise and improve the signal-to-noise ratio.

Spatial Binning applies to an image cube and involves down-sampling by summating adjacent pixels. Because pixels contain spectra, spatial binning results is the summation and subtraction of those spectra. This reduces the file size and processing time, increases signal-to-noise ratio, and reduces file size. Because the camera has a high resolution, information loss is minimal. Summation allows each pixel’s spectrum to contribute to down-sampled data. Pixels adjacent to one another are often from the same leaf and have similar spectra. A Gaussian filter can be used to achieve similar effects and increase spatial coherency. pixel classification. When spatial reduction is not desired, a Gaussian filter might be helpful.

“In certain cases, image correction can be achieved by collecting dark images twice daily. These images are used to calculate and remove sensor noise during processing. Two steps are required to process dark images. First, dark images’ spectra (?dj?Di) are binned to ensure that they are consistent with other cubes. Next, the spectral mean (?dmean:i) is calculated (Eq. 1) is used to estimate sensor noise.

“s ? “s???? i mean = ? ? ????? j S i ? S i? s???? j?? ( 1 )”

“During data processing,?dmean:i is removed by subtraction from each spectra for each sample. This is the standard way to remove sensor noise.

“Reference image cubes (Ri) are taken twice daily and contain spectra that can help measure and correct lighting inconsistencies. Any sensor drift or variation will be eliminated by applying a correction using the reference image. This is a critical step as any changes in the lighting conditions or sensor response drift could adversely impact the system’s performance. Additionally, pixels in shadow have a low signal-to-noise ratio and should be removed.

“Most signals have a maximum peak that is created by the spectrum signature of the light. This value is proportional both to the amount of light hitting it and to the sensor’s response. Maximum peak thresholding is used to detect shadows.

{“pixel = “pixel = not? ? Shadow? shadow? ( s ? ( s???? i?)? thresh Shadow? : max ( s ? ( s????????? i?) threshnwhere maximum(?si), returns the maximum component in the spectra,?si).

Shadow spectra are those with a maximum peak that is less than a user-defined threshold. These spectra are ignored during spatial binning and spectral extraction as well as spectra matching.

Images of tobacco samples are made up of thousands of spectra. Many of them are almost identical and correspond with similar sensorial effects and material properties. The image cube contains information that can be summarized into a spectral profile. This is a collection of characteristic spectra, and the rate at which they occur within a sample. Two primary steps are required to create a spectral profile.

“1. “1.

“2. “Spectra Matching”?matching an image cube?s spectra with characteristic spectra.

The first step in building a spectral profile involves creating a set or end-members. End-members can be manually chosen by selecting pixels that correspond to a particular class or contain unique materials. Many agricultural products, such as tobacco, can have distinct characteristics that are difficult to identify manually. Class-specific spectra are not available for many of these products. An automated spectra extraction method is preferred in such cases.

“Spectra extraction is different from other automated end member extraction techniques because it divides the spectrum feature space using an evenly spaced grid. Spectra extraction searches for spectra that are more similar than a threshold (??) set by the user. It is assumed that two spectra will be more alike than one another. They can be considered duplicates if they are identical to each other. All unique materials can be identified by comparing all the spectra within a data set. Un-mixing models are another popular automated spectra extraction algorithm. These assume that individual pixels have unique spectral signatures derived from multiple materials. Two examples are the Sequential Maximum Angle Convex cone (SMACC) or Support Vector Machine Based End-member extract. This technique was chosen because it is simple. For smaller scale agricultural imaging, the un-mixing model may not work well. In this case, individual pixels are measured in millimeters, as opposed to aerial imagery, which is measured in meters. In the first step of the spectra extraction process, each image cube is independently analyzed in what is called local spectra extract. These results are then combined in global spectra extract. This order of extraction reduces processing time and allows outliers of individual cubes to disappear.

Once the characteristic spectra have been extracted from the data set, the spectra matching step can be applied to each image cube. Each?sji?Si is matched to the most similar?ck?Call. A spectral profile is the sum of all matches for each??ck?Call as an image cube. Each component of the?pi represents the number of matches for one?ck?Call. After all?sjiSi have been matched,?pi represents the percent chance of each??ck?Call in SI.

“Inclusion of unidentified pixels may be beneficial in certain situations, such as when tobacco samples have been contaminated with non-tobacco materials. Shadow detection may not be reliable or only a few spectra should make it into the spectral profiles. Spectra which are less similar than???Call will be matched. Unidentified pixels are considered unidentified. The spectral profile is not affected by unidentified pixels. ??? ???????? is a parameter that can be set by the user. Larger values of???? will allow for more dissimilar spectra to match to??ck.Call, while smaller values will only allow matches with similar spectra. Higher values of??? will allow for more dissimilar spectra to be matched to?ck??Call, while smaller values will allow matches only with similar spectra. A high?? Setting a high?? will result in a match with the closest “?ck?Call.”

“Spectra matching is the main goal of hyperspectral imaging analysis, as previous hyperspectral image classification problems focused on pixel-by-pixel classification. These applications use machine learning algorithms, such as support vector machines or decision trees, to classify spectra. This method of matching requires a set of characteristic spectra, which isn’t practical. Full-automated techniques are preferred, as well as spectral feature fitting. SFF is used to identify spectra by using specific features. SFF is possible to achieve success, but SAM is more suitable as it seeks to determine the similarity of two spectra using all bands, as confirmed by the results.

“Feature selection is about selecting a subset that can summarise the data without any information loss. To avoid dimensionality which can reduce classification performance, it is used before classification. It is possible for spectra profiles to contain redundant features, especially when the threshold for spectra extraction,??, is low. Low spectra extraction thresholds can lead to redundant features in spectral profiles. Experiments suggest that the selection of the appropriate?? Experimental work suggests that the selection of the appropriate?? is possible without the need to select features. This is due to the ability of support vector machines, (SVM), to handle redundant or uninformative information. In some cases, however, we found that the Jeffreys Matusita Distance was an effective information measure in selecting the best subset of features.

Summary for “Hyperspectral imaging system to monitor agricultural products during processing or manufacturing”