Naoshi Kondo - Kyoto U · Food Traceability Based on Robotic Agriculture - Precision Agriculture...
Transcript of Naoshi Kondo - Kyoto U · Food Traceability Based on Robotic Agriculture - Precision Agriculture...
4/27/2011
Food Traceability Based on Robotic Agriculture- Precision Agriculture Oriented Machines -
Naoshi KondoDivision of Environmental Science and Technology, Graduate School of Agriculture, Kyoto University Kitashirakawa-Oiwakecho, Kyoto 606-8502, Japan
The world is worrying about the quality of food supply.
food poisoning by bacteria, illegal unregistered agricultural chemicals,
camouflage products, BSE, O-157…and Bio-terrorism.
Food Traceability Systemduring production, distribution and consumption
4/27/2011
Robotization in Agriculture
Transplanting robot
Strawberry harvesting robot
Fruit grading robot
Leek pre-processing robot
Robotic sprayer
Real-time Soil SensorAutonomous tractor
Autonomous transplanter
Autonomous combine harvester
Under developmentCommercialized
Field management
Residue process InformationShipping
Seedling production
Plant management
Harvesting
Pre-Processing
Grading
Cutting sticking robotGrafting robot
Measurement BoxPower source, PC,
Spectroscopy,PLC, amp board
Antenna of DGPS
Chisel(sensor probe
housing) ※ Date: 2004. 11. 21 Depth: 150mm, Speed: About 30cm/ secCooperation with SHIBUYA MACHINERY CO.,LTD., TUAT
Commercialized Real time soil sensor
Field management
4/27/2011
Arrangement of Sensors in chisel
Illumination fiber
EC Electrode
Laser Displacement sensor Color camera
Condensed fiberGround surface
Soil flattener
4/27/2011
MoistureContent
NO3-NSOM
pH EC
Real-time soil sensor
N, P, K , EC, pH, moisture content, SOM,soil temperature, compaction
Maps from Soil Sensor
phosphoric acid, potassiumBy courtesy of
Prof. Shibusawa, TUAT
4/27/2011
Autonomous vehicles
Robots keep operation record automatically.
By courtesy of Prof. Noguchi, Hokkaido U.
4/27/2011
Seedling production
operation recordsCrop varietyPlant ID, Color and size of seedlings
Grafting robotCutting sticking robot
Transplanting robot
4/27/2011
Crop management
Spraying robot
Bagging robotCombination of fertilizing robot
and field soil sensor
“information oriented” field
When, where and what kinds of
chemicals, fertilizer???
operation records:irrigation, chemical spray, Fertilization Turf mowing robot
Harvesting robot (strawberry, tomato, cherry tomato, cucumber, eggplant, cabbage, mushroom, orange, grape, hay, rice, wheat and etc.)
4/27/2011
4/27/2011
A strawberry harvesting robot
Robot constitution3 DOF manipulatorSucking and cutting end-effectorStereo vision by use of color CCD camerasDL (Lighting devices) with PL filterRail type traveling device
Product location
Harvested time and date
Crop ID
4/27/2011
A harvesting robot in plant factory
4/27/2011
Other operation robots
Twig cutting robot
Milking robotWool sharing robot
Meat inspection
4/27/2011
Remote cowshed watching service
4/27/2011
PearApplePeachPersimmonTomato
Fruit Grading Robot
4/27/2011
②
Pusher
TV camera
Halfway stage
x
Lifter
①③
④⑤
⑥
⑦
⑧
z
Blower
Fruit providing robot
Gradingrobot
Pusher
Rotation
Constitution of robotic grading system
4/27/2011
Stroke:1165mm
①④
③
⑤
②
100
467
308
290
Path
Speed
① ④③ ⑤② ④
1000mm/s
250 317 308 145 145 (mm)
(s)0.5 0.317 0.683 0.6 0.3 0.3
2.7 s
Cycle time of the stroke:4.25 sItems
2.7 s (Go)
0.4 s (Initial movement)
1.0 s (Return)
0.15 s (wait)
Rotation
Performance of grading robot on a condition of 80% 24 fruit containers and 20% 15fruit containers:12/4.25×3600×0.8=8131 fruits10/4.25×3600×0.2=1694 fruitsTotal 9825fruits/hour/robot
Rotation
Action and performance of fruit grading robot
(mm) Halfway stage
Tray on line
4/27/2011
Information of fruit appearance
Side(1) Side(2)
Side(3) Side(4)
TopBottom
Side(1) Side(2)
Side(3) Side(4)
Top Bottom
Side(1) Side(2)
Side(3) Side(4)
Top Bottom
+ Internal quality information(Sugar & acid contents, rot, ….etc.)
Original images Color conversion images Processed images
4/27/2011
4/27/2011
0
500
1,000
1,500
2,000
2,500
1 6 11 16 21 26 31 36 41Producer No.
Fruit No.
L1
L2
L3
L4
L5
Grading result of pear on Sep.24, 2002
L�2�
4/27/2011
Machine Vision Technologies in Bioproduction
Oct.21, 2008, AITPL filtering Dome or diffuser
MirrorDiffuserLight
True color HalationWall reflectance
CameraLight
Camera
Reflecting plate
PL filtering Light and Dome
PL filter
4/27/2011
PL Filtering Light
Ready to sell! 10 year warranty“DL” is an SI Seiko-original illumination equipment for image acquisition to make direct lighting possible.
Small power(50W) and high conversion efficiency from electricity into light.
PL filter Heat absorption filters
Fan(more than 1000nm)
(800~1000nm)
Halogen lamp
120 white LEDs
4/27/2011
Direct luminaire (PL) Indirect luminaire (Dome)
True color Inavoidable reflection
4/27/2011 ComparisonTomatoes with plastic package
With drops
Indirect light
With drops
PL
With drops
PL
With drops
Indirect Light
4/27/2011
Outer epidermis cell
Parenchymatous cell(Inner tissue)
Cuticular layer(Wax layer)
Fruit, leaf, stalk
4/27/2011
Popular round fruits
Citrus fruitsTomatoWaxed apple (Tropical fruit)PotatoPersimmonOnionPepper
4/27/2011
Larger round fruits
Water melonMelonPumpkinCabbageLeaf vegetables
4/27/2011
History of the NIR inspection system
In 1989, A NIR sensor to inspect the peach was introduced (Mitsui Co., Ltd.)Yamanashi Pref. (Japan)In 1995, Orange sugar content and acidity
(FANTEC)
At present, NIR sensors more than 300 place grading machines are installedMitsui Co.Ltd. has distributed more than 800 sensors
Price of sensor (1 pc.): $100,000-- 400,000Maker: 3 – 4 companies in Japan
Grading machines which use NIR inspection systems are supported by national government
4/27/2011
Commodity Measurement
Orange Sugar content, Acidity (Void and low moisture content)
Kiwi fruit Sugar content, Acidity (degree of hardness, post harvesting ripening process)
Apple Sugar content, Acidity, Maturity degree, degree of ripeness, rotten portion, (blackheart)
Pear Sugar content, Acidity, Maturity degree, degree of ripeness, rotten portion, (blackheart)
Peach Sugar content, Acidity,
Onion Sugar content, blackheart
Potato Cavity, blet
Water Melon Sugar content ,Ripeness
Melon Sugar content ,Ripeness
Persimmon Sugar content, tanin
Others (Pine apply, Mango, etc)
Measurement details
4/27/2011
System details (Semi-transmissive)
Optical sensor
Lamp
Light transmission carrier
Optical measurement system
Products Apple・Peach・Pear・TomatoWater melon・Melon etc
Gauge control Unit Below orange areaConveyor speed Max 36m~60m/minNumber of Lamps 2~10
Light source are kept at the side and transmitted light is received at the bottom by optical sensor
4/27/2011
Lamp
System details (Total transmittance)
Products Orange and others
Gauge control Unit Full Internal
Conveyor speed Max 60m/min(5~10/sec)
Number of Lamps 1~ onward
In this case fruit is allowed to pass between source and sensor. Transmitted Optical rays are received by the optical sensor
Optical measurement system
Optical sensor
4/27/2011
Other sensors to measure sugar content
Portable type
Handy Type
(Mitsui Co. Ltd)
(FANTEC) (ASTOM)
Can find in the department stores
for consumers’ checking
Use at the farm for producers
4/27/2011
Coffee Break
What is terahertz waves?
Characteristics of THz waves- Permeability - Easy handling- Reasonable spatial resolution - Safe for humans
The spectral region of THz wave between microwave and infrared still remains largely undeveloped mainly because of the lack of proper tunable sources.
4/27/2011
Stop Drug Addiction
Stop Drug Smuggling at
Post Office
By Terahertz (THz)
4/27/2011
Elongated fruits
Eggplant, Cucumber
4/27/2011
Specific longer products
Preprocessing before grading
Leek
4/27/2011
size, color, shape, disease, internal injurysugar and acid contents, residual chemical
4/27/2011
FertilizingChemical sprayingIrrigation……
Accumulation to DB
•Appearance•Internal quality
Data from grading system can be used for precision farming
On Geographical dataGrading systemGrading robot
Farming support system based on GIS
Field information
Grading information
Management information
4/27/2011
Field information
Weather information
Chemical information
Yield information
Quality information
Mapping on GIS dataGeographical information
4/27/2011
Roles of Fruit Grading facility
1. Efficient sorting, and labor saving2. Uniformization of fruit quality3. Enhancing market value of the products
(Establishing local region brand of products)4. Fair payment to producers based
not only on quantity but on quality of each fruit5. Farming guidance from grading results and GIS6. Contribution to the Traceability system
for food safety and security
Producer Operator Distributor Consumer
4/27/2011
Flow of information from field and product
Operation record:Irrigation, fertilization,chemical spraytransplanting, pruning
Plant variety, Crop ID, Size, Color, Disease
3D location of productHarvesting time &date
Crop ID, Size, Color,Disease
Appearance(color, size, shape, bruise, disease)Internal quality (Sugar & acid contents, rotten core, inside defect)Graded rank, Chemical residue, IngredientsDistrict name
Harvesting GradingSeedling productionCrop management
Selling price, selling time, information of arrival of goods, quantity to sell, Opinions of costumers (on taste, freshness….)
Field ID (address, height above sea level), Producer IDClimate, weather information (Temp, humidity, irradiation, precipitation)
Field management
N, P, K, pH, ECSOM, MC, Temp.,Compactness,
Transportation Environmental condition (Temp., humidity)Package type and method, transporting method, time, and distance
Production information
Consumption information
Distribution information
4/27/2011
Traceability System with a fruit grading robot
4/27/2011
ROM-Writer
Main flow on traceability data
TraceabilityDatabase
1
34
5
76
2 SizeColorShapeDefectProducer ID
Field IDReceived dateVarietyContainer numberFruit number
PC
Farming operation recordsChemicalsFertilizers……..
Grading robot
Product ID
Sorting
Barcode reader
Packing robot
Product ID
Transportation dataEnvironmental condition
Sales dataPrice, quantity
IC memory
Field Information
Reception ID
Grading ID
Product ID issued date and timePacking robot No.Grading and reception information
http://www.jau-brand.jp
4/27/2011
Soil analysis centerProduct information center
Agricultural Cooperative Association
Soil sensor
Grading robot
Intelligent farmingPrecision farming
Consumer Market (Distribution)
Product informationField information
Farming guidanceDSS for farmers
ResidueCarbonization
BRAND BRANDVoice of consumer Quantity andmarketing value
Analysis of soil and chemicalsGIS, DSS
Variable distribution channelMarketing route
BiomassRe-uses
Transport
Operation recordsSensing information ID tags
FreshProduct
New flow of product and information
Information oriented field Information added product
FPAC, Province
Price Difference between US & Japan
At Meijier, Columbus OH, and Fuji supermarket, Matsuyama, Ehime, June and July, 2005
0100200300400500600700800900
1000
Tomat
o
Cucum
ber
Eggp
lant
Baby
eggp
lant
Color s
w-pe
perLe
ek
Cabba
ge
Lettu
ce
Broc
oryCor
n
Napp
a
Daiko
n
Apple
(Fuji)
Water
melon
Melo
nKiwi
US
JPN
Pric
e (Y
en)
vegetables fruits
4/27/2011
2 dollar melon (Cantaloupe)USA
5 dollar (¥500) melon Japan
Difference between American & Japanese Melons
4/27/2011
More precise in future…
4/27/2011
Mobile fruit grading robot
Field
Canopy
Mobile grading robotN
Course of mobile grading robot
Harvesting date and time, location of product Yield mapQuality mapTree management
Appearance information, Internal quality information
(Information from harvesting robot)
(Information from grading robot)+
Mobile grading robot for orange fruits
This robot results will be presented soon.
4/27/2011
DL End-effector
ZX
Power unitsServo driversElectric balance
Container
Computer
TV camera
Mobile grading robot for sweet pepper(A 3DOF manipulator, an end-effector,
5 color cameras, 9 Direct Lightsand three photo sensors to give triggers)
4/27/2011
Thank you
Any Questions?
4/27/2011
4/27/2011
Soil Reflectance 400-900 nm for 256 channels900-1770 nm for 128 channels
RTK-GPS Location within 2 cm accuracyMicro CCD Camera Soil Color Image with 512X512 pixLoad Transducer Soil Strength, Cutting ResistanceEC Electrodes Soil Electric Conductivity
Examples of soil images captured every 1m in a paddy field at 30 cm depth.
IR_1525
0.2
0.3
0.4
0.5
0.6
800 1000 1200 1400 1600 1800
Wave length (nm)
Reflecta
nce
系列1
系列2
系列3
系列4
系列5
系列6
系列7
系列8
系列9
系列10
系列11
系列12
系列13
Vis_1525
0
0.1
0.2
0.3
0.4
0.5
400 900
Wave length (nm)
Reflecta
nce
系列1
系列2
系列3
系列4
系列5
系列6
系列7
系列8
系列9
系列10
系列11
系列12
系列13
Examples of soil reflectance collected every 1 m for visible-NIR (left) and NIR (right) wave ranges in an upland field.
Data Collected by the Soil Spectrophotometer
46.8 50.8
48.5 48.6 50.3
51.8 49.0 50.1
53.0 48.4 49.5
51.7 44.5 49.7
49.9 45.0 50.2
49.5 48.0 51.2
50.7 46.3 51.3
51.4 45.8 51.6
49.9 49.5 50.9
50.7 45.9 51.0
48.7 44.6 51.6
48.5
51.4
50.5 47.9
46.2 51.6 48.1
46.4 52.1 48.9
46.9 52.7 50.3
48.3 54.4 50.7
48.9 54.4 51.9
50.5 51.8 52.0
52.3 53.5 53.1
50.0 54.0 54.4
49.2 54.4 54.4
48.2 54.1 56.0
48.1 56.3
47.8 53.7
45.9 43.0 46.3
48.3 47.7 47.4
47.3 50.4 48.7
46.4 52.4 45.9
45.1 49.2 44.9
47.6 47.6 43.6
45.9 45.5 44.1
47.3 44.9 44.3
46.3 44.8 43.7
46.2 43.6 44.2
46.4 44.2 43.4
46.6 44.9 43.0
45.8 45.8 44.5
45.9 46.1 44.4
46.6 47.3 43.8
46.1
44.6 43.6
46.0 44.8 43.9 47.1
45.2 45.1 44.8 42.9
44.0 45.2 46.0 41.5
46.8 45.9 45.2 43.0
47.0 46.4 46.7 46.5
48.4 47.3 47.0 46.8
45.1 48.3 47.6 47.4
47.1 48.5 48.7 47.3
47.2 48.6 47.3 50.3
50.8 48.7 51.4 49.8
53.5 51.1 58.1 51.1
53.3 53.0 55.2 52.8
53.1 55.1 54.9 52.7
55.5 57.9 58.1 56.3
58.1 58.6 60.0
70.0 60.4
40.1 41.1 41.7 43.4 42.3 40.5
40.5 40.7 40.6 39.4 41.1 41.2 40.4
44.1 43.5 43.5 43.7 45.1 44.7
39.4 41.6 42.0 41.6 40.0 40.4 45.4
40.6 41.7 43.0 43.5 42.1 43.9
40.1 40.1 43.4 41.0 40.8 40.2
40.9 40.2 40.1 40.8 41.5 43.1
41.0 40.9 41.7 43.3 45.7 50.6
42.3 41.2 43.0 39.8 44.0 47.6 49.0 45.8 46.9 47.3 45.6 44.8 44.1 44.8 45.2
38.5 40.6 40.7 41.4 42.1 41.2 40.8 40.5 37.8 37.7 38.5 37.7 38.9 40.5
25.8 36.2 40.8 43.1 41.8 40.5 37.6 41.3 39.2 38.4 39.6 39.8 39.3 41.3 44.6
36.2 36.6 38.0 40.5 41.1 38.7 40.5 43.9 41.1 40.7 34.8 43.5 42.7 42.8
49.3 47.0
45.6 44.3 41.9 46.4
46.1 48.3 41.5 47.6
44.5 47.8 40.5 48.5
45.2 47.5 40.6 41.2
45.6 46.3 40.1 42.6
44.1 45.2 41.5 43.9
43.2 46.8 41.3 42.7
43.6 46.8 43.4 43.8
45.0 44.6 43.2 45.6
47.3 50.7 40.9 46.6
43.3
41.5 45.1 44.2 43.8 48.3 45.8 44.2 46.1 47.7 45.4 43.0 44.7
44.0 45.4 46.1 46.5 46.6 46.3 44.9 46.4 46.0 44.6 43.9 44.5
45.3 44.9 45.0 44.8 44.6 45.1 44.5 45.2 44.8 40.9
43.9
40.9 41.7 41.1
40.4 42.6 40.5
42.4 40.2 40.8
41.9 42.1 40.6
41.3 42.8 40.5
39.7 42.7 41.9
39.2 42.6 40.6
39.3 43.2 40.8
39.6 42.5 41.9
38.3 41.2 46.2
34.0 43.9
49.6 62.5
48.2 47.4 53.2 58.2
48.1 47.8 48.9 60.0
46.8 45.0 46.0 57.9
47.2 44.5 46.3 56.9
42.9 48.7 47.7 54.4
43.1 46.0
Unit:μS/cm
0
135
240
345
450
555
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
0
20
40
60
80
100
1 2 3 4 5
頻度
(%
)
148149 91131
144
147
157
120-1 120-2
50
49
Average: 49.3
Average: 51.4
Average: 46.0
Average: 49.3
Average: 41.9 Average: 42.0
Average: 44.7
Average: 41.1
Average: 45.1 Average: 41.5 Average: 49.9
Soil Variability Maps : MC, pH, EC, TC, TN
4/27/2011
4/27/2011
Display during measurement
Visible regionNIR
Subsurface Soil image
Spectral Data
START
STOP
FINISH
Compactness
Fluctuation on surface
Latitude
Longitude
Image Analog DataSpectrumMeasureStop