��«Ӱҵ

Some day the manual labor of harvesting chili peppers may be replaced by instructions in a computer manual.

Artificial intelligence is increasingly being used to teach robots how to perform traditionally manual tasks such as planting, weeding and harvesting of specialty crops like berries, peppers and other fruits and vegetables. 

A team within ��«Ӱҵ’s Machine Vision and Optical Sensor Lab in the Department of Agricultural and Biosystems Engineering is studying that technology for three varieties of chili peppers. Assistant professor Pappu Yadav leads the lab and directed the 2024 study “AI-Driven Computer Vision for Detection and Pose Estimation of Chili Peppers for Robotic Harvesting.” 

The AI aspect of the study was handled by Amee Parmar, a junior computer science major from Brookings who is one of this year’s Future Innovators of America. 

The Future Innovators of America Fellowship is a creation of the Jerome J. Lohr College of Engineering. Twelve recipients were named for 2024-25. They are awarded $5,000 with $4,500 as a stipend and $500 to cover the cost of lab supplies or travel to disseminate the results of their project. 

The fellowships were created to provide unique research opportunities for undergraduate students in the college. Any student is eligible to apply as long as they are attending full time and have a GPA of 3.0 or higher. Each student works with a project mentor, who must be a faculty or research staff member, to develop and submit a research plan that entails learning by doing.

A series of first for Parmar

Yadav was in need of an undergraduate assistant to help on the research, and his department head was aware of Parmar’s computer interest.

Parmar took an interest in computer coding while a ninth grader at Brookings High School and chose to major in computer science to improve her coding skills. She has enjoyed developing her skills in C and C++ computer coding while attending SDSU, but the pepper project produced several firsts for her:

• First undergraduate research experience

• First time to use Python computer language

• First time to use AI for anything more than a personal project

Parmar estimates she took 1,000 images of Hungarian hot wax, poblano and black Hungarian chili peppers under various orientations (horizontal, vertical and diagonal) and stages of ripeness while growing in the SDSU greenhouse. Those were then scanned into a computer to create a model that would direct a robotic harvester when a pepper was ripe.

Testing was a mixed bag, Yadav reported. When detecting ripeness, the AI-directed robot was accurate 53% of the time for black Hungarian and 79% of the time for Hungarian hot and poblano. Orientation accuracy was consistently high at 89% to 90%.

Project to yield paper, journal article

To take the roboticharvester beyond the greenhouse, a new algorithm would have to be written that would account for different lighting in nature, and the robot itself would need a more stable base, Parmar said.

She started on the project in mid-May after spring semester ended. Her role with the project is wrapped up except for writing a paper that can be presented at the Society of Photographic Instrumentation Engineers Defense + Commercial Sensing conference in Orlando, Florida, April 13-17.

Parmar said it was gratifying to have her work used in a real-world application and to effectively use AI as well as work in a team.

Yadav said Parmar was “highly enthusiastic and motivated to learn about AI and robotics. She grasped the coding aspect pretty fast and was a great team player. She worked diligently with other group members in the lab and was always ready to assist them in their research projects.”