How Cameras Enable Vision in New-age Agricultural Robots

This blog post was originally published at e-con Systems’ website. It is reprinted here with the permission of e-con Systems.

Embedded vision has been transforming agriculture by enabling autonomous mobile robots to ‘see’ their environment for the purposes of picking & harvesting, plowing, weed & bug detection, etc. Jump right into this article to learn how embedded cameras have been a catalyst to this transformation.

Auto farming or agriculture automation is no longer a distant dream for an industry that has historically dealt with high labor costs, poor manual productivity, and siloed data. There are also constantly major problems concerning policy changes, weather volatility, and other environmental factors that could cause untold losses. It’s why AI-powered autonomous mobile robots have been such a boon to the modern agriculture industry.

And embedded vision has been at the forefront of this revolution by leveraging camera technology to help robots ‘see’. Embedded vision and artificial intelligence have, after all, enabled agriculture businesses of all sizes to significantly cut down manual labor costs while accelerating the speed of operations. In addition, it has led to an unprecedented improvement in crop yields, which has increased profitability.

Today, the scope of embedded vision applications stretches as long as the horizon. Agricultural robots, drones, and tractors used for auto farming make use of camera-based systems for the purposes of navigation, crop health assessment, pest detection, harvesting, planting, weed detection, watering, spreading fertilizer, etc.

Equipped with the right sensor, interface, processor, and ISP (Image Signal Processor), the camera acts as the eye – capturing high-quality images of crops and vegetation. Images obtained from these camera modules are analyzed using computer vision and AI algorithms to identify patterns and check for anomalies or other requirements. The farmer gets access to real-time data about crops, terrains, and weather patterns. It provides actionable insights that tremendously improve their decision-making capabilities.

In this article, we look at some of the modern embedded vision applications in agriculture where robots and drones are used, and how cameras play a part in each of those.

Applications of embedded vision in agricultural robotics

When we say AMRs (Autonomous Mobile Robots) or simply robots in agriculture, we refer to any autonomous equipment that can automate agricultural activities by seamlessly navigating through fields and performing tasks that were traditionally carried out by humans. Hence, we will look at the role of cameras and embedded vision in agricultural robots along the following types of machines:

Agricultural robots
Agricultural drones
Autonomous tractors

Agricultural robots

Agricultural robots come in a very wide range of sizes starting from a little more than a couple of feet in length to many meters long. They are used predominantly for the following tasks:

Picking, harvesting, and planting
Weed and bug detection
Crop detection and assessment
Watering and spreading fertilizer
Land preparation and soil analysis

Picking, harvesting, and planting

As the term suggests, picking and harvesting involve plucking fruits, vegetables, flowers, etc. The items or produce picked are then placed in a basket or container. This entire process can be automated using pick and place robots. Here, cameras help to identify the exact coordinates of fruits and vegetables, and capture images for identifying ripeness, size, and type of the produce. AMRs can also be used for planting purposes.

Weed and bug detection

This involves detecting harmful weeds and bugs in order to precisely spread fertilizers. Embedded cameras used in such robots need to deliver clear images even in challenging conditions for the AI algorithm to accurately detect weeds and bugs. Robots that perform these tasks are typically called automated weeders.

Crop detection and assessment

This involves activities like measuring NDVI (Normalized Difference Vegetation Index). NDVI measurement helps in assessing crop growth, density, presence of water & soil, etc. This requires a camera that comes with high NIR (near infrared) sensitivity. This is because, NDVI is calculated by measuring the difference between near-infrared (which is reflected by vegetation) and red light (which is absorbed by vegetation).

Watering and spreading fertilizer

Based on the data collected on weeds, bugs, plants, etc., agricultural robots are used to precisely water crops and spread fertilizers on them. Here, the quality of images plays a significant role since the accuracy of detection by the AI algorithm will heavily depend on the details captured by the camera.

Land preparation and soil analysis

Agricultural robots equipped with cameras also help in land preparation and analyzing soil texture, water content, soil type, etc. These can be done using the same robot or two different robots. Cameras are used to capture visible or NIR images of the soil to enable this.

It can be noted that most agricultural robots are multi-purpose in nature. This means that many of them perform two or more tasks mentioned above.

Camera-enabled autonomous navigation in agricultural robots

In addition to the specific agricultural tasks mentioned above, one of the most important (if not the most important) tasks performed by a camera is depth sensing and obstacle detection. For this, robots use 3D depth cameras to measure depth for easy navigation and avoiding accidents.

Agricultural drones

Drones used in agriculture perform similar tasks as robots with the key difference being their ability to cover a wide area leveraging aerial view. For instance, drones with high-quality cameras that can capture images while they are in motion can cover tens of thousands of acres in a single day! They can be used to monitor granular details of crops, such as exact plant growth, vegetable color, field row spacing, flower count, etc.

Autonomous tractors

Autonomous tractors are also a type of auto farming equipment. Like robots, they operate completely autonomously. They can automate functions like plowing, harvesting, and capturing crop details. Cameras play a part in helping these tractors in depth sensing, obstacle detection, and image capture.

To learn more about how embedded vision is transforming agriculture using autonomous tractors, please read the article How embedded vision is contributing to the AI revolution in autonomous tractors.

Why camera selection is a crucial process in agricultural robots

Now that we have understood various agricultural applications where cameras are used, let us look at why camera selection and evaluation are critical to ensuring the reliable performance of agricultural robots.

Selecting the right camera solution can be a grueling process. Remember that these applications operate in unpredictable outdoor environments where the lighting can vary without warning. Therefore, capturing high-quality images is no easy task. These robots and equipment also come in large sizes which require the cameras to be installed at a large distance from the host processor. Furthermore, these applications need cutting-edge AI processors that are capable of running machine and deep learning algorithms for accurate data analysis.

Some of the critical features required for a camera used in agricultural robots are:

Flexible cabling
Shutter type and frame rate
High dynamic range
Multi-camera synchronization and external trigger support
NIR sensitivity
Enclosures for use in harsh environments

Why e-con Systems is the camera partner of choice for your agricultural robots

e-con Systems offers state-of-the-art embedded cameras to empower modern agriculture applications, whether to detect bugs/weeds, evaluate crop health, monitor seasonal growth, or simply automate your entire harvesting process. e-con’s wide portfolio includes cameras with features such as high dynamic range, global shutter, GMSL2 and FPD Link interfaces for long cable support, multi-camera synchronization, external trigger support, NIR sensitivity, and IP-rated enclosures. We also offer a 3D time of flight camera that offers an accuracy of less than 1% while delivering both the depth and RGB data in a single frame. In addition, we also recently launched NVIDIA Jetson AGX Orin cameras that are suitable for agricultural robots.

Other pertinent features of e-con’s cameras include

Lightweight, versatile, and optimized for power consumption
Inbuilt image signal processor for auto exposure and auto white balance functions
High signal to noise ratio for image output with less noise

Below is a list of e-con cameras suitable for various agricultural applications:

e-con Systems also offers extensive customization services to ensure its cameras meet the specific requirements of your agricultural robots, vehicles, or drones. We can help you choose the right sensor, interface, optics, and processing platform to make sure your agricultural equipment meet the desired performance standards.

We have worked with numerous agritech customers where we helped them smoothly integrate our cameras into their device.

See how e-con Systems helped a large US-based auto farming solution provider to automate crop yield assessment.

Looking for help in choosing the right camera for your auto farming device? Please write to [email protected]. Our camera experts will be happy to help you. You could also visit the Camera Selector to have a look at our complete portfolio of cameras.

Gomathi Sankar
Head of the Industrial Camera Business Unit, e-con Systems

If you're building AI or vision-enabled products, you've come to the right place.