Autonomous self-driving greenhouse tow-tractors with Cyberworks Robotics retrofit system based on Intel RealSense cameras
For Fernlea Flowers, implementing AGV tractors was key to providing a safe working environment for their employees while improving efficiencies and order fulfilment. Retrofitting these vehicles for autonomous driving ensures safe working distances can be maintained without sacrificing efficiency while minimising the risk of infection during the Covid-19 pandemic. The Retrofit Autonomous Vehicle System (RAVS) by Cyberworks Robotics features an Intel RealSense based depth camera from technology partner FRAMOS.
Self-driving systems have been widely used in intralogistics for transporting materials. Driverless transport vehicles move items around by towing or carrying them, for example from the warehouse to the assembly area. Various technologies have in the past been employed to control these vehicles, including ground-based guide systems with magnetic strips or inductive wires. However, using depth cameras with image processing technology to sense the environment in three dimensions and enable autonomous driving is a relatively new development. The field has advanced rapidly in recent years.
Cyberworks Robotics is an innovative company that develops and deploys technologies for reliable fully autonomous driving in mobile machinery and vehicles in mission-critical production environments. The company’s solutions are based on advanced and robust proprietary autonomous navigation software, fail-safe systems, smart cloud-based fleet management and stable hardware architectures enabling autonomy in electric tow-tugs, floor cleaners and wheelchairs. At the heart of the solution for retrofitting manually operated ground vehicles is the Intel RealSense D435i depth camera from FRAMOS for sensing the environment.
Automation in Agriculture and Horticulture
Farming is one of the many verticals that are benefiting from automation. The current smart farming trend shows that modern agriculture can be much more resource-efficient, productive, and sustainable than before. This includes using automation, robotics, and artificial intelligence to convert manually driven vehicles to autonomous operation. In addition, the current Covid-19 pandemic is forcing agricultural enterprises and large-scale nurseries to adhere to strict social distancing rules, which means the allocation of work and the activities performed by employees need to be re-assessed. The higher the level of automation, the easier it is to maintain required spacings for employees.
Cyberworks Robotics’ solution is ideal for horticultural businesses, where many employees perform manual tasks in close conditions. For example, plants are moved from one place to another with the aid of various electric towtugs, which are manually controlled by the operator. The Retrofit Autonomous Vehicle System (RAVS) by Cyberworks Robotics is a great way to convert these manually operated ground vehicles to AI controlled, driverless operation.
Fernlea Flowers Among the First Users to Retrofit Ground Vehicles
Fernlea Flowers is one of the largest greenhouse operators in North America. The horticultural sector is a key contributor to Canada’s economy in terms of exports and jobs. According to a statistical report by Agriculture and Agri-Food Canada (AAFC), in 2019 there were a total of 838 commercial greenhouse vegetable operators with 17.5 million square metres of production area, producing over 660,535 metric tonnes of vegetables. And this trend is rising.
Retrofitted electric tow-tug for autonomous self-driving operation
Yet the agricultural sector faces chronic labour shortages. Furthermore, in typical farm/greenhouse operations, drivers of tractor vehicles can account for approximately 20% of farm staffing. Automation of the operation and routing of such vehicles will decrease operating costs, increase production efficiency, alleviate staff shortages, and decrease worker density to improve compliance with current social distancing rules.
In the past, Fernlea Flowers used an inductive strip that was taped to the floor which outlined paths that the tractors follow through the various greenhouses. The challenge with this implementation was that it lacked the flexibility to dynamically adjust to the ever-changing conditions within the environment, while proving to be very unreliable at navigating the tractors precisely to their needed locations. To help address this issue, Cyberworks Robotics retrofitted 18 tow-tractors at the Fernlea Flowers location in Ontario, Canada. This represents the first deployment of autonomous tractors in an industrial greenhouse anywhere in the world.
Cyberworks Robotics Autonomous Retrofit Tow-Tugs Ready for Shipment to Customers
Sensing the Environment in Realtime with Intel RealSense Camera
Due to the constantly changing visual landscape, applying a self-driving technology in greenhouses has been challenging. What makes Cyberworks’ solution different is that it senses the changing environment in real-time. As a result, the electric tow-tugs can reliably avoid any obstacles or workers in their path of travel as they move about.
Part of what makes this solution work is the use of the Intel RealSense D435i depth camera as it provides the “eyes” of the tractor. The output of the camera is used to assess its surroundings and determine the best path through the greenhouse while avoiding any obstructions, both static and dynamic, that present themselves in its path, instantaneously. FRAMOS has proven to be the ideal partner due to its experience and knowledge of the Intel product technology.
Vivek Burhanpurkar, CEO of Cyberworks Robotics, on their reasoning for integrating the D435i into their self-driving technology says: “The two main factors that influenced our decision to use the D435i in our system were the small form factor and low cost. Also, implementation and development challenges were quickly addressed and supported by both FRAMOS and Intel , which made the process from design to production quicker and easier.”
Intel RealSense D435i depth camera from FRAMOS allows the vehicle to see
Success with Expertise and Vision Technology from FRAMOS
In the early phases of developing the self-driving platform, it was decided that a low-cost depth camera would be needed. It is for this reason that the Intel RealSense D435i was selected. But the engineers discovered that standard, off-the-shelf, USB cables, and hubs would not work well in the harsher environments that are common in greenhouses. More robust, locking cables and connectors, along with rugged USB hubs would need to be sourced to ensure that communication between the cameras and the onboard computers was never lost.
These components not only needed to withstand the vibrations and bumping that happens as the tractor travels inside the greenhouse and outside on the grounds between buildings but would also need to survive in changing conditions such as sun, heat, cold, rain, humidity, and dust. FRAMOS’s team of engineers worked in collaboration with Intel to identify key products to support their implementation and provided suggestions for their installation to minimise the stresses placed upon the vision system elements. As a result, all navigation issues were overcome, and the system’s reliability was assured.
As described by Sarah Wu, Director of Sales–Americas at FRAMOS, “Many times, our engineering support team is called upon to suggest stronger, more robust products and components that are not directly related to the products we sell but have an impact on the reliability, integration and performance of the overall vision system. Our team has a very broad knowledge of many technologies and, in most cases, can assist with overcoming challenges that vision designers face.”
Advantages of the Camera and Assistance from FRAMOS
- D435i provides low-cost depth data in a compact form factor
- Easy-to-use software for simplified integration
- Simplified cabling, power, and communication in one cable
- No additional infrastructure required to navigate
- Ability to work indoors and outdoors, over rugged terrain
- Dynamically adaptable to changing conditions and paths
- In-depth knowledge of vision systems and components, design and development
- In-depth knowledge of peripherals, accessories, and interconnectivity
Locate and Control Every Vehicle in the Whole Fleet
Cyberworks implemented a cloud-based fleet management solution for Fernlea Flowers’ 18+ tractors. It provides real-time vehicle data back to a central processing station (CPS) for analysis and deployment. The location and status of each tractor can be individually tracked wirelessly by the CPS, and updated instructions can be relayed back to the tractors for their next tasks. Coordinating the positions and paths for each tractor has also minimised logistical bottlenecks and traffic jams.
Conclusion
Cyberworks self-driving technology allowed Fernlea Flowers to retrofit their existing vehicles and enjoy all the benefits of AGVs including advanced fleet management, without having to change the working environment or infrastructure. Retrofitting the Cyberworks technology meant they only needed to reinvest in their fleet, which resulted in significant cost reductions when compared to equivalent human labour. In addition, coordinated fleet management has brought a marked increase in profitability as more orders are now fulfilled daily.
About FRAMOS
FRAMOS is a global supplier of imaging products, embedded vision technologies, custom solutions, and OEM services.
FRAMOS enables machines to see. Imaging and embedded vision technologies play a key role in automation, robotics, and the IoT-connected factory; they are key drivers in cognitive systems, the smart home, intelligent mobiles, and autonomous vehicles.
Vision experts at FRAMOS have been assisting customers since 1981 as a technical consultant, development partner, and as a supplier of individual components, complex system integrations, or customization options.
FRAMOS is based near Munich, with local offices in Canada, USA, UK, Italy and Croatia. With a team of more than 150 employees working worldwide, FRAMOS aims to find the fastest and most efficient imaging solutions for their customers.
For more information visit: www.framos.com
