Machine Learning and Data Engineering Applications in Agriculture

Machine Learning and Data Engineering Applications in Agriculture

“In God we trust, all others bring data.” William Edwards Deming

In this article, we show the importance of interference between Machine Learning, Data Engineering, and Agriculture. The main problems that farmers face on an everyday basis, such as climate changes, manual time-consuming tasks, and changes in consumers’ everyday dietary are presented in the article. Along with possible solutions that Machine Learning and Data Engineering can offer by applying such tools as IoT, data collection, computer vision, blockchain, and Augmented Reality.

Impact of cultural transformations on Productivity in Agriculture 

Over the past centuries, human beings have gone through certain transformations that led to agricultural revolutions afterward. Each revolution played a key role in the increase of production and optimization of the production process. A long way from the first revolution to the fourth traces a path from hunting and gathering to digitalization and artificial intelligence. Let’s dive into details, there are four agricultural revolutions:

1. First agricultural revolution (1700- onwards). This revolution is characterized by stationary farming with core principles mainly based on manual labor, horsepower, and simple tools, which means that productivity remained relatively low during that time period.
2. Second agricultural revolution (1914-1980s). At this time happens a shift from natural nitrogen supplementation to synthetic fertilizers. The introduction of crop rotation and drainage dramatically increased crop and livestock yields, improved soil fertility, and reduced fallow. An increase in production and a decrease in labor demand led to migration and urban expansion.
3. Third agricultural revolution (1920s-present). This revolution is all about combustion engines and rural electrification, the introduction of biotechnology, and genetic engineering, alongside computerized programs. The result of this revolution we could observe nowadays in the markets.
4. Fourth agricultural revolution (the 1970s onwards). This revolution made a transition from industrial production to a digital model that optimizes production processes, reduces time and cost, and enhances customer value. At this point, the agricultural industry started to operate with keys like the Internet of Things, Big Data, Artificial Intelligence, cloud computing, remote sensing, ingestion, and processing of big data into Data Lakes as a foundation for the decision-making approach. 

The first and Second agricultural revolutions show a transition from manual labor to production, while the third and fourth revolutions show the importance of computerization and the collection of data. Nonetheless, there are still enormous problems in the agricultural sector of economics that seek solutions in Machine Learning.

Problems of traditional agriculture and the Role of Machine Learning  

Demand for agricultural production has highly increased over the last year and ought to be one of the major causes of inflation all over the world in the nearest future. In the meantime, intensive agricultural yield increase is limited due to a number of external reasons: 

• A limited global land surface that is suitable for cultivation based on climate conditions, good soil, and urban development. According to up-to-date statistics, approximately 40% of the land is covered by jungles, deserts, urban places, or other natural land states such as forests. So, very little land is left for agricultural expansion. 
• Constantly changing consumer dietary habits and patterns push farmers to shift from one type of production to another. For example, demand for meat products is rising rapidly in societies due to inequality among the population. 
• Climate changes and natural disasters that have increased in the last century are likely to result in more extreme weather patterns, with average temperatures increasing, resulting in fluctuating yields and production shortfalls. 

The application of Machine Learning in the agricultural sector can smooth the above-mentioned problems. In order to better understand the interference between these two fields, let’s look at the example. Let’s imagine that there is a farmer who relies mainly on calculations of input efforts and output yields. This farmer forecasts his/her profit based on scientific calculations. At the same time, he/she operates with data from sensors on the machinery, such as crop and GPS data, whilst other data is retrieved from a drone, and correlated against GIS information. At the same time, this farmer can also observe pricing data, livestock position, and demand for his product based on third-party cloud servers. All this together creates a picture of the potential crop value and demand. Also, weather forecast comes from open sources and clearly predicts conditions for the upcoming week. At the same time, sensors detect moisture in the ground and check the health status of plants and animals. All this data is gathered and stored for future analysis.

Tracking of every product entity is simple and can be clearly observed on the dashboards for the farmer and the final customer. Combining all this information together could save a lot of effort, help organizations work more efficiently, and solve the main problems that farmers face nowadays.

Integration of Machine Learning and Data Engineering into Agriculture

The covid-19 pandemic had a huge impact on Agriculture and at the same time on the development of Machine Learning and Data Engineering. The main goal of the integration between Agriculture and Machine Learning — is to increase final crop yield, save efforts and resources, and help control every step of plants’ and animals’ growth. There is a number of techniques in Machine Learning that help collect agricultural data and ease the process for farmers. The following applications could be of great use for the development of communication between data collection and actual production.

1) A unified protocol

It is useful to have a single, unified protocol for cross-manufacturer compatibility of electric and electronic components. All mechanical and automotive devices have to be combined like a “LEGO” piece into one big machine. All parts of the final construction should communicate with others via protocols. This unified protocol is based on the International Standard ISO 11783 and started to be applied all over the world in 2008.

2) Internet of Things (IoT)

Different devices in a system have to be connected to the Internet and can interact with one another in real-time. The number of sensors and their application is growing every year and will probably be around 250 billion in the next 5 years. According to this incredible number of sensors’ development of software products, it becomes a non-trivial task to collect and store information in a single place.

3) Drones and remote sensing

Development in information technology and agricultural science has made it possible to merge drones and Sensing, leading to the rise of precision farming. Such a scheme brings maximum profit and production with minimum input and optimal use of resources. One of the interesting applications is based on the global positioning system (GPS) and GIS technologies that help calculate optimal paths for tractors. With Machine Learning Algorithms challenging trigonometry task from university is transformed into simple solution and real money that wasn’t spent on extra fuel.

4) Data collection and social network communication

The key to this is the creation of an efficient chain with local food production systems and livestock systems. This approach will create a greater understanding of the entire food supply chain efficiency and the integration of these two systems will generate long-term positive environmental impact and will deliver greater food security.

5) Computer vision

Image analysis and detection are among the most intensively growing fields in informatics research. All automated machines start from sensing, most often using cameras to obtain data that provides information about the crop and location of the harvesting system. Typically, this is an RGB camera, depth camera, or lidar system. Images are passed to machine learning pipelines that are based on effective classification approaches, including support vector machines(SVMs), neural networks, k-means, principal component analysis (PCA), feature extraction, etc. Among applications that were developed by different teams, the following could be mentioned:

1. Plant disease identification. Traditionally, farmers are not very well informed on early-stage disease detection as they lack knowledge about crop diseases and require support and suggestions from specialists. However, diagnosis of infections in the early stages could save a lot of crops in the end. 
2. Fruit sorting and classification. At the market, fruits are sorted by size and appropriate price. Usually, these fruits are sorted out manually, which is biased and time-consuming. The process that could be characterized above is among the first to be automatic.
3. Crop and land assessment. Information that is gathered by satellites has been increased through the use of image sensors. For example, the areas of growing or disappearing forests could be collected and analyzed in time series to detect potential problems.
4. Weed recognition. All plants are started from weeds. Hence, weed detection, collection, and filtering of the best ones are extremely important for future agricultural yield. Researchers proposed image processing for analyzing agricultural parameters and describing how image processing on different spectrums, such as infrared and hyperspectral X-ray can be useful in determining the vegetation indices, canopy measurement, irrigated land mapping, and more.

6) Data transparency and blockchain

A blockchain is a method of encrypted data that conducts a search for every single transformation that has been applied to a target entity such as storing, linking, and recovering. The modern agricultural industry has accelerated and now uses a blockchain in the agriculture value chain because it is seen as a mechanism for optimizing different issues, such as transparency, cost-effectiveness, traceability, quality supply systems, etc. For example, the French food market “Carrefour” has been using blockchain solutions for the traceability of its products since 2018. The aim is to provide a QR code scan where consumers can retrieve data on the product on their mobile phones. The information available through the code includes the place and date of production, the product’s composition, the method of cultivation and etc.

7) Augmented Reality

This field is only at the development stage but has already demonstrated high potential in a specific field that requires 3D image resolutions. Visualization of animals, their diseases, and crops’ damages in order to assess and carry out treatment. Augmented Reality promises a lot in the near future, especially if it is combined with Artificial Intelligence (AI).

The importance of Machine Learning and Data Engineering in Agriculture should not be underestimated. Implementation of new techniques could definitely benefit farmers by increasing revenue and customers by saving them time.

Conclusion

The evolution of human transformations in agriculture outlines the main changes that the agricultural sector historically went through and at the same time points out problems that farmers face nowadays. In the era of computers and digital communication, farmers seek ways to increase profit, while consumers demand quality service in a short amount of time. In order to satisfy these needs, the Sigma Software group researched and developed several approaches, such as data communication, computer vision, blockchain, augmented intelligence and etc. These approaches along with other tools of Machine Learning and Data Engineering create a core outline for the agricultural sector. As a result, in order to increase revenues in the agricultural sector and create a more efficient chain supply, farmers need to rely more on machinery.

About the author

This article was written by Ihor Oleinich, Data Engineer at Sigma Software Group. With over 4 years of software development experience, he excels in Python, SQL, and data processing. Also Igor has 10+ years of experience with Data Cleaning, Transformation and Processing. Familiar with data architecture including data ingestion pipeline design, data modelling, machine learning and advanced data processing.