It was, after clustering and optimization of cultures
Collective farms and agricultural holdings in Russia are almost not automated. And there, on an almost level place with minimal costs, you can get up to 10% increase in profitability due to the choice of the optimal portfolio of crops grown, the exact distribution of equipment for work and generally normal planning. We came to several objects and carried out calculations for them, which I will tell you about now.
Three fundamental questions were formulated:
- What proportions do you need to grow and where to earn more?
- When what equipment and where will it work?
- What should be in the fleet so that there are no risks of disruptions in the timing of agricultural operations or high costs for hiring?
We solved all these problems, and there are a lot of interesting features. We will discuss the abstract collective farm “The Way of Communism”, located in a random place (we just liked the fields on the satellite map), because I can’t name the real customers yet.
In such places, of course, rational agents act. But sometimes there is a drinking agronomist, sometimes a kosyachnik-machine operator and other recognizable characters from real life come across. We are waiting for hail, a broken harvester and other adventures. And now we will go into it all with our automation.
Why are we asking such questions?
- What proportions do you need to grow and where to earn more?
It is important to optimize agricultural crops for the fields, that is, how much crop to grow, in which area which of these crops grows best taking into account the type of soil, on which field that grew in past years, which routes will be more convenient to harvest for harvesting and plus you still need to remember take into account a large number of restrictions. Now the answers to these questions are formed by a very laborious multi-stage process in Excel without taking into account historical data, but with a high level of managers' subjectivity.
- When what equipment and where will work?
The cultures were distributed across the fields, and there is a need now to distribute the equipment among them. And every day, line managers solve the following problem: to distribute X techniques on Y fields to perform Z types of operations on W crops. And you also have to consider if you managed to complete all the operations yesterday on time, and if you didn’t, what share? .. Doesn’t it seem like too many factors for the process that is being solved in Excel? But at the same time, this problem is daily solved in all agricultural holdings and collective farms, relying on experience and sometimes using rough assumptions, neglecting the optimality of the final result.
- What should be in the equipment park so that there are no risks of disruption in the timing of agricultural operations or high costs for hiring?
Often, a technique is a bottleneck in processes, so it is very important to have an optimal fleet of equipment or a plan for its formation for the coming years. But in order to get this coveted “optimal technology park”, it is first necessary to solve the problem described above correctly, without assumptions, and on the basis of these work schedules to assess the possible risks and form possible equipment options that would reduce the likelihood of overstepping the agricultural level or the level of irrational hiring, or to avoid them altogether.
With questions sorted out. Let's move on to how we looked for answers to them.
What are we growing and where?
So, we arrived at the “Way of Communism”. Immediately regretted his NB. They are not comme il faut at all, here the boots decide. They asked us to slightly shift the planned “tea party” for the evening, but while the working day was in full swing, we immediately introduced us to the agronomist who is engaged in the formation of the distribution of crops by field. We first went to him.
You, of course, have a good idea of ​​the ATX case, cheerfully crunching with a hard disk, as in childhood, for which agronomist Vasily works. He has an exel open, in which a frightening table of all fields is expanded. In total there are 300 fields.
It contains notes on which field what type of soil and detailed indicators of its soil quality (soil quality is such an integral assessment of soil productivity), plus to this - what crop grew last year on this field, what productivity was on it and etc. And in a good way it is necessary to distribute the cultures considering all this.
Why is this so important? Because there is a sea of ​​peculiarities and nuances, for example, different cultures change the soil properties in different ways and sometimes the soil is allowed to “rest” by sowing it with steam (in fact, this means sowing with nothing). Fertilizers and pesticides also significantly change the soil tolerance, and you need to wait for the neutralization of some elements, and there are many, many other interesting things that were documented back in the USSR in the form of a rigid set of rules, but the fundamental rules are generally several centuries, if not millennia. If the agronomist Vasily simplifies his life and distributes the cultures randomly across the fields, then the “Way of Communism” will suffer SUCH losses due to the significantly low productivity in most of the fields, and in some places a complete crop failure that his beautiful cap will fly off his curly head.
And we began to understand:
- What data the agronomist usually relies on.
- What additionally will we still need.
- Where do the legs grow from all this data in terms of which systems they come from.
- How elegantly this process of distribution of cultures can be automated in order to minimize distract workaholes with our smart, reasonable optimization scripts, but at the same time fit them into everyday work processes.
Fortunately, in the “Way of Communism” automation began at zero, and we already had enough accumulated data to solve a number of our problems, namely, this is five years of history. At one of the other real objects, we were incredibly lucky, and there was paper for about a 30-year period, which, of course, we are not ready to digitize yet, and there is no need.
In addition to data on the history of the fields and their condition, we also needed additional restrictions arising from the features of the business process “The Way of Communism”, for example, whether it is necessary to take into account the needs of the market or other groups of enterprises. Here is an example of such a restriction - “We must have at least 50% of winter wheat, because we are in the export zone of production. At the same time, there are vertical links with oil and fat plants, and we need sunflower at least 10% of the total volume. We devote no more than 5% to the cultivation of chickpeas and other niche legumes. ”
Taking into the work all the information received, we went to think. Having returned a couple of weeks with the first result to agronomist Vasily, we agreed with him on a number of additional restrictions that we ourselves identified on the basis of these five years of history. For example, that for two crops, additional additional dysfunctional predecessor cultures (those that grew in previous years) were identified that underestimated yield and discussed with him the moment of whether we are ready to remove several crop rotation from the simulation (crop rotation is such a scientific reasonable alternation of crops and vapors in time and in the territory) in which such pairs-sequences of crops are present. And, of course, they showed him our first optimization results.
Our optimization is based on a model that allows us to create an optimal strategic scenario for seven years, balancing factors such as crop profitability, sales constraints, fleet limitations, permissible crop rotation by field and taking into account the specifics of each field, storage and logistics restrictions for them, as well as introduced a new concept as a "cluster of fields" and worked with optimization already with it.
Having looked at our dashboards with the results, the agronomist Vasily said that the result “looks like the truth.” But then questions and doubts rained down:
- But you made a plan for seven years, but if we decide to buy some of the fields next year?
- But we are thinking of reducing the production of wheat and sowing part of the beet fields, because we still believe that the sugar plant will finally be launched?
- What to do if suddenly there is a drought in a season?
- And indeed, there is no time for us to look at your numbers in the season! Fields are calling!
All questions and comments were taken into account, but in addition to Vasily, there is also a supplier of equipment, a financial director and many other people who determine the future budget and the future of lands. With each run of results, someone new was connected and we learned about more and more nuances and limitations that should be taken into account. But in one of the results run, we lost our Vasily. As it turned out later, he had a “home office” due to poor health caused by a protracted depression amid stopped field work associated with weekly rains.
As a result, the version of the optimization script that was close to the finish line was called “final_666_13_2019”, because any differences constantly surfaced or there were so many conditions that there was no optimal solution and I had to figure out how to take everything into account.
As a result, a tool was formed that works as follows:
- There is an interface where the user can first view the values ​​of all the constraints input into optimization. If necessary, change them and start the recount.
- Next, the optimization model is launched.
- Optimization results are loaded into desks.
Thus, the agronomist user can independently create different optimization scenarios and then make a comparison of several in the reports already prepared. But when only such projects are implemented in agricultural holdings or collective farms, for the first time we are the operators of such a system in the "question = answer" mode.
Operational planning
A long-term crop rotation is agreed, everyone is happy or should be happy (this is an order from the director). Now you need to make a schedule of work on each field.
That is, we have a set of sequence of operations for each culture and 300 fields with cultures already fixed on them as a result of our optimization. It is necessary to distribute the set of available vehicles with these introductory data for these operations, taking into account the changing weather and preferably, naturally, minimizing the amount of hired equipment.
On the fingers it looks something like this. First, you need to make a daily plan for each field with the maximum production of equipment for each unit for the entire season, in order to determine which equipment is most lacking and in which operations. Analyze this moment, think about how to balance the technique. Balance. Then re-do the daily chart for each field and make sure that it has become really more optimal than the previous one. And if a hail suddenly passed in a season and killed a crop on a dozen fields, then naturally you don’t need to drive equipment to these fields, that is ... you need to recalculate the daily plan for the rest of the season in all fields!
Schedule of operations and necessary / used equipment
In real life, linear managers are involved in such tasks. And if you imagine line managers as the heroes of Zvyagintsev's film Leviathan, then I dare to upset you - that’s not so! At least in our "Way of Communism." But Russia is a big country and you can plunge into the “Leviathan” with interest. Checked. But still, for us, the people sitting in the metropolis, the daily routine of the workers of our “Way of Communism” will be wild. How could it be without these hour-two-hour routes from home to office and back? We can’t even think about the possibility of getting to the house by bike on lunch, have time to eat and watch “Overview. Extraordinary events ”or“ SashaTanya ”, and then again return“ to the fields ”!
And there are about 30 fields per line manager. And, of course, there is no end-to-end planning between these line managers, to say nothing of the fact that it’s unrealistic to even think about the optimal schedule, not to do it. The very dimension of the problem does not allow it to be solved optimally by the forces of several people.
You already roughly understand that in agriculture it is not so simple - an ultrafactorial sphere with a difficult to predict outcome, and everything categorically depends on the market, weather and even a bunch of external factors. This means not just uncertainty, but uncertainty of degree n. In this chaos, it is wildly difficult to formulate something.
Operations work
But we were planning a technology park, remember? There was a long-term calculation by type in advance: we see that there are distortions in the distribution, we make balancing, we transfer three tractors of the sixth series from one unit to another. Someone is cutting a tractor with incomplete disposal. We will have deficits everywhere, but less acute, without failures. The “path of communism” is somewhat shocked, because we untie the equipment from the units and give the optimal distribution by these movements. This is not changing the structural park, but you can still take into account the receipt of new models and the rental of cleaning equipment and trucks (in fact, there is no conditional rental market for the equipment, except for these two types).
Hitch Data
It is necessary to take into account the probability of rain: these days the equipment does not work. It is necessary to take into account the real availability of equipment: something broke, something did not have time to repair. Knowing the real availability, we are building a growing front of work. We are building rolling operational planning. We write down the superdetailed level of tasks: in this field on such and such a day, such and such kind of operations are specifically by these units.
Then we make a schedule for the next day the night before it. Before us, this was solved by the method of "who shouts louder, he gets the equipment" and more like a panic. Everyone used greedy algorithms from their heads, not really thinking about changes in the front of work in future days.
Pea routing example
Total
We have an optimal model of what to plant, where to plant and what to do with all this, there is a distribution of equipment, there is a daily work plan that changes from the situation. The result of our decision is a 10% increase in gross margin. It is clear that the outcome will largely depend on the specific set of input for the task, fleet, fields, but I would say that from 2 to 15% you can definitely play purely on optimizing crop rotation and better utilization of equipment. Well, get a tool for more or less informed decision-making, which is important when people argue among themselves. The predictions “what if” help a lot at the stage of approvals.
At the input from different agricultural holdings, there were different data formats. We cited them for our model in the following directories:
- Cultures: technological maps for each culture, where there is a list of operations and agronomic terms for implementation.
- Technology Reference: how many units.
- Types of operations and permissible couplings.
- Data on fields with agro-restrictions, about which I spoke at the very beginning.
In fact, our headache in minor manual operations was just the unification of the process. The transition from intuitive solutions to numerically evaluated is very necessary, but the lack of any clear methodologies for solving details and unified description formats for the same technique is very holding back. In terms of data, we have achieved a full-fledged process, part has not yet been fully completed, but in general, decisions have already been made that are quite optimized. As a result, we come to agricultural holdings with an analytical platform that provides a set of hypotheses. A digital double is being built. If the holding is ready or is already moving from “extinguishing fires” to management through planning, then they are usually interested and asked. Further, seeing the forecast for several fields and comparing with their own, they believe.
If you are interested in looking at a product or if you suddenly want to tell about it to familiar agronomists - please write to my mail Alexander.Efimov@sas.com.
Before the start of projects with agriculture, I knew that everything was not just in it, but I had no idea how much, and how much there could actually be any data science there, but real mathematics. Now I can confidently say: it is surprising that we have not reached it yet with data analysis methods. Because the topic is big, very important and surprisingly well optimizable.