Neuronomics. Idea for startup 2030
Five years ago, an idea struck me. I decided to simulate as much as the global economy in order to know exactly what dynamics to expect from stocks, bonds, foreign exchange rates, etc. There was only one small problem, the model of all this does not exist. And even worse, economists have argued for decades which of their longest econometric models describes these complex processes at least approximately correctly. Therefore, instead of trying to pull a fictitious (and of course insanely simplified) model on a many-sided reality, I embarked on a conceptually different way - modeling agents at the micro level so that in the exhausting struggle for survival, through trial and error they would come to an equilibrium state, which at the macro level is expressed in the dynamics of prices, rates, etc.
The result of my research is described below. For visualization, I recorded the movement of agents and each pixel in this video hides a neural network, i.e. the brain of an agent struggling to survive in competition with his own kind in the cruel world of profit and cleanliness.
Why should this be easily understood at all, for example, in the current disputes over the "trade wars". Some (mostly professional economists) say that tariffs ultimately harm the economy as a whole. Others think that the country will benefit.
There is nothing to check and prove on the facts, because no one can put an experiment, all else being equal, on the same economy. Whereas the method described below allows you to control the conditions and obtain evidence of the presence or absence of the benefits of a solution.
Content:
Despite the fact that a fairly good understanding of the basic principles of the functioning of the economy has been achieved, there are great difficulties in applying knowledge in practice.
For example, there are the following models (theories) regarding the principles of the formation of the exchange rate:
And this is not a complete list. Moreover, the same factors, for example, the growing deficit in different models, give the opposite direction in terms of their impact on the exchange rate. It would seem that it could be easier than looking at the dynamics of the budget deficit and comparing it with the exchange rate?
In practice, the difficulties of this approach are associated with the “combinatorial explosion” problem, when hundreds of multidirectional factors are in effect, and although the effect of each of them individually can still be somehow understood, their result is almost impossible to calculate. The situation is complicated by the fact that in real life it is impossible to put in a controlled experiment, fixing all the parameters except one.
You might think that the exchange rate is a very complicated thing, but it concerns the most basic and key concepts, such as inflation. This is the main parameter, which almost all central banks of the world are called to regulate. It would seem that thousands of theoretical scientists and practitioners had to understand the mechanism of this phenomenon a long time ago. However, a member of the board of directors of the Federal Reserve System (US Central Bank) Daniel Tarullo writes in 2017: " We still do not have a theory of inflationary dynamics that would be effective in real time ."
Meanwhile, inflation is one of the main grounds for the Fed's decisions to raise or lower the key rate, which has a powerful effect on the many trillion-dollar stock and bond markets around the world.
Thus, an understanding of the principles of macroeconomic functioning makes it possible to apply the correct models and, as a result, effectively predict the development of the situation.
To achieve this understanding, it is necessary to implement a large-scale model of the economy, which includes all the main factors and industries that exist in real life. The main difference between this model and the existing ones is in abandoning attempts to make predictions of obviously “correct” results. Instead, in this paper, we use the approach of the evolutionary struggle of agents at the micro level, when incorrect decisions lead to the elimination of agents from the gene pool, and those decisions that lead to a positive financial result are saved and successfully distributed using the genetic algorithm.
In other words, just as a real market economy develops, the proposed method implements the principles of competition and continuous development, rejects those who make ineffective decisions. And already looking at what agents have come to in this struggle, we will be able to judge what is an equilibrium state under given conditions.
To test the operability of the concept, i.e. evidence that agents, acting at the micro level, are in principle capable of reaching equilibrium states with only one goal - to survive, a very simplified (compared to the real economy) model was developed, described below.
This is a world inhabited by agents - i.e. residents, each of which is a separate neural network, which analyzes the surrounding reality and makes certain decisions. At the time of starting the program, neural networks are initialized with random values and only those who, by chance, “figure out” how to get resources and multiply transmit their genes to their descendants, similar to what happens in our wonderful objective reality.
Each of these residents receives a certain amount of money. The agent is also a resident of one of two countries: the conditional “USA” or “Eurozone”. Each country has its own currency: “dollar” and “euro”. Agents have the ability to produce one of two types of goods: “cheese” and “wine”.
In each country, ~ 50% of agents are “girls” and they know how to produce cheese well. They do it especially well if they have wine. They themselves also know how to produce wine, but it turns out badly for them. Those. each unit of goods has a measurement - quality (from 0 to 100%) and if the girls make cheese, but they don’t have wine, then the quality is 10%, but with a vinyl everything is 100%.
The second half of the agents are boys and theylie well on the couch making wine. Especially if the cheese is eaten before. And they make cheese badly, in general, everything about them is symmetrical with respect to the girls.
Goods tend to deteriorate over time, i.e. every day the quality of the products is falling. For life and for successful propagation, each agent must consume a product that it does not know how to produce. Those. if the agent produces cheese well, then it is most effective for him to sell cheese and buy wine. If you project the model on real life, then we can say that the baker makes good buns and successfully sells them, but for happiness he needs vodka, herring and a garage, because some buns can not do.
In addition, agents can trade currency. For example, an agent from Europe can sell the cheese produced (or even previously bought) for dollars, then exchange them for euros and buy food for them. All trading is carried out by bidding for auctions. The application indicates which product the agent wants to sell (or buy), the desired price, quality and delta that he is ready to move on. Agents make all these decisions based on commands from neural networks. Those. information about the world around, such as:
And the agent receives information about his own condition:
The neural network processes all the input information and submits decisions, such as: how much to produce, what price and for what currency to sell, at what rate to exchange one currency for another, etc.
If the agent acts efficiently, that is, it is capable of producing a product that it cannot produce and consumes it, then it receives “life force”. Maintaining any activity requires this vitality, including the ability to reproduce requires especially a lot of energy. If there are two agents of different sexes, then they can decide (remove the children from the screens) to have sex and give birth to another agent.
With each day lived it becomes harder to live (more vitality is taken) and when their supply runs out, the agent dies and transfers the accumulated supply of money to his heir.
The key fact in this experiment is that the number of dollars and euros in the system is strictly limited and the ratio of the total number of currencies is about 1 to 3. The agents themselves do not know about this. They have only a certain amount on hand and they have no idea how much money is circulating throughout the economy. Those. they are not told at what rate the money should be changed, and indeed they set all prices themselves. The ability of agents to come to an equilibrium course, which we know, owning the entirety of information, is 3 to 1, is a proof of the concept of the possibility of finding an effective solution without a teacher.
In other words, if agents find an equilibrium rate as a result of natural selection in a simplified model (two countries, two currencies, two goods), then it is likely that they can do the same in a model closer to the real world.
In the picture below we see how much the euro is walking around inside the Eurozone and how much the euro is going outside (i.e., in this case in the USA). We know the same thing about the dollar.
The rapid dynamics at first is connected with the implementation peculiarity (it was easier this way) and does not play a role. At this moment, the agents are only learning to walk and breed at least somehow. In the picture below we see the real ratio of the money supply of the two currencies (red line) and the average exchange rate (based on the results of all transactions for the day) that the agents formed.
It can be seen that the average transaction rate after some wanderings reached the target level. The target level is the red line, i.e. the real ratio of the volume of currencies and this information is not available to agents.
However, so far we have no confidence that this is not an accident. Change the conditions of the experiment, let the ratio of the amount of money be about 1 to 4.3 and see what happens in this case:
And again, the agents come to an equilibrium value. Now, looking at the graphs, it may seem that all this goes without saying, but in reality several years were spent on making the process begin to converge. In particular, it turned out that the number of agents plays a critical role, when there are relatively few of them, the process is much less stable, and to solve the problem we had to write a distributed application running on three PCs at the same time.
The main difficulty was that simply increasing the number of agents is not enough, the key component of success was the need for clustering. In other words, the development of not 100 (conditional number) agents in one common space turned out to be effective, but when they are divided into 10 “cities” of 10 pieces each. Those. Trading platforms remain global and any agent can trade with any other, but the genetic code evolves within its group. Only under this condition, the course began to converge to equilibrium.
Here are some of the most important parameters of the working version:
If it’s still not very clear why all this is needed, try to answer the following question - how will the eur / usd rate change if:
And now let’s take into account that there is the influence of China, Canada, Japan, etc. Of course, we can count this in the form of VAR regressions (which is done only without much success, see Tarullo's comment), however, firstly, this is an endless simplification of real processes, and secondly, the number of known combinations is less by orders of magnitude than potential ones. So you can imagine the quality of such a model.
An alternative to this is the simulation of real processes at the micro level with only more computing power. If it turns out that the results of many launches will converge to the same solution and this will be confirmed by observations in practice, then this will be an ineffably powerful tool for both investors and central bank regulators.
Obviously, much more modules are needed to moderate a macroeconomic system that is close to reality. It is necessary to develop an application architecture that can run in a distributed mode across the cluster (simultaneously on all nodes) and implement the following functionality:
Such a set of functions will need to be implemented for several major economies (USA, China, the Eurozone, etc.). Proportions in the number of agents, key parameters, ratios, etc. should be adjusted in accordance with the results of international studies (IMF, UN, etc.).
Key difficulty: implement fast and reliable distributed transaction and transaction algorithms. If you do everything sequentially, no resources will be enough for all the described functionality to work in an acceptable time. It is necessary to provide for high scalability, which allows connecting additional computing power "on the fly", as well as the simplest possible addition of functionality during the development progress.
And all this of course requires BigData technology. The prototype was written in C #, however today would choose the easily scalable SpringBootYarn + HBase. Actually, I think the main reason why this has not yet been realized is that it requires a huge amount of resources, in the sense of hardware + good expertise in cheap technologies like Hadoop. Well, the idea itself is also not trivial. At least, I have never seen anything like it anywhere. Taking this opportunity, I ask the collective mind to throw links if someone is familiar with work in this direction.
Naturally, if so far this has been kept on enthusiasm, in your free time, in order to make sure that the concept is working, then further development requires full-time and large financial investments. Honestly, I do not have faith in the multimillion-dollar investments that are necessary to move from PoC to a full-fledged model. When I started, I thought that it would require an order of magnitude less resources. However, I am convinced that someday this idea will certainly find its embodiment. At least, when the iron needed for such volumes of calculations will be much cheaper and it will become available not only to financial (and clumsy) giants, but also to enthusiasts like me.
Lyrical conclusion:
Money is all of course great, but there is some special pleasure in sitting on long winter evenings, looking at the swarming of these little creatures of my will. There is something pleasant about it, from the feeling of being a creator, albeit such a primitive, but still life. It is possible that this fleeting pixel tremor is a love dance, designed to show the little girl how generously gifted by life force our male is and what a profitable party he is for her. And I can’t even exclude that they feel in their being some kind of inexpressible pixel meaning ...
The result of my research is described below. For visualization, I recorded the movement of agents and each pixel in this video hides a neural network, i.e. the brain of an agent struggling to survive in competition with his own kind in the cruel world of profit and cleanliness.
Why should this be easily understood at all, for example, in the current disputes over the "trade wars". Some (mostly professional economists) say that tariffs ultimately harm the economy as a whole. Others think that the country will benefit.
There is nothing to check and prove on the facts, because no one can put an experiment, all else being equal, on the same economy. Whereas the method described below allows you to control the conditions and obtain evidence of the presence or absence of the benefits of a solution.
Content:
- General description of the concept
- Description of the implementation of the prototype (PoC)
- Development direction
1. General description of the concept
Despite the fact that a fairly good understanding of the basic principles of the functioning of the economy has been achieved, there are great difficulties in applying knowledge in practice.
For example, there are the following models (theories) regarding the principles of the formation of the exchange rate:
- Obstfeld-Rogoff Model
- Dornbusch Model
- Frenkel-Mousse Monetary Model
And this is not a complete list. Moreover, the same factors, for example, the growing deficit in different models, give the opposite direction in terms of their impact on the exchange rate. It would seem that it could be easier than looking at the dynamics of the budget deficit and comparing it with the exchange rate?
In practice, the difficulties of this approach are associated with the “combinatorial explosion” problem, when hundreds of multidirectional factors are in effect, and although the effect of each of them individually can still be somehow understood, their result is almost impossible to calculate. The situation is complicated by the fact that in real life it is impossible to put in a controlled experiment, fixing all the parameters except one.
You might think that the exchange rate is a very complicated thing, but it concerns the most basic and key concepts, such as inflation. This is the main parameter, which almost all central banks of the world are called to regulate. It would seem that thousands of theoretical scientists and practitioners had to understand the mechanism of this phenomenon a long time ago. However, a member of the board of directors of the Federal Reserve System (US Central Bank) Daniel Tarullo writes in 2017: " We still do not have a theory of inflationary dynamics that would be effective in real time ."
Meanwhile, inflation is one of the main grounds for the Fed's decisions to raise or lower the key rate, which has a powerful effect on the many trillion-dollar stock and bond markets around the world.
Thus, an understanding of the principles of macroeconomic functioning makes it possible to apply the correct models and, as a result, effectively predict the development of the situation.
To achieve this understanding, it is necessary to implement a large-scale model of the economy, which includes all the main factors and industries that exist in real life. The main difference between this model and the existing ones is in abandoning attempts to make predictions of obviously “correct” results. Instead, in this paper, we use the approach of the evolutionary struggle of agents at the micro level, when incorrect decisions lead to the elimination of agents from the gene pool, and those decisions that lead to a positive financial result are saved and successfully distributed using the genetic algorithm.
In other words, just as a real market economy develops, the proposed method implements the principles of competition and continuous development, rejects those who make ineffective decisions. And already looking at what agents have come to in this struggle, we will be able to judge what is an equilibrium state under given conditions.
2. Description of the available prototype
To test the operability of the concept, i.e. evidence that agents, acting at the micro level, are in principle capable of reaching equilibrium states with only one goal - to survive, a very simplified (compared to the real economy) model was developed, described below.
This is a world inhabited by agents - i.e. residents, each of which is a separate neural network, which analyzes the surrounding reality and makes certain decisions. At the time of starting the program, neural networks are initialized with random values and only those who, by chance, “figure out” how to get resources and multiply transmit their genes to their descendants, similar to what happens in our wonderful objective reality.
Each of these residents receives a certain amount of money. The agent is also a resident of one of two countries: the conditional “USA” or “Eurozone”. Each country has its own currency: “dollar” and “euro”. Agents have the ability to produce one of two types of goods: “cheese” and “wine”.
In each country, ~ 50% of agents are “girls” and they know how to produce cheese well. They do it especially well if they have wine. They themselves also know how to produce wine, but it turns out badly for them. Those. each unit of goods has a measurement - quality (from 0 to 100%) and if the girls make cheese, but they don’t have wine, then the quality is 10%, but with a vinyl everything is 100%.
The second half of the agents are boys and they
Goods tend to deteriorate over time, i.e. every day the quality of the products is falling. For life and for successful propagation, each agent must consume a product that it does not know how to produce. Those. if the agent produces cheese well, then it is most effective for him to sell cheese and buy wine. If you project the model on real life, then we can say that the baker makes good buns and successfully sells them, but for happiness he needs vodka, herring and a garage, because some buns can not do.
In addition, agents can trade currency. For example, an agent from Europe can sell the cheese produced (or even previously bought) for dollars, then exchange them for euros and buy food for them. All trading is carried out by bidding for auctions. The application indicates which product the agent wants to sell (or buy), the desired price, quality and delta that he is ready to move on. Agents make all these decisions based on commands from neural networks. Those. information about the world around, such as:
- exchange rates current for the current day (i.e. not the history of changes, but what rate was set during the last held session);
- the number of goods sold worldwide, the average price of a product and the average age.
And the agent receives information about his own condition:
- how much currency he has (and at what price he bought it)
- how much product, quality is the product, its cost, etc.
- own health
The neural network processes all the input information and submits decisions, such as: how much to produce, what price and for what currency to sell, at what rate to exchange one currency for another, etc.
If the agent acts efficiently, that is, it is capable of producing a product that it cannot produce and consumes it, then it receives “life force”. Maintaining any activity requires this vitality, including the ability to reproduce requires especially a lot of energy. If there are two agents of different sexes, then they can decide (remove the children from the screens) to have sex and give birth to another agent.
With each day lived it becomes harder to live (more vitality is taken) and when their supply runs out, the agent dies and transfers the accumulated supply of money to his heir.
The key fact in this experiment is that the number of dollars and euros in the system is strictly limited and the ratio of the total number of currencies is about 1 to 3. The agents themselves do not know about this. They have only a certain amount on hand and they have no idea how much money is circulating throughout the economy. Those. they are not told at what rate the money should be changed, and indeed they set all prices themselves. The ability of agents to come to an equilibrium course, which we know, owning the entirety of information, is 3 to 1, is a proof of the concept of the possibility of finding an effective solution without a teacher.
In other words, if agents find an equilibrium rate as a result of natural selection in a simplified model (two countries, two currencies, two goods), then it is likely that they can do the same in a model closer to the real world.
In the picture below we see how much the euro is walking around inside the Eurozone and how much the euro is going outside (i.e., in this case in the USA). We know the same thing about the dollar.
The rapid dynamics at first is connected with the implementation peculiarity (it was easier this way) and does not play a role. At this moment, the agents are only learning to walk and breed at least somehow. In the picture below we see the real ratio of the money supply of the two currencies (red line) and the average exchange rate (based on the results of all transactions for the day) that the agents formed.
It can be seen that the average transaction rate after some wanderings reached the target level. The target level is the red line, i.e. the real ratio of the volume of currencies and this information is not available to agents.
However, so far we have no confidence that this is not an accident. Change the conditions of the experiment, let the ratio of the amount of money be about 1 to 4.3 and see what happens in this case:
And again, the agents come to an equilibrium value. Now, looking at the graphs, it may seem that all this goes without saying, but in reality several years were spent on making the process begin to converge. In particular, it turned out that the number of agents plays a critical role, when there are relatively few of them, the process is much less stable, and to solve the problem we had to write a distributed application running on three PCs at the same time.
The main difficulty was that simply increasing the number of agents is not enough, the key component of success was the need for clustering. In other words, the development of not 100 (conditional number) agents in one common space turned out to be effective, but when they are divided into 10 “cities” of 10 pieces each. Those. Trading platforms remain global and any agent can trade with any other, but the genetic code evolves within its group. Only under this condition, the course began to converge to equilibrium.
Here are some of the most important parameters of the working version:
- 60 cities
- 5,000 inhabitants in the city
- 4 blocks of neural networks (reproduction, production, trade in goods, currency trading)
- neural networks are direct, two-layer, with regularization (there is a suspicion that LSTM will work better)
3. The direction of development
If it’s still not very clear why all this is needed, try to answer the following question - how will the eur / usd rate change if:
And now let’s take into account that there is the influence of China, Canada, Japan, etc. Of course, we can count this in the form of VAR regressions (which is done only without much success, see Tarullo's comment), however, firstly, this is an endless simplification of real processes, and secondly, the number of known combinations is less by orders of magnitude than potential ones. So you can imagine the quality of such a model.
An alternative to this is the simulation of real processes at the micro level with only more computing power. If it turns out that the results of many launches will converge to the same solution and this will be confirmed by observations in practice, then this will be an ineffably powerful tool for both investors and central bank regulators.
Obviously, much more modules are needed to moderate a macroeconomic system that is close to reality. It is necessary to develop an application architecture that can run in a distributed mode across the cluster (simultaneously on all nodes) and implement the following functionality:
- Agents are individuals. They can learn, that is, increase their productivity. They can get a job with agents: companies, banks or the state apparatus. They can shop, get sick, retire, make deposits or take loans. They can buy a stake in firms (companies), i.e. become shareholders.
- Agents - firms (companies) that hire individuals and pay them salaries. The manufactured product is marketed. They can make deposits or take loans. Able to pay taxes and evade them. Firms can produce goods and services, such as: food, clothing, transport, housing, oil, capital goods, entertainment, private medicine.
- Agents - banks, offer deposits and loans. They hire employees (agents of individuals), interact with the Central Bank.
- The Ministry of Finance - is managed directly (i.e. not through modeling by neural networks). Defines tax rules (VAT, personal income tax, social taxes, corporate income tax, export / import duties, etc.). Determining the level of expenditures on the spheres of healthcare, education, defense, and pension provision Issues government bonds. All these values are published by government agencies and introduced exogenously into the model.
- Central Bank - also exogenous parameters, such as operations to generate and sterilize the money supply. Key bid management.
- The state apparatus is a set of specific agents of individuals implementing public services (health, education, defense, law enforcement).
- The judicial system is the degree of chaos in the model. It is a set of coefficients that increase or decrease the risks to the business. For example, a coefficient of 0.1 means a 10% chance of losing the company as a result of a raider takeover. Or unfair tax assessment, etc.
- Venues for transactions with currency, stocks, bonds, goods, vacancies, financial services (loans / deposits).
- Auxiliary exogenous parameters: oil reserves, ease of doing business, average life time, etc.
Such a set of functions will need to be implemented for several major economies (USA, China, the Eurozone, etc.). Proportions in the number of agents, key parameters, ratios, etc. should be adjusted in accordance with the results of international studies (IMF, UN, etc.).
Key difficulty: implement fast and reliable distributed transaction and transaction algorithms. If you do everything sequentially, no resources will be enough for all the described functionality to work in an acceptable time. It is necessary to provide for high scalability, which allows connecting additional computing power "on the fly", as well as the simplest possible addition of functionality during the development progress.
And all this of course requires BigData technology. The prototype was written in C #, however today would choose the easily scalable SpringBootYarn + HBase. Actually, I think the main reason why this has not yet been realized is that it requires a huge amount of resources, in the sense of hardware + good expertise in cheap technologies like Hadoop. Well, the idea itself is also not trivial. At least, I have never seen anything like it anywhere. Taking this opportunity, I ask the collective mind to throw links if someone is familiar with work in this direction.
Naturally, if so far this has been kept on enthusiasm, in your free time, in order to make sure that the concept is working, then further development requires full-time and large financial investments. Honestly, I do not have faith in the multimillion-dollar investments that are necessary to move from PoC to a full-fledged model. When I started, I thought that it would require an order of magnitude less resources. However, I am convinced that someday this idea will certainly find its embodiment. At least, when the iron needed for such volumes of calculations will be much cheaper and it will become available not only to financial (and clumsy) giants, but also to enthusiasts like me.
Lyrical conclusion:
Money is all of course great, but there is some special pleasure in sitting on long winter evenings, looking at the swarming of these little creatures of my will. There is something pleasant about it, from the feeling of being a creator, albeit such a primitive, but still life. It is possible that this fleeting pixel tremor is a love dance, designed to show the little girl how generously gifted by life force our male is and what a profitable party he is for her. And I can’t even exclude that they feel in their being some kind of inexpressible pixel meaning ...
All Articles