SviMik is a modest freelancer from Estonia who wants to keep his incognito. He writes code for PCs and microcontrollers. However, in OSM he is the recognized king of bulk imports. That's what we talked with him about: why the community is afraid of imports, how best to carry it out, and what should not be done in any case - he told all this in an interview.
- How did you find out about OpenStreetMap? How did he “hook you”? Why did you start mapping?
- I don’t remember how I learned about OSM. It was registered in 2007, but began actively mapping at the end of 2012, and in 2013 decided to organize mass import from the Estonian cadastre. The OSMAnd mobile app pushed me to this. When I installed it, I saw that the map of Estonia is not accurate enough. And I began to edit the map for myself, so that I could find any addresses in the navigator and come where I needed.
At some point, I realized that if I manually draw houses and put down their addresses, I will never complete this process. Then, as befits a programmer, I began to look for ways in which this could be accelerated and automated. Our Estonian cadastre helped me in solving the problem, which by that time had not only posted its information in the form of open data, but even gave written permission for use in OSM. To organize imports in such favorable conditions is a matter of technology, which I did.
- Why did the choice fall on OSMAnd?
- This is now a lot of navigation applications and everywhere there is mobile Internet. Then everything was a little different. Firstly, OSMAnd attracted by the fact that it worked offline. Secondly, the ability to correct the card if something was wrong.
- Tell us about the OSM community in Estonia?
- We can say that there is no community. Only a few people in the country are regularly active. Despite the fact that we have an inventory, people are not in a hurry to enter this data into OSM. Our community last met in 2013 when the State of the Map Baltics conference was held in Tartu. But in fairness it should be noted that there were more foreigners on it than local ones. I think that this is due to the fact that we do not have such a large community and we are not so interesting to each other to go somewhere and meet someone.
- Why are the Estonian authorities so boldly sharing data?
- I do not know. Ask them yourself :)
- How do Estonian authorities understand the potential of OSM? Do they use it themselves?
- It is difficult to say for the whole country, since I have not specifically studied this issue. But I saw that, for example, in Tallinn, the city map of public transport was made on the basis of OSM. They even correctly indicated attribution.
- With what help do Estonians orient themselves?
- I think that 99% use Google services. This is already a certain standard and it does not occur to anyone to use something else. Mobile Internet is everywhere, and therefore few people need offline maps. At least in Estonia itself. An exception may be trips abroad, where there is a risk of being left without a connection - in such cases, applications that use OSM data really come to the rescue and come first.
- How do you see the Russian OSM community from the outside?
“You have people who draw a map with their hands.” In Estonia, 70% of OSM data is imports. You have everything alive. We have the planet Shelezyak, inhabited by robots, on which there is no life. Why? Because there is no community. It's cool that you have a community that does something that really maps. Appreciate it.
But you are out of luck with your inventory. Do not despair, someday and he will open his data. Moreover, I heard that in some Russian cities, administrations are opening their GIS portals for OSM. It's also good. But understand that creating and joining a vibrant community is harder than revealing data. If there is no open data, then you can take it from the street - you have people for this. But if we hadn’t had open data, then we would never have drawn with our hands even a tenth of what we have.
- Why mapap hands - is it important?
- Any data set, even the Estonian cadastre, is limited by itself. Yes, I can import the contours of houses and their addresses, but who will draw the lawns, trees and roads around them? Who will put down the POI in the end? By the way, I can take the roads from the cadastre, but we all at OSM remember the infamous US road import - TIGER . In addition, importing roads is already technically very difficult. The script must somehow be able to connect new roads with existing ones. I don’t yet know how to implement this. Therefore, not everything can be imported, even if you really want to. Something can only be drawn with hands, but this requires people, and for them to be - you need a community.
- What is the import organization algorithm in OSM?
- It is detailed in the article Imports / Guidelines in WikiOSM. But in short, it looks like this.
First, you need to get permission from the data owner to download them to OSM. Most problems arise already at this stage - there is no resolution. Many believe that if they saw something on the network, then they can take it and dispose of it at their discretion. No, you can’t do that. And you cannot upload such data to OSM. Personal permission needed. Secondly, it is necessary to coordinate mass import with the OSM community and get approval from it. If these two points are successfully completed, then you create an import page on WikiOSM, on which you collect all the necessary information. And only after that you start importing.
When I started my import from the Estonian cadastre in 2013, I went through all these stages. Here, for example, is the wiki page of my import. From experience, I’ll say that writing a script is easier than getting the consent of the community, as there will always be someone who will be against it.
- In general, import into OSM is rather good or bad?
- As is customary in OSM, there are two opposing opinions on this score, and each side of the barricade has its own, impenetrable arguments. Someone thinks that imports scare away newcomers, since a person will open a map, see that everything has already been done before him and leave the project. Others, on the contrary, believe that imports are good, and they cite the story with the same abstract newcomer, but this time he is frightened by the empty map and the amount of work that he has to complete.
Speaking about my experience, I was interested in OSM quite different - the ability to edit exactly what interests me and use it in the future. That's why I came to the project, that's why I'm still in it. If you look at the example of Estonia, you can draw the following conclusion: there was an empty map - the community was small, there were a lot of imports - people also died. Maybe people in the community are attracted to something else? But I know for sure that after imports the map of Estonia became like a map. Is it bad?
Do you know what else you paid attention to? The community has some bias towards what bots do. People are allowed to make more mistakes. Bots have more stringent requirements. If the bot made a mistake, then everything is a failure. A bunch of posts write about this, and the example is immediately taken by the opponents of imports. Little is written about good imports; this is of no interest to anyone.
- Why are imports so critical?
- This attitude to imports did not appear out of the blue. Many did them carelessly, broke data, which caused kickbacks, and this is always a big problem. In my case, the import was relatively safe - I import only those houses that are not in OSM. When you insert an object outline into OSM from scratch and do not delete anything, it’s difficult to break something.
It is quite another thing when you import large objects, or many related objects, such as a road network. Someone corrected one, another did not, and what to do after this? Execution cannot be pardoned, that is, it is no longer possible to replace, and it is also impossible to consider this data good.
A similar story was with the import of Estonian forests. In the absence of the best, and this was long ago, when OSM just appeared, it was decided to import from CORINE, they say, there will be at least something. At the moment, the initiator of this import regrets this, because now it is not clear what to do with it and how to update. Somewhere already corrected by hands, somewhere - no. And the forests themselves have already changed in 15 years.
The community does not condemn him - at that time it really was a good decision. But as an example of “why it is not necessary to import”, this case continually pops up in conversations, especially when it comes to further updating the imported data.
- It turns out that Estonia is a country of mass imports in OSM?
- Exactly. Now in Estonia there are 827 thousand buildings, 67% of them are imported from the cadastre, and almost all of the addresses are from there. Of all the houses, 81% now have the correct address, and 73% also have a zip code. Our indices have also recently been integrated into inventory data. We managed to achieve such indicators in only one way - by mass import of data.
- If someone is interested in your import scripts, are you ready to share them?
- Ready, but you need to understand that they were written for a specific task, as a one-time script, because for other situations they will be useless. I recommend looking in the direction of Conflator - it was written later, and was immediately conceived as a universal tool.
SviMik'a import algorithm for the contours of buildings from the Estonian cadastre
- You praised our community. It is time to scold him. What would you change and do better in it?
- I can scold only your cadastre, which cannot open its data. I repeat, you have a community - this is a big plus. Moreover, I perceive the RU-OSM channel in Telegram not as a meeting place for Russians, but everyone who knows the Russian language. I met people from completely different countries, not only Russia. It is convenient that everyone can communicate and find a common language. And it’s also convenient because if I see the edit of a person who is in the Telegram chat, then I can immediately ask him why he did it and not otherwise, to clarify some nuances that I wouldn’t become through the OSM website write. It’s convenient to coordinate through Telegram, the main thing would be to coordinate whom. But this is me about Estonia.
- I saw that you have a device that can write a GPS track even underground. Tell us about it.
- This is a project of one person from Russia. Incidentally, he is on the OSM forum. He assembled such a box with a gyroscope that can work for 15-20 minutes without a GPS signal, which allows you to map such underground objects as parking lots or tunnels. I have already drawn some underground parking for the shopping center using this device. When there is also a signal from the satellite, it is more accurate than a regular GPS receiver - the deviation is only 1-2 meters. I note that this device is installed in the car. Walking with him will not work.
By the way, when I decided to purchase this box, I contacted its developer. It turned out that just a week later he was going on business to Tallinn. Therefore, we can say that I had free VIP delivery from the master. I did not expect this. Surprisingly, this thing, which can write a track underground, did not cause much interest in people. Almost no one knows about her.
The same box
“Do you map yourself?” Or are you only importing?
- I'm mapping. I use, of course, JOSM. Although I started with the Merkaartor editor, since I had a bias towards Java. Merkaartor, by the way, is written in C ++. But then I tried JOSM and could no longer leave it.
Most often I paint in Tallinn, because I understand that for me everything is not enough. In addition, I’m a sofa mapper - I draw on a satellite. And not according to Bing, but according to the orthophoto of the Estonian cadastre, which is simply of incredible quality and also under a free license. Many simply do not know about him, even Estonians. Try switching the layer. You just need to choose ESRI, which in Estonia coincides with the orthophoto of our cadastre - maybe you bought it, or maybe you got it for free, open data after all.
By the way, not so long ago I drew public transport parking in Tallinn. The public transport map I mentioned earlier helped me with this. On it, in real time, you can observe the movement of buses, trolleybuses and trams. Recording their tracks revealed the nuances such as trolleybus depots and places where buses parked.
Public transport tracks recorded via an online map
- Do you use OSM in your personal life or work?
- Yes, like a navigator. And I go to work. After importing, it became convenient: you drive in the desired address and go where necessary. Now there is almost no such thing that some address is not found.
- Your site has not only import tools. But also various validators and services. Tell us about them.
- Almost all of my validators are aimed at correcting errors detected during imports: incorrect addresses and postal codes, house outlines . But there is one, it seems to me, the most interesting. As already said, I realized that I could not import roads. Then I thought it was worthwhile to speed up the process of rendering them. Moreover, almost all the available buildings in the country are marked on our map. I wrote a script that finds buildings around which within a radius of 300 meters there is not a single road or path. Such places are automatically added to my table, which, as I noticed, people use. One person we have mapped hundreds of access roads along it. This has indirectly helped improve the Estonian road graph.
In addition, since 2012 I have been doing self-conversion of cards for OSMAnd. Basically, I do this for the Baltic states, but along the way still capture a small part of Russia. Every day I post map updates on my website. Why am I doing this? Then there was no OSM Live and new cards for OSMAnd had to wait a whole month. What if you need fresh cards today? Fortunately, thanks to the authors of OSMAnd, they then laid out a free utility for self-generating cards so that anyone could do it themselves. But the one who first realized the idea of a daily map update for OSMAnd was me.
- Do you have so many all kinds of interesting developments, are you working on something new?
“In the near future I plan to order myself a 360-camera and start making spherical street panoramas for Mapillary.” Because while in Estonia it’s really bad with them. I saw these only 3-4 tracks, the rest - pictures from car recorders. At Google, of course, everything was filmed, but they cannot be used to clarify OSM, and Yandex does not remove it from us. Even the Mirrors. While I am developing an external camera mount on the roof of the car. As will be ready - I will definitely tell you about the results.
Communication of Russian OpenStreetMap participants is in the Telegram chat room and on the forum .
There are also groups on social networks VKontakte , Facebook , but they mainly publish news.
→ Join OSM!
Previous interviews: Kirill Bondarenko , Artem Svetlov , Sergey Sinitsyn , Natalya Kozlovskaya , Victor Vyalichkin , Ivan aka BANO.notIT , Anton Belichkov , Elena Balashova , Ilya Zverev , Timofey Subbotin , Sergey Golubev .