AR technology at the service of painting
I work as a business architect in the field of digital transformation. Recently, in her free time, she began to engage in graphics and painting, and created the Digital OKO cycle of paintings with augmented reality. AR-technologies give the viewer the opportunity to get into an alternative world, located on the other side of the canvas. In this article I will tell you how to combine a daytime job with a professional hobby and get something so unusual that you will have a personal exhibition.
I am engaged in business analytics and business architecture, that is, helping my clients find new models in terms of digital business transformation. I am a programmer by education. But at a certain point I realized coding is not mine, and began to develop towards analytics.
The idea of ​​a symbiosis of painting with information technology came to my mind somehow itself. I wanted to connect the two worlds in which I live. And it so happened that I wrote several paintings on which there were video cameras and suddenly I thought, where are they looking? What if you let the viewer see the world from behind the canvas of the picture, show what the camera in the picture sees. And I decided to display the video from the cameras depicted in the pictures. After all, the main theme of the cycle is observation, the main image of which is cameras. But then I remembered: "There is augmented reality." And this means that you can create a separate world on the basis of the picture, into which you can look, to better understand the meaning of the work.
Before the idea arose, I did not see similar projects. I even checked the availability of such solutions - I could not find it. AR and VR in the field of art, of course, are used, but most often these solutions are in the area of ​​responsibility of museums, less often are a personal project of some young artist.
43 ° 46'05.0``N 11 ° 15'19.0''E
For example, at the Detroit Institute of the Arts, with the help of the augmented reality platform Tango from Google, you can look inside the sarcophagus and restore the faded colors on the exhibits. And at the National Museum of Natural History in Washington, visitors are sent to the Jurassic using the Broadcast AR augmented reality system. In the State Tretyakov Gallery, at the VR exhibition, visitors can look into the workshops of artists Natalia Goncharova and Kazimir Malevich, create their own version of Shishkinsky's “Morning in a Pine Forest” and hear Munch’s “Scream”. Grechten Andrew, an artist from London, created the VR project Alternate Reality . When working on paintings, the author often used Google Glass and shot videos and photos of the process, and then included the footage in VR. The visitor wore virtual reality glasses and saw the story of the creation of the picture. In addition to AR and VR, projections, interactive panels and screens, QR codes and RFID tags are more common technologies for designing expositions.
Returning to my project ... A little more than a year has passed since the creation of the first picture to the first Digital OKO exhibition. If my memory serves me right, then the idea with AR arose at the end of 2018. I raised the topic of application development three months before the first exhibition, which was in May 2019. And not because developing such an application is difficult, but because all risks must be taken into account, especially when working with a new team. We also started shooting videos sometime in March. With the development of the application, a group of professional developers helped me.
55 ° 55'13.9``N 37 ° 32'59.7''E
The augmented reality application is developed on the Unity engine with the Vuforia AR library. Thanks to this technology, the image transmitted by the tablet’s camera is analyzed by the computer vision software module and, by the given unique characteristics, detects pictures. When a picture is identified, the video content associated with it is loaded, the position of which in space is synchronized with the position of the picture. If the user with the tablet moves relative to the picture, the position of the video is adjusted due to the incoming information from the accelerometers and gyroscopes of the device and is refined using the computer vision system.
It should be borne in mind that the pictures are well tracked, provided that the viewer with the tablet stands so that the whole picture fits on the screen. If he stands closer and the camera sees only part of the picture, then tracking may fail. It depends on whether enough reference points fall into camera visibility.
There was a problem with one of the paintings: it was difficult for machine vision to track it, since there are too few contrasting points and clear angles that are reference points for machine vision. I preferred to leave the picture without AR, rather than change the image on it. Still, the paintings are primary. And the funny thing is that the picture depicts Jesus. This situation was perceived by the audience as a kind of mysticism: Jesus does not want to be recognized, and indeed, he does not need any cameras, he knows everything without them.
44 ° 18′47.9``N 09 ° 10′30.95''E
A story with augmented reality is rather a performance part. I did not upload applications to the Play Market and Apple Store, but provided tablets with a pre-loaded application for the duration of the exhibition. Firstly, I know for myself, not everyone likes to install “disposable” applications on their smartphones. There is no need to strain people. Secondly, we all know how difficult it is to upload an application to the Apple Store. Who has Android and who wants to install an application for himself, he can install it through the APK-file . But keep in mind that the application weighs a lot, eight high-quality videos are loaded into it.
Of course, it’s better to watch the pictures at the exhibition, but you can open my website , download the application and see the pictures along with the video.
41 ° 41'10.5``N 44 ° 48'19.0''E
My cousin Aleksey Alafiev , a very talented photographer and videographer, helped me with the video. He not only organized and carried out professional filming, together with him we came up with ideas. I sketched topics, and he already offered to visualize how to imagine it better. And here we have such a joint result.
By the way, he is the author of the video of my first exhibition Digital OKO .
42 ° 43'00.9``N 12 ° 06'48.0''E and 45 ° 04'50.2''N, 6 ° 30'61.0''E
I’ll tell you a little about the concept and the exhibition itself. Digital OKO is an augmented reality art project that reflects the impact of technology on society and blurring the boundaries of private life. Two shows of this project have already taken place. The first was just an exposition of paintings with AR. Then it was decided to develop the idea to a larger project and involve additional participants. An integral part of the opening of the second show was a performative program. The production of "I. They. We ”, which included immersive elements, tells about you and me. As we undergo manipulation and herd feelings, we are afraid of individuality and at the same time do not want to look banal. We are capable of being strong and resisting what deprives us of freedom.
55 ° 45'42.8``N 37 ° 36'12.1''E
Also, the audience was waiting for a MarsiAnka hang concert. Harmonious, soft and rhythmic sounds of Hang on an intuitive level return us to an almost lost nature. And the DJ ended the evening.
I think that I will continue in my work the theme of the impact of digital technology on humans. The Digital OKO (cameras) is just the tip of the iceberg. I am concerned about social topics, how a person feels himself in the modern world, where he is led by a changing world. This is the topic of finding yourself human. But I will hardly apply augmented reality further. Played with trends, surprised, impressed and enough.
My colleagues from DTG have visited my exhibitions more than once, and they liked it. One of our developers noticed that when three pictures hang on the wall and you point a smartphone at them, the application reads only one single picture. He said that it was a bug and needed to be fixed. But in fact, that was what was intended. Three videos are difficult to perceive.
There were also professional jokes about the blockchain that we are doing. They suggested recording the history of the creation of the picture on the blockchain.
It is also nice that journalists are also interested in my work. If you are interested in learning more about my work, you can read the interview in the Snob magazine.
Creativity helps my work. The ability to concentrate has increased, the ability to see something new has developed, not only visually, but also to pay attention to details. Creativity broadened my horizons, enriched me with ideas in analytics too.
I follow contemporary art. The work of avant-garde artists and conceptual projects are especially inspiring. And if in the directions, then, probably, avant-garde, hyperrealism, abstractionism, surrealism, installations and ready made art. Although I do not want to belittle the significance of, for example, post-impressionism, suprematism, social art and pop art. I am inspired by the pictures of my mentor - Timofey Smirnov. He has amazing graphics with a deep philosophical meaning.
For me, painting is not art therapy. I do not relax in the writing process. I'm interested in the process, but more - the result.
Come on our site for vacancies, by the way.
I am engaged in business analytics and business architecture, that is, helping my clients find new models in terms of digital business transformation. I am a programmer by education. But at a certain point I realized coding is not mine, and began to develop towards analytics.
The idea of ​​a symbiosis of painting with information technology came to my mind somehow itself. I wanted to connect the two worlds in which I live. And it so happened that I wrote several paintings on which there were video cameras and suddenly I thought, where are they looking? What if you let the viewer see the world from behind the canvas of the picture, show what the camera in the picture sees. And I decided to display the video from the cameras depicted in the pictures. After all, the main theme of the cycle is observation, the main image of which is cameras. But then I remembered: "There is augmented reality." And this means that you can create a separate world on the basis of the picture, into which you can look, to better understand the meaning of the work.
Before the idea arose, I did not see similar projects. I even checked the availability of such solutions - I could not find it. AR and VR in the field of art, of course, are used, but most often these solutions are in the area of ​​responsibility of museums, less often are a personal project of some young artist.
43 ° 46'05.0``N 11 ° 15'19.0''E
For example, at the Detroit Institute of the Arts, with the help of the augmented reality platform Tango from Google, you can look inside the sarcophagus and restore the faded colors on the exhibits. And at the National Museum of Natural History in Washington, visitors are sent to the Jurassic using the Broadcast AR augmented reality system. In the State Tretyakov Gallery, at the VR exhibition, visitors can look into the workshops of artists Natalia Goncharova and Kazimir Malevich, create their own version of Shishkinsky's “Morning in a Pine Forest” and hear Munch’s “Scream”. Grechten Andrew, an artist from London, created the VR project Alternate Reality . When working on paintings, the author often used Google Glass and shot videos and photos of the process, and then included the footage in VR. The visitor wore virtual reality glasses and saw the story of the creation of the picture. In addition to AR and VR, projections, interactive panels and screens, QR codes and RFID tags are more common technologies for designing expositions.
Returning to my project ... A little more than a year has passed since the creation of the first picture to the first Digital OKO exhibition. If my memory serves me right, then the idea with AR arose at the end of 2018. I raised the topic of application development three months before the first exhibition, which was in May 2019. And not because developing such an application is difficult, but because all risks must be taken into account, especially when working with a new team. We also started shooting videos sometime in March. With the development of the application, a group of professional developers helped me.
55 ° 55'13.9``N 37 ° 32'59.7''E
The augmented reality application is developed on the Unity engine with the Vuforia AR library. Thanks to this technology, the image transmitted by the tablet’s camera is analyzed by the computer vision software module and, by the given unique characteristics, detects pictures. When a picture is identified, the video content associated with it is loaded, the position of which in space is synchronized with the position of the picture. If the user with the tablet moves relative to the picture, the position of the video is adjusted due to the incoming information from the accelerometers and gyroscopes of the device and is refined using the computer vision system.
It should be borne in mind that the pictures are well tracked, provided that the viewer with the tablet stands so that the whole picture fits on the screen. If he stands closer and the camera sees only part of the picture, then tracking may fail. It depends on whether enough reference points fall into camera visibility.
There was a problem with one of the paintings: it was difficult for machine vision to track it, since there are too few contrasting points and clear angles that are reference points for machine vision. I preferred to leave the picture without AR, rather than change the image on it. Still, the paintings are primary. And the funny thing is that the picture depicts Jesus. This situation was perceived by the audience as a kind of mysticism: Jesus does not want to be recognized, and indeed, he does not need any cameras, he knows everything without them.
44 ° 18′47.9``N 09 ° 10′30.95''E
A story with augmented reality is rather a performance part. I did not upload applications to the Play Market and Apple Store, but provided tablets with a pre-loaded application for the duration of the exhibition. Firstly, I know for myself, not everyone likes to install “disposable” applications on their smartphones. There is no need to strain people. Secondly, we all know how difficult it is to upload an application to the Apple Store. Who has Android and who wants to install an application for himself, he can install it through the APK-file . But keep in mind that the application weighs a lot, eight high-quality videos are loaded into it.
Of course, it’s better to watch the pictures at the exhibition, but you can open my website , download the application and see the pictures along with the video.
41 ° 41'10.5``N 44 ° 48'19.0''E
My cousin Aleksey Alafiev , a very talented photographer and videographer, helped me with the video. He not only organized and carried out professional filming, together with him we came up with ideas. I sketched topics, and he already offered to visualize how to imagine it better. And here we have such a joint result.
By the way, he is the author of the video of my first exhibition Digital OKO .
42 ° 43'00.9``N 12 ° 06'48.0''E and 45 ° 04'50.2''N, 6 ° 30'61.0''E
I’ll tell you a little about the concept and the exhibition itself. Digital OKO is an augmented reality art project that reflects the impact of technology on society and blurring the boundaries of private life. Two shows of this project have already taken place. The first was just an exposition of paintings with AR. Then it was decided to develop the idea to a larger project and involve additional participants. An integral part of the opening of the second show was a performative program. The production of "I. They. We ”, which included immersive elements, tells about you and me. As we undergo manipulation and herd feelings, we are afraid of individuality and at the same time do not want to look banal. We are capable of being strong and resisting what deprives us of freedom.
55 ° 45'42.8``N 37 ° 36'12.1''E
Also, the audience was waiting for a MarsiAnka hang concert. Harmonious, soft and rhythmic sounds of Hang on an intuitive level return us to an almost lost nature. And the DJ ended the evening.
I think that I will continue in my work the theme of the impact of digital technology on humans. The Digital OKO (cameras) is just the tip of the iceberg. I am concerned about social topics, how a person feels himself in the modern world, where he is led by a changing world. This is the topic of finding yourself human. But I will hardly apply augmented reality further. Played with trends, surprised, impressed and enough.
My colleagues from DTG have visited my exhibitions more than once, and they liked it. One of our developers noticed that when three pictures hang on the wall and you point a smartphone at them, the application reads only one single picture. He said that it was a bug and needed to be fixed. But in fact, that was what was intended. Three videos are difficult to perceive.
There were also professional jokes about the blockchain that we are doing. They suggested recording the history of the creation of the picture on the blockchain.
It is also nice that journalists are also interested in my work. If you are interested in learning more about my work, you can read the interview in the Snob magazine.
Creativity helps my work. The ability to concentrate has increased, the ability to see something new has developed, not only visually, but also to pay attention to details. Creativity broadened my horizons, enriched me with ideas in analytics too.
I follow contemporary art. The work of avant-garde artists and conceptual projects are especially inspiring. And if in the directions, then, probably, avant-garde, hyperrealism, abstractionism, surrealism, installations and ready made art. Although I do not want to belittle the significance of, for example, post-impressionism, suprematism, social art and pop art. I am inspired by the pictures of my mentor - Timofey Smirnov. He has amazing graphics with a deep philosophical meaning.
For me, painting is not art therapy. I do not relax in the writing process. I'm interested in the process, but more - the result.
Come on our site for vacancies, by the way.
All Articles