Jenny Sabin is sitting on a perch like a perch on the scissor lift; her head sticks out of the opening of the porous structure, and she struggles to pull the material onto the exoskeleton of the installation, which is being built in the spacious atrium of building 99 of the Microsoft campus in Redmond.
“It will be amazing,” she says.
For Sabin and her Microsoft colleagues, the luminous, transparent and airy tent that they erect in Building 99 is both a research tool and a way to look into the future, in which artificial intelligence is woven into architecture.
“As far as I know, this installation is the first architectural design to be driven by artificial intelligence in real time,” says Sabin, founder and chief designer at Jenny Sabin Studio in Ithaca, New York. The specialists of this bureau designed and constructed this tent as part of the Microsoft Artist in Residence program.
The two-story structure consists of nodal modules made on a 3D printer, fiberglass rods and fabric, which in the digital space is interwoven with photoluminescent yarn. The design uses artificial intelligence to translate anonymous data on the expression of a human face, sounds, tone of voice and language into the choreography of the dance of color and light.
Using art and design to visualize information gathered by microphones and cameras from different places in the building, Microsoft designers and researchers hope to inspire viewers through the interactivity of architecture to think about the role of artificial intelligence in our lives.
“Craftsmanship and artistry, creativity and humanity play a very important role in technical innovation,” says Eric Horvitz, director of research at Microsoft. He is also the chairman of the Aether committee, whose efforts are focused on a responsible approach to the development and deployment of artificial intelligence technologies. The Committee, in particular, considers the complex issues of the use of AI and data confidentiality, the objectivity and bias of AI systems, as well as the interaction and cooperation of humans and AI.
The Artist in Residence program was designed to encourage artists to implement creative ideas at the intersection of art and computer science, where they could work together with Microsoft researchers and engineers and promote creative and innovative thinking throughout the company.
“Jenny’s creation,” adds Eric, “is the embodiment of the expectations and capabilities of machine learning and pattern recognition technologies, as well as concerns about their increasing impact on our lives. Nowadays, they are increasingly penetrating the surrounding reality, making the world around them more interesting, more beautiful, but at the same time potentially more vulnerable. ”
Living, breathing matter
The project, called Ada, weighs about 800 kg. The exoskeleton contains 895 unique nodal models specially made using 3D printing, connecting 1274 fiberglass rods into a hexagonal web, which forms a rigid ellipsoid tent.
“The project has many parts,” Sabin said with a plate of Thai food in her hands. For many weeks, she manually set up a complex structure for late hours.
Photoluminescent filaments are connected into a porous, luminous network of cells and cones, which are fixed on the exoskeleton and extend inward, lining a soft inner wall and giving Ada a honeycomb feel.
A holistic cone the height of a whole floor, wrapped in nylon tulle and laced with fiber optic cables, passes through the core of Ada and provides compressive forces to keep the entire structure in a coordinated tension.
Microphones and cameras from different locations in building 99 collect anonymized data, which the AI algorithms convert to a change in the color and light intensity, which can be seen through the LEDs woven into the fabric of Ada and through the backlight around the installation.
“This is a living, breathing structure and it is located in the heart of the building. How does she change the psychology of people in relation to the space in which they live, how do they influence this space and vice versa? ”Asks Asta Roseway, chief design researcher at Microsoft's Urban Innovation Lab in Redmond, managing the Artist in Residence program.
Eric Horwitz (left), director of the Microsoft research lab and Jenny Sabin (right), chief designer at Jenny Sabin Studio in Ithaca, an architectural design bureau in New York and a Microsoft project artist, discuss the installation of Ada. Photo: John Brecher.
Paradigm Inspiration and Scrapping
Roseway, co-founder of the Artist in Residence program in 2015, earlier this year, comfortably sitting on a sofa in the spacious atrium of building 99, and subsequently busy with Ada, said with a smile from ear to ear that when people smile at Ada, she will metaphorically smile back to them .
The installation of Ada symbolizes the onset of a new wave of AI - built-in intelligence. Now it is already moving from research laboratories to products and services that people use in everyday life. This, in turn, raises different questions and even concerns about how deeply AI integrates into our lives.
“Where is all this going? What else could this lead to? What other way can this develop? Will it be useful, say, in a hospital, where people should feel calm and comfortable? ”Asks Roseway.
Previous projects in the Artist in Residence program have prompted reflection on various topics, for example, the future of food - what will happen when enhanced perception capabilities allow people to communicate with plants, or research on technologies that will open the visual arts to visually impaired or blind people , and clothing will respond on the personal level of human moisture .
“The program actually aims to inspire and fundamentally change our minds, get us out of our bubble and how we see specific technologies,” said Mira Line, Microsoft’s co-director of ethics for AI and society, which began working with Roseway after writing the latter expert document on the value of the artist for the corporate environment.
For example, as she says, designers and engineers often create technology with a specific business task in mind. When artists join the development cycle, they tend to use the technology at their discretion, break it down and reconfigure it in a new way, and this helps a new look to see the potential opportunities, risks and ways for further research.
Lane and Roseway first met Sabin in the spring of 2017 at a charity event at the University of Washington Art School. Sabin, who grew up next to Seattle and is herself a graduate of the University of Washington, made a keynote talk about creating completely new materials and forms, the ideas of which she borrowed from nature, moving away from the traditional right angles of architectural design.
“There was, by definition, something new and fresh,” says Roseway. “Isn't it a terrific idea that our homes will harmonize much more with natural forms and, moreover, still have built-in intelligence ?!”
“Then, for a year and a half, we discussed what the joint project with Microsoft would look like,” Sabin continues. - The first negotiations concerned the idea of creating such a project, in the center of which a person will be put and who could discover the value of data in a completely different perspective. It was also supposed to raise questions about the possibility of incorrect interpretation and confidentiality of personal data, which the Lane Ethics and Society group was engaged in. ”
“There are many intangible structures and spatial aspects of data that we really cannot see, feel or understand until they come in contact with some kind of material boundary of contact,” Sabin continues. “For me, the most fascinating thing about this project is that it itself already represents research at the most fundamental level.”
Artist Jenny Sabin shows her laboratory at Cornell University in Ithaca, NY, where she is a professor of architecture. Photo by John Brecher for Microsoft.
To teach computers to understand people
Microsoft's Daniel McDuff is researching technologies that allow machines to accurately perceive human emotions. This allows the intelligent assistant to perform various actions, such as fixing when the patient missed the time of taking the medicine or notifying the nurse.
“Computers that recognize emotions can transform the gaming industry, environmental accessibility, and architecture,” he says.
His platform, which controls Ada, responds to signals related to emotions. Algorithms then turn the data into numbers that represent a gradient of emotional tone from negative to positive and from weak to intense. Ada, in turn, presents data through changes in color and light.
McDuff and his colleagues will store anonymized data (without video, audio or text in accordance with Microsoft's data privacy requirements) accumulated by Ada for three years. They are going to use them for research, for example, how weather conditions and current events affect the expression of our faces, the tone of voice and words we use, and also to study how people's behavioral patterns change throughout the day.
“I drew attention to my own data,” says McDuff. - On average, in the morning I smile twice as often as in the evening. I had no idea. I think this is a pretty big difference. I hope this project will help to recognize many types of behaviors. "
Participation in the project is voluntary, and the Microsoft team, which deals with the confidentiality of personal data, has ensured compliance with the relevant conditions for the provision of data and their protection through anonymization of data. McDuff explained that if desired, employees can avoid contact with sensors by entering the building through special doors and using numerous kitchens, meeting rooms and relaxation rooms that are not tied to the project.
“As more and more people decide to join the project - and building 99 employees can install the system on their local computers and decide when to keep it turned on and when to turn it off - we are beginning to better understand how we interact and influence each other on a friend, ”emphasizes McDuff.
“Together we decide how the building will feel today,” he says. - Each of us is able to control our emotions and how we relate to others and interact in a team. If I am friendly towards all the people around and they smile at me in response, then I will have a greater impact on visualization than when I am alone, because thanks to me, the touch sensors of each of them detect smiles. "
Recognition of facial expressions is similar to other developments in the field of AI aimed at improving the understanding of a person, including understanding his goals and aspirations. For example, Cortana’s intellectual assistant recognizes when people promise something to others in their email messages, and then reminds them of it if necessary.
For quality work, artificial intelligence systems need to better understand people. The data collected by Ada will help McDuff improve his system and, as he hopes, start a dialogue on the competent and reasonable use of these technologies.
“If I try to take the place of someone not involved in this project, it may seem to me that technology corporations are trying to hide the data collected from us,” says McDuff. “We intentionally made the installation transmit this data and so that people began to have questions that they could not always ask before.”
Daniel MacDuff, chief specialist at Microsoft's research organization, is checking Ada’s installation. McDuff has created a measuring platform that collects data related to feelings. Artificial intelligence algorithms translate this data into a dance of color and light, which displays Hell. Photo by John Brecher for Microsoft.
Blurring the borders
Curiosity is the main engine of Sabin, who, in addition to private practice in an office in the center of Ithaca, is also a professor of architecture at Cornell University, where she strongly encourages her students to try different approaches to architectural design. “We often imagine an architect drawing sketches on a napkin. We call this a party diagram, from which a great idea for a project can be born. And I say, let's turn it upside down and think about how the final form arose from a set of relationships, ”she says, accompanying us to her laboratory in the basement of the Cornell School of Architecture.
The shelves of the laboratory are filled with prototypes, such as a sequence of weightless, porous bricks printed on a 3D printer, which once again speak about the uniqueness of its approach, based on collaboration with biologists and engineers. This approach leads to the idea of rethinking the walls. A lone nylon sleeve woven from photoluminescent yarn hangs from an air hole and reminds everyone of the study of the interaction of light and perception that Ada demonstrates.
A variety of 3D printers, a laser knife, a drying oven and a robotic arm, more suitable for conveyor assembly, clearly demonstrate the advanced manufacturing technologies that Sabin owns and which can transform the look at architectural design and its significance. Ada's cells and cones, for example, are created as a result of a process called 3D digital knitting.
In addition to using tools from other subject areas, Sabin teaches his students how to apply ideas and methods from different areas of science. For 14 years, she worked with cell biologists, material scientists, and mechanical engineers to create completely innovative materials and architectural designs that would be adaptive, immersive, and responsive to the environment and the people under the dome.
“How much do we make our space personal? “How often do we think about architecture not only as providing sustainability, functionality, and meeting the required standards, but also as a deeply human and personal area?” Asks Sabin.
Ada's Appearance
Working at Jenny Sabin Studio is more like an intellectual activity than a transformation of space. Designers click on digital models of Ada on computer screens. Glass racks are filled with memorable statuettes and honorary tablets. In a place of honor behind a tiny Sabin table, a newspaper article about the Lumen project, a MoMA PS1 street installation in Queens (New York), which announced the creation of Ada, hangs in a frame.
In the studio, Sabin and her design team are working on projects and exhibition facilities.
“I'm seriously interested in how research can affect architecture, not just in terms of teaching material or as a way of thinking, but how it affects living architecture,” she says.
On a cold March day at the beginning of this year, almost finished digital images of Ada showed how this aerial structure would fill the atrium of building 99 and translate the data into color and light.
Sabin named her installation in honor of Ada Lovelace, the famous scientist of the nineteenth century. In 1843, Lovelace proposed the use of punch cards to solve the mathematical equations of the Analytical Engine, invented by Charles Babbage, who was the forerunner of a digital computer, but was never built.
“Ada is often credited with the fact that she was the first programmer,” says Sabin. “It was especially correct to name the installation in her honor, because the project combines production technologies, such as 3D-digital weaving, with the visualization of data that feeds AI.”
Jonathan Lester (left), chief electrical engineer at Microsoft Research Laboratory in Redmond (Washington), and Asta Roseway (right), chief designer, managing the Artist in Residence program, are preparing LED light elements to weave into Ada's fabric. Photo by John Brecher for Microsoft.
Architecture Meets AI
For more than three weeks this summer, at 99 Sabin, Roseway and their team of researchers, designers and engineers connected nodes and rods into an exoskeleton, pulled fabric, tied LEDs, set up stage lights and hung a tensegrity cone.
Meanwhile, McDuff’s sensors began to collect data in public places throughout the building: in the atrium, office kitchens and recreational areas.
Each sensor is a small box and consists of a webcam and microphone connected to a black box the size of a modem that sends data via Wi-Fi to a secure Azure database.
“If I complain about something or I'm angry, or something goes wrong, then the system picks it up as a negative signal, and if I'm happy and everything around is sunny and positive, then the system registers and responds accordingly,” explains McDuff
Horvitz, director of a research organization at Microsoft, noted that his teams had been using AI technology in building operations for some time. So, for example, in 2012, the AI team under his leadership developed and introduced the 99 elevator ordering system in building 99 based on monitoring the movement of people in the building, using machine learning and reading data from sensory elements. The system is part of everyday life in this building.
Regardless, Schneider Electric, a global company that digitally transforms energy management in homes, buildings and industrial plants, is currently testing with Microsoft researchers whether AI can help reduce carbon dioxide emissions in heating, ventilation and air conditioning systems in large buildings.
Going beyond real-time sensor applications, AI is beginning to play a fundamental role in design and engineering design. He uses methods to help research non-standard approaches and capabilities in design in real-world operating conditions in terms of form, tensile strength and utility, ”says Horwitz.
“You can imagine other uses,” he adds. “For example, how can the environment change in the future to become more receptive and responsive to the goals and needs of participants?”
Ada's algorithms are designed to turn any data source into color and light. For example, scientists believe that a smart canvas can be used to visualize music, vibrations of a building, reflect the number of people in a given place, and there are still scenarios that have not even been invented yet.
“Anyone here,” says Roseway, pointing with a wide gesture around him, “can connect and get their share of the system’s response.”
At this point, the McDuff touch platform controls Ada's light response network. The last sensor is attached to the base of the tensegrity cone, which allows people inside the pavilion to control Ada. A smile comes by itself when, standing here inside, you look in admiration at this whole incredible construction. Ada's task is to answer the same.