Space researchers turn away from heaven to help you decide what to wear, what to watch, and what to listen to. But both star data and Stitch Fix store data are processed using machine learning.
Stitch Fix is one of the companies that use physics to better understand all the style problems of its customers.
Chris Moody knows firsthand the universe. As an astrophysicist he did simulations of galaxies, simulated on the supercomputers the expansion of the Universe and collisions of galaxies. One evening, shortly after defending a doctorate at the University of California at Santa Cruz, he met with a group of other astrophysicists for a glass of beer. But that night, none of them spoke of galaxies. They talked about fashion.
A couple of Moody's buddies, astrophysicists, had recently quit science and moved to Stitch Fix, an online styling company that is now priced at $ 2 billion. Moody looked at them in surprise. “They asked me: Don’t you find this task interesting?” He says. And he really thought so. However, when his friends, using phrases such as “Bayesian models” or “Poincare’s space”, described in detail their work consisting in predicting what kind of clothes a client might like, it strangely looked like the work he was doing for the doctoral. He found that a quantitative assessment of style "turned out to be a very close analogy to the general theory of relativity."
Four years have passed, and now Moody also works for Stitch Fix. He is from a gradually growing group of deserters from astrophysics who stopped exploring space and began to create recommendation algorithms and data models for the technology industry. They are members of data science teams at companies such as Netflix, Spotify, and Google. And even in elite universities, less and less astrophysicists remain in the academic environment after defending a doctorate. More and more of these people are sent to Silicon Valley.
To understand that astrophysicists are attracted to startups involved in consumer products, let us recall the recent surge in interest in machine learning (MO). Astrophysicists who process huge amounts of data collected by powerful telescopes looking to the sky have long used MO models, “teaching” computers to perform tasks based on the examples provided. Tell the computer what you need to find in one photo of the intergalactic space, and he will be able to do this for the remaining 30 million photos, and then start making predictions. However, MO can also be used to predict user behavior, and in 2012, corporations began to recruit people who knew how to apply this method.
Today, MO is at the heart of just about everything from clothing boxes in Stitch Fix to personalized movie recommendations on Netflix. How does Spotify manage to so perfectly predict songs that will surprise and delight you on weekly personalized lists? It works machine learning. And although the MO already forms its own field of research, since scientists from areas such as astrophysics have been working with such models for many years, they make ideal candidates for filling teams working with data science.
“We already did big data before big data became a separate area,” said Sudip Das, a former astrophysicist at Netflix.
Das defended his doctorate at Princeton by examining relict radiation - electromagnetic radiation left over from the Big Bang [ more precisely, it was formed 380,000 years after the Big Bang when the “dark ages” ended / approx. perev. ]. After this, he studied the data obtained by the Atacama Cosmological Telescope in Chile for several years. The telescope every night collected about a terabyte of data from space, and in this huge array of data, Das discovered an elusive astrophysical signal. It was a rare reward for years of meticulous work. This discovery attracted the attention of the University of Michigan, where he was offered the position of assistant professor.
However, Das refused and instead moved to Silicon Valley - first to work as a data specialist at Beats Music, then at OpenTable, and now at Netflix.
Not many factors influenced the decision to leave the scientific world: the salary is higher, and the job is richer. “There are obstacles to becoming a full-time member of the institute,” he says. And in the San Francisco Bay area, neither he nor his wife — also astrophysicist — needed to worry about finding a job. However, a real surprise for him was that the work in technology companies was really interesting. He met at Beats with "like-minded people working on tasks similar in intellectual complexity." The mathematics is the same, the application is different.
Das notices how more and more physicists are changing the heavy share of the scientist - where you can do the precarious financial work of a postdoc for ten years - for easy and well-paid work in technology companies. “Of all my fellow students who defended their doctorate in Princeton, only two did not go into commercial companies,” he says. “To stay there, you need to be a scientist to the core.”
This big bang has captured the whole industry. “Astrophysicists are our number one group,” says Eric Colson, chief Emeritus Specialist in Algorithms at Stitch Fix. “Most people have a doctorate in the field of working with numerical data, but if you build a graph, I think astrophysicists will come first. They teach mathematics very well - very many physicists are better versed in mathematics than mathematicians. They also teach programming well. They are better versed in computer science than most computer scientists. ”
Moody, who joined the team at Colson in 2015, directed the knowledge gained while working in the field of astrophysics to solving such problems as marking up the “hidden style” of a client - a unique personal taste in clothes. Stitch Fix does not ask customers to define their style using some commonplace labels. She collects data on people’s preferences for shopping and with tools like Style Shuffle — a kind of clothing tinder, where people can mark whether they like or dislike certain things. After collecting, all this data forms a “style space” - a map of everything that customers like and how these items are related to each other. Moody and the team use this model to predict what else the customer might like. The algorithm may conclude that if you like thick beads, you may also like bead beads - similarly, Netflix algorithms suggest that you might want to watch another comedy with a woman in the lead role.
Moody says that such tasks are not so different from those with which he dealt with while working on a doctorate. Hidden style card? “This is the Poincare space. This is what Einstein used to describe relativistic spaces, ”says Moody.
Other physical principles are involved in understanding the hidden style. Moody’s team uses a thing like spectral decomposition of a matrix , a concept of linear algebra to separate individual “notes” in an individual style — something like “pulling a guitar string and listening to a few notes.” A client may like feminine things, but more casual than professional ones. Each person’s style has many data points — few people can be attributed to clearly defined styles — and Moody says that with the help of physics, his team better understands all the complexities of a customer’s attitude to style.
“None of those who study physics are going to do clothing, but it turns out that this area is phenomenally rich,” says Moody. “It's amazing to try to think about a person’s personal style from a science perspective.”
Colson says that many astrophysicists in his team are attracted to work at the company “because of the visible results that are rarely found in theoretical science.” Here they can send a thing into production and see the results. ” When Moody does everything right, Stitch Fix is more likely to offer customers things that they like - and his team can track and improve this metric on a daily basis.
In the scientific world, astrophysicists can fight over the same task for years. And many of the most interesting problems have already been resolved, says Amber Roberts, a former machine learning engineer and astrophysicist, and now an employee at Insight Data Science, which helps scientists move into the industry. “We learned the size of the universe. We measured the speed of light. We found pulsars. We found black holes, ”she says. - Many of these major discoveries, for example, an understanding of the principles of space-time or gravitational distortion, made people interested in space exploration and cosmology. However, what you are doing is real - it expands a very small fraction of the field of knowledge, and for three years you have been working on writing a scientific work that will interest a dozen people around the world. ”
Das, an astrophysicist at Netflix, says it's hard to give up the romance surrounding exploring the universe. “When I explain what is happening to my parents, they say: You did such amazing things with the Universe, and now you are giving people recommendations on films!” Says Das. However, he agrees that his routine work is more related to technical issues such as “trying to reduce the error in measuring a parameter from 50% to 5%,” instead of exploring the universe.
In Netflix, the technical work looks something like this. But when he thinks about what he really does at work — he unites people around the world with films and stories that will help them better understand each other — he feels no less satisfaction from his contribution than when he worked as an astrophysicist. “It's like exploring another universe,” says Das. “The universe of human beings.”