Python, devouring the world: as a side project of one developer has become a popular programming language

Disappointed with the shortcomings of existing programming languages, Guido Van Rossum created the Python language. Now this language is used by millions of people, and Nick Heath talks with van Rossum about the past and future of the language.

Guido Van Rossum at Dropbox Headquarters in 2014



In late 1994, a group of selected programmers from across the United States gathered to discuss their new secret weapon.



Barry Worso was one of two dozen developers who gathered for the very first workshop on the new Python programming language , and recalls how obvious the joyful excitement of its first users was.



“I especially remember one person who said: 'Don’t tell anyone that I am here because the fact that we use python is a competitive advantage.” It was their secret weapon. ”



Even at this early meeting at the U.S. Bureau of State Standards in Maryland, according to Uorso, it was obvious that python was offering something new in the matter of easy code writing and simple results.



“When they first showed me python, I knew that it was something special. It was a combination of readability and the joy of writing code in python, ”he recalls.



Today, the enthusiasm for python has spread far beyond the boundaries of this handful of developers, and some of them predict that it will soon become the most popular language in the world, as it is gaining new users faster than any other language. Millions of people use python daily, and the exponential growth of its users shows no signs of slowdown.



Python is used for large and small tasks by professional and novice developers, and is especially popular among web developers, data science experts, and system administrators. It was with the help of python this year that the first images of a black hole, located 500 million trillion km from us, were obtained, and also countless scripts written on whip up on desktop computers all over the world work.



Python plays a major role in some of the world's most famous organizations: it helps Netflix stream video to more than 100 million homes around the world, underlies the Instagram photo-sharing phenomenon, and helps NASA explore space.

Early years of python

In some aspects, the growth of python’s popularity is just as surreal and amazing as the popularity of the British comedy group in whose honor it was named, and in its niche the language has become just as famous and influential.



This programming language appeared as a third-party project of a programmer from the Netherlands, Guido Van Rossum . In the late 1980s, Van Rossum worked on a distributed system at Centrum Wiskunde & Informatica (CWI), the Netherlands State Research Center for Mathematics and Computer Science. Frustrated by the shortcomings of existing programming languages, he decided to make a new one - one that would be easy to use, but at the same time with great potential.



For an outsider, the task of creating your own programming language is similar to a statement like “I'll build my own plane”, but van Rossum, who was then over 30, had a certain handicap. For three years at CWI, he worked with the team that created ABC, an interpreted programming language, so he already had an idea of ​​how to make an interpreter that executes programmer's instructions, and what kind of syntactic building blocks are needed for the new language.



It is also important to understand how difficult it was to do something with the limited capabilities of programming languages ​​available to van Rossum at that time. The Amoeba distributed computing system he worked with required him to use C or the Unix command line — both options had serious limitations. C demanded that developers not only deal with the difficulties of manual memory management and minefield navigation of possible errors - it did not have a code library for the developer’s everyday tasks that could be reused, which is why developers had to reinvent with each new project bicycle. And the Unix command line had its own problems - there was a wide variety of tools for common tasks, but it worked so slowly that it could not cope with complex logic.



The restrictions placed on developers by these languages ​​were such that creating their own interpreted language - which would take the best features from ABC - really seemed to Van Rossum the best option.



“I, in general, thought:“ Well, why don't I create my own language ”, stealing ideas from ABC, but reducing the size of the project from what we have been doing for three years, to what I myself could do in three months, and that's how python was born, ”he says. Van Rossum really started working on it at the end of 1989, and he borrowed the name from his favorite comedy troupe, Monty Python's Flying Circus - an association with snakes and a logo with a typed python appeared later - and worked on the project whenever possible.



“At that time I did not have a rich social life. So, instead of staring at the telly, I wrote the code, or sometimes I did both at once, ”he admits.



And although nominally he created python to facilitate his work, van Rossum understands that the real motive may have been the challenge of creating his own language.



“I don’t know how seriously I believed that this would make me more productive. I think that in part I just enjoyed the idea of ​​starting my project, writing the code that I wanted, and developing it the way I wanted. I like to program, ”he says.



And if for an ordinary person the idea of ​​creating his own YP may seem unusual, van Rossum had a good company. In the late 1980s, due to disappointment in the capabilities of existing tools, many new and popular languages ​​appeared. Larry Wall's famous saying is that he was prompted to create the Perl language by the complexity of problem solving, as well as the abundance of “laziness, impatience and arrogance”. Finding the best language for creating interactive integrated circuit design tools prompted John Osterhout to create Tcl.



In just three months, Van Rossum created a working prototype of the interpreted python, in which, according to him, there were no modern capabilities, but in which it was already possible to find out a modern version of the language.



“There are many important abstractions in python today that didn't exist then, but the language was very similar,” he says.



“At that moment, I had all the basic components of an interpreter and language. Simple python programs written for the first version of the interpreter are likely to work today, ”he says, adding that functions were defined the same way, indentation worked the same way, dictionaries and tuples were created identically, and there was a familiar interactive line input, where you can write code and immediately get the results.



However, when two of his colleagues took this language and immediately began to use it, van Rossum did not have high expectations about its wide distribution - he already knew how difficult it was to promote YP in the era preceding the Internet.



Today, it takes a few clicks to share software with the whole world. In the 1980s, this task was much more difficult, and van Rossum recalls the difficulties of trying to distribute the python predecessor, ABC.



“I remember in 1985 I first visited the United States on a vacation with a magnetic tape in my backpack,” says Van Rossum.





Guido van Rossum in 2001, when the Python Software Foundation was founded



Armed with the addresses and phone numbers of people who expressed interest in ABC through the rudiments of e-mail, which existed at that time - and by which it was impossible to send anything large, like the source code - he traveled from one address to another, handing out a tape. But, despite all efforts, ABC did not take off.



“It's no surprise that we were not able to actively distribute ABC, despite all its wonderful qualities,” he says. But the Internet revolution was gaining momentum, and distributing python was already much easier - a suitcase with films was no longer required.



Van Rossum released python into the world through the alt.sources newsgroup in 1991, almost under the terms of an open source license - although such a concept will appear only after six years. Although the python interpreter had to be downloaded from the Usenet network in the form of the 21st part, which then had to be assembled into a single file and unzipped, this delivery mechanism was still much more efficient than the delivery of physical copies, undertaken several years earlier.



“I was hoping for some success, but I already had the experience of releasing another project, which eventually turned into a fiasco,” he says.



With such experience, Van Rossum did not immediately draw any conclusions from the growing python user base. Gradually, he realized that the set of popularity did not stop, and at some point, after regular contacts with the python community, he realized that he had created something that could be successful.



“In my opinion, understanding came to me very slowly. After the first release of open source, I built the rhythm of releasing the following versions and interacting with the emerging python community. And all this felt like something serious. ”

Why python won

After the language began to gain momentum in the early and mid-1990s, van Rossum got the feeling that the python turned out to be the language whose time had come.



Van Rossum believes that the python of developers was attracted by the same thing that made him create this language. They wanted to find a high-level scripting language that provided a balance between ease of use and an abundance of features, and did not have the limitations that the Unix command line suffered from when working with complex logic. They wanted to put an end to the insistent requirements of C messing with memory management and constantly re-writing code for the same basic tasks at the beginning of each project.



Worso says that python has reached a balance in usability without sacrificing features - languages ​​that existed in the early 1990s could not offer such a thing. “I programmed a lot of Perl, Tcl and C, and it was not very interesting. Then a python came on the scene, and I thought: “Wow, now it's interesting to program again,” he says.





Barry Warso at the first python user work meeting in 1994



Python proposed, and offers, a clear, unambiguous syntax in which code is grouped into blocks through indentation, which makes it easier for developers to understand the code.



Fintan Ryan, research director of the application platform strategy team at analytic firm Gartner, says code clarity has played a big role in winning developer sympathy, both now and in the 1990s, although giving the importance of indentation has divided the community. “It suggested a very clear syntax. In other languages, it was also possible to write code like this, but python forced this to happen automatically. Some programmers love it, while others hate it, ”he says.



The fact that python puts clear and readable code first is not an accident - van Rossum said that programming languages ​​are needed not only to tell the computer what to do, but also to transfer ideas from one developer to another.



Besides readability, Ryan says, python almost immediately offered built-in functionality that set it apart from other languages. “There was such functionality as classes and exception handling. Python also supported features like lambda, map, and filter, which were very useful in many cases, ”he says.



Python might not have appeared if the popular YPs in the 1980s were of better quality. One of Van Rossum’s motivations for creating python was the incompatibility of the Perl scripting language and the Amoeba distributed computing system, which he worked on in CWI. “Python was lucky that Perl could not be ported to Amoeba,” he says. “If Perl could port to Amoeba, I would never think about creating my own language.”



Although python created a true adherent base after the release, in the 1990s he was an outsider among the YaP. Van Rossum says that Tcl / Tk and Perl competed for first place, and both of them had the same goal as python - ease of use and rich features.



“In the 90s, Perl definitely took the first place in the first three — it was something like a 400-pound gorilla — then went Tcl / Tk, and python took a modest third place,” he says.



Today, python is growing faster than all other PLs, according to active developers - this follows from the annual survey of developers of Stack Overflow, one of the most comprehensive reviews of PLs - and Perl has shrunk to the point that it was no longer mentioned in the last report.



This explosive growth is shown in the graph below, which shows how the number of views of questions related to python on the Stack Overflow website grew faster than the number of views of questions from other languages.







So how did python defeat his longtime rival, and how to explain the so different fates of the two YPs? Van Rossum believes that in particular this was due to how easy it is to maintain a code base when it goes beyond a certain framework. “People thought Perl was perfect for writing 10-line scripts,” he says. “But when you have 500 lines of main code and several thousand lines in libraries, a huge discipline is required in order for the code to be supported in the future.” In python, even if you don’t have such discipline, the code will be easy enough to read and maintain. ”



This property of python, a combination of a simple language for beginners, which at the same time is reliable enough to write large applications on it, van Rossum attributes to its initial success in the 1990s.



“There were several developers on the Internet - most of this story relates to the early stages of the Internet’s development - who needed to write more and more large applications, and who liked that you can spend less effort on python than using C, C ++ or Java. "



With the growing popularity of python in the 1990s, van Rossum, still working at CWI, discovered that his brainchild more and more connected him with people from all over the world. “I did something, and it connected me with people from all over the world - people from Australia, from the USA, from other European countries. I really liked it, ”he says.

Python and web

By the mid-1990s, they began to use python in a new way, from scripts to record and play audio, to the first raids of the language on web development, which in the end would be the main stronghold of the python.



“One of the big areas of language usage was web development, which was also very interesting for me personally,” says Van Rossum, and adds that they started using python to create backend scripts on web servers along with Perl and command line scripts. “You could create dynamic web pages, and this has become one of my favorite uses of python.”



Gartner ryan says python’s popularity among 1990s developers has made it his favorite choice for quickly creating powerful scripts. “The fact that it was a scripting language with extensive functionality reduced the entry barrier for many users,” he says.



In addition, Ryan says, YP was flexible enough and so easy to learn that it was liked by very different users with different technical backgrounds. “Sysadmins and other people began to use this language to automate systems and programming, developers because of such things as functional programming and classes with inheritance; Perl, for example, had far fewer such features, he says. “After people got acquainted with the language, and the learning curve was fairly gentle, they very quickly began to reach a high level of productivity.”



By 1994, python had attracted the attention of Michael McLay, one of the executives at the US Bureau of Standards (NBS), which has now transformed into the State Institute of Standards and Technology (NIST). Anticipating the love of researchers and scientists for the python for its ease of use, MacLay wondered whether NBS scientists could use the python, who, according to van Rossum, “collected significant amounts of data, but did not have good programmers.”



To help sell the idea of ​​using python to other members of the bureau, Maclay invited Van Rossum, who was still working at CWI in the Netherlands, to work at NIST for a couple of months as a visiting researcher. This was the catalyst for the future evolution of python and seriously changed the life of van Rossum.



It was at that time that the first working meeting of python users was held, which took place in the NBS offices, at which van Rossum, Barry Warso and other first python enthusiasts met to discuss how they use python and what hopes they have for its future development.





Van Rossum at the first working meeting of python users, along with Uorso and Roger Masset, who was working with Uorso at CNRI at the time.



At the NBS offices, van Rossum met Bob Kahn, known for his role in developing one of the main Internet technologies, the TCP / IP protocol.



At the meeting, Van Rossum was offered to work with Kahn at the Corporation for State Research Initiatives (CNRI), a nonprofit research group from Virginia engaged in strategic development of network technologies.



As the python responded to the need for a new type of PL, so the offer of work, according to van Rossum, came just at the moment when he was wondering about his future at CWI.



“CWI was a more academic organization, they constantly pressed me to either defend my doctorate or find work elsewhere,” he says.



“I was not very pleased with the prospect of doing a doctoral dissertation at the age of 35. Before that, also thanks to the python, I had several other possibilities when, after several phone calls and some time for reflection, I decided to sweep them away. But here I liked the people, I liked their proposal, the project - and I took advantage of the proposal. "



It was at CNRI that van Rossum put together many of the structures needed to manage a python YP using a team of the same language enthusiasts. Having settled in CNRI in April 1995, van Rossum led a small team of developers working on Knowbot Programs software, mobile agents designed to run on distributed computer systems such as the Internet.



The team worked in the python language, and Jeremy Hilton, Roger Masset, Barry Worso, Ken Mannheimer and Fred Drake joined van Rossum - all of them later will play important roles in the python community.



“We got a group of 4-10 people, most of whom worked at CNRI, and they became the core of python development,” says van Rossum.



While van Rossum worked at CNRI, this team helped him build python.org, a CVS server for managing code base changes, and a mailing list for Python Special Interest Groups, groups that worked to improve and support Python python.



The community of python users has grown rapidly since its release in 1991, and in the second half of the decade, this language has attracted a significant user base from around the world.During this period, formalization of language management began, the developers founded the predecessor of the Python Software Foundation (PSF), which will appear in 2001. With the growth of the community, twice a year working meetings on the language, which began in 1994, turned into larger annual events, and in as a result - in the annual PyCon from PSF, which is held to this day.



By the beginning of the 21st century, the language outgrew the capabilities of any individual, a large community played an active role in the development of the language - a long way has been covered since the summer of 1994, when early language users were worried about what would happen to him if Guido gets on a bus.



But van Rossum continued to play a key role, and did not cease to be the core of the language - this is evident from the playful nickname of the " magnanimous life-long dictator"(BDFL), which has existed for many years.



" For the most part, I have been able to cope with pressure for quite some time, and also have developed project management skills that have allowed me to delegate a lot of things and give people control over their own destiny, ”says van Rossum .



Ryan from Gartner said that there is nothing unusual in the fact that the creator of the language remained his manager, Larry Wall with Perl and Ryan Dahl with Node.js, but he said that van Rossum is known for its objectivity in the management of a python. "as for project development directions and Managing it, everyone considers it very balanced, "he says.



It is the open nature of the development of python, using public discussions of the community and key developers, that Van Rossum considers the decisive factor in the success of the python.

Python evolution

In the intervening years, python took gigantic steps, in 2008 a modernized Python 3.0 was introduced, and recently serious changes have been outlined in language management. They began last year when van Rossum lost the lead in the BDFL post after the community shared a proposal to introduce assignments into the language in expressions discussed in Python Enhancement Proposal PEP572 .



A new opportunity was introduced in order to write more efficient code, but van Rossum faced serious criticism of this proposal from the opponents of the changes - some of them argued that its use would degrade the readability of the code and complicate its support.



Van Rossum said he was used to arguing about innovations, but in this case, the intransigence of some slanderers and the nature of personal insults forced him to resign.



“I was disappointed with the fact that people who didn’t agree with the proposal due to technical reasons left the social network and began to rant that the decision-making process was broken, or that I was making a serious mistake. It felt like they were attacking me from the back, ”he says.



“In the past, it was always clear that if a language needed to be changed or improved, some of the key developers would come together and discuss the advantages and disadvantages of the innovation. And then either an explicit consensus was developed, or everything was not so clear, and then I considered this proposal and made a decision. And with PEP572, although the proposal was controversial, I decided that I want to approve it, and people did not agree to agree that we did not agree. It wasn’t a riot, but it seemed to me that I didn’t have the trust of a sufficient number of key developers to continue working. ”



He believes that a change in the dispute resolution process has occurred, in particular, due to the huge number of people using python today.



“It's probably all about how big the python community has become. It’s more difficult to reach some kind of agreement, because there will always be supporters of opposing points of view, no matter what you decide. ”



Earlier this year, the key python developers - supporting and updating the CPython interpreter - chose a governing board that should plan the future of the language. This included Van Rossum, Warso, and their colleagues Brett Cannon, Carol Willing and Nick Coglan. Worso says this change was necessary to manage the language with such a rapidly growing user base.



“I think Guido really seemed to be holding the whole world on his shoulders alone,” he says. - It rolled 25 years ago when the python community was much smaller and the language was smaller, but today it is too much for one person, understand? I think that it’s good for health and the opportunity to participate in community actions, it was good to distribute this burden to five, ”says Warso.



The first election to the governing council will take place after each release of the new version of python, which, according to Warso, will pave the way for the next generation of leaders. “If the python is alive after 25 years, then it will not be Guido and I who will manage it, understand?” He says.



The governing board organization was also welcomed by the community of key developers. One of them, Mariatta Vijaya, says that this action seems to be a step in the right direction. “The governing council is better than one person who makes all decisions - this is too much responsibility and burden,” she says. “This is a good sign, and thanks to this, the community will have more incoming information.”





One of the key developers, Mariatta Vigaya

Future python

Although python continues to attract an exorbitant amount of new users, some members of the community see difficulties ahead, and believe that the language needs to evolve in order to remain relevant.



At a meeting of users of python this year, BeeWare co-founder Russell Kit-Maggie warned everyone that python would face “existential risk” if it did not improve support for mobile and new web platforms. “Phones and tablets have penetrated the market so much that desktops and laptops have never dreamed of it, and the community has no idea how to use python on these devices,” he said, indicating that his son uses an iPad at school. “What will happen to python when laptops turn out to be niche devices?”



Kit-Maggie, whose BeeWare project aims to run the program once in python anywhere, rolled out a list of problems that python now has in the area of ​​supporting platforms other than standard x86 PCs.



He pointed out the following shortcomings: the need to improve compiled code support for new platforms not compatible with x86; python test suite crashes on mobile and web platforms; the installation size of python applications is bloated and cannot be reduced to a small subset of dependencies; the need to spend efforts on finalizing the asyncio library to work with the GUI for Android, Windows and the web; the number of modules in the standard library that are incompatible with python interpreters other than CPython.



Uorso says Kit-Maggie says the right thing, and considers important issues related to updating python for new platforms - phones, tablets, new web technologies like WebAssembly.



“So far, python cannot offer anything good,” says Warso about the dream of being able to download an application for iPhone or Android, and “not even knowing that it is written in python.”



With the growing number of processor cores in modern chips (up to 48 cores in the latest server processors from Intel), Uorso is also looking forward to when python is better able to distribute tasks across several cores.



He is very interested in the potential of Eric Snow's work on subinterpreters and how they can expand the capabilities of python in the simultaneous operation of processes. “I would really like to see more work on taking full advantage of multi-core systems,” he says.



Snow is working on long-term projects that make python easier to efficiently split tasks between multiple processor cores. At the same time, he concentrated on using a revised version of such an interesting python feature as subinterpreters and changing how each subinterpreter interacts with the global interpreter blocker (GIL).



"For Python 3.8, it will not have time to get ready, perhaps it will be in version 3.9," says Worso. “I think we will see this in the next two or two and a half years.” I am optimistic. I’m very glad that Eric continues to work on this, because I think it is important. "



He says that even from failed projects supporting multithreaded python, for example, The Gilectomy, you can learn interesting lessons related to how python can improve by distributing work between the cores.



The Python community is also taking steps to upgrade the standard language library with embedded code, and recently published a proposal to remove obsolete modules from it; this is a response to recent criticism about the state of the python standard library. It is usually considered one of the strengths of the language, it can be used to solve a wide range of ordinary tasks, therefore it is often said that python has “batteries in set”. However, at a meeting of python developers this year, a question was asked whether it would be better to let people choose libraries from the PyPI repository, instead of delivering all this functionality along with the language as a standard library.



There were also questions about whether the group of people who manage the development of the language - key developers and the governing council - could better reflect the diverse python user base in 2019. “I would like to see the best display for a variety of metrics, not only in terms of gender balance, but also by race, and everything else,” says Vijaya.



“I spoke at PyCon with girls from the PyLadies band , from India and Africa. They said: 'When people hear about python or PyLadies, they imagine people from North America or Canada, but in reality in other parts of the world there is also an extensive user base. Why are they so rare? ' And that seems reasonable to me. Therefore, I would like to see this, and I think that we should all take part in this. ”



As a result, despite the humorous idea with a “generous dictator” in charge of all issues, in recent years all ideas that have changed python have come from the community, says Worso, thanks to people using python to the maximum and opening doors for new users. “In reality, everything happens from the bottom up, from the community, and not vice versa,” he says.



As an example of how simple community projects can seriously affect python, Type Hints is a feature of Python 3.5 inspired by the mypy project launched by the graduate student in 2012. These "hints" allow for optional type checking, which allows developers to catch errors of a special type, which otherwise could have escaped them.





This year, python helped put together the first image of a black hole from pieces.



This additional level of security is not just an interesting opportunity if you have a project in which many people work on a large code base. An extra level of understanding of what code should and should not do can be crucial.



“The fact that this allows the python to scale for the needs of larger companies - such as Instagram, which essentially runs on Python 3 — is simply invaluable to me,” says Worso, citing asyncio as another example of an idea community that can seriously affect language features.



With the governing board and the largest user base of all time, Van Rossum is optimistic that “language evolution through the community” will continue to achieve “incredible successes.”



“We have a very dense community of key developers who have a new system of government, and I think that we are better prepared for the requests that come with python and its evolution,” he says, praising a deep understanding of the specific aspects of the language that exists in the community.



If someone has doubts about the community’s ability to find unexpected new ways to use the language, all you have to do is explore the role of python in getting the first image of a black hole, says Worso.



“It just blew me away. There are a couple of people in the python community whom I consider to be our crazy scientists. They are always trying to push the boundaries of language. ”