Guiding philosophy
1. Programming languages for people
Programming languages are how people speak with computers. The computer will be happy to speak any language that is not ambiguous. The reason why we have high-level languages is because people cannot handle machine language. The essence of programming languages is to prevent our poor fragile human brain from being overloaded with a mass of details.
Architects know that some design problems are more mundane than others. One of the clearest and most abstract design issues is bridge design. In this case, your job is to cover the required distance with as little material as possible. At the other end of the spectrum is the design of chairs. Chair designers should spend their time thinking about human asses.
Software development has a similar difference. Designing algorithms for routing data over a network is a good, abstract problem, like designing bridges. Whereas designing programming languages is like designing chairs: you need to deal with human weaknesses.
Most of us have a hard time realizing this. Designing elegant mathematical systems sounds much more attractive to most of us than indulging in human weaknesses. The role of mathematical elegance is that some degree of elegance makes programs easier to understand. But everything is not limited to elegance.
And when I say that languages should be designed to take into account human weaknesses, I do not mean that languages should be designed for bad programmers. In fact, you should design software for the best programmers, but even the best programmers have their limits. I don’t think that at least someone will like to program in a language where all variables would be denoted by the letter “x” with integer indices.
2. Design for yourself and for your friends
If you look at the history of programming languages, most of the best languages have been designed for use by their own authors, and most of the worst have been designed for other people.
When languages are designed for other people, it is always a specific group of people: people are not as smart as the creators of the language. So you get a language that speaks condescendingly to you. Cobol is the clearest example, but most languages are imbued with this spirit.
This has nothing to do with how high-level the language is. C is quite low-level, but it was created for use by its authors, which is why hackers love it.
The argument for designing languages for bad programmers is that there are more bad programmers than good ones. Perhaps this is so. But this small number of good programmers write disproportionately more software.
I am interested in the question of how to create a language that the best hackers will like? It seems to me that this question is identical to the question of how to create a good programming language ?, but even if it is not, then at least it is an interesting question.
3. Give the programmer as much control as possible
Many languages (especially those created for other people) behave like nannies: they try to warn you from things that, in their opinion, will not be useful to you. I have the opposite opinion: give the programmer as much control as you can.
When I first studied Lisp, what I liked most was that we spoke on equal terms. In other languages that I had studied at that time, there was a language, and there was my program in that language, and they existed quite separately. But in Lisp, the functions and macros that I wrote were the same in which the language itself was written. I could rewrite the language itself if I wanted to. It had the same appeal as open source software.
4. Brevity - the sister of talent
Brevity is underestimated and even despised. But if you look into the hearts of hackers, you will see that they love brevity very much. How many times have you heard hackers lovingly say that, say, in the APL, they can do amazing things with just a couple lines of code? I believe that really smart people really like to pay attention to this.
I believe that almost everything that makes programs shorter is good. There should be a lot of library functions, everything that can be implicit - should be like that; the syntax should be more concise; even entity names should be short.
And not only programs should be short. Manuals should also be short. A good part of the manuals are filled with explanations, disclaimers, warnings and special cases. If you need to shorten the manual, the best option is to fix the language, which requires so many explanations.
5. Recognize what hacking is.
Many people would like hacking to be mathematics, or at least something similar to natural sciences. I think hacking is more like architecture. Architecture is connected with physics, in the sense that the architect needs to design a building that will not fall, but the architect’s real goal is to create a great building, and not make discoveries in the field of statics.
What hackers love is to create great programs. And I think that, at least in our own thoughts, we should remember that writing wonderful programs is wonderful, even when this work is not easily translated into the ordinary intellectual currency of scientific works. From an intellectual point of view, it is equally important how to develop a language that programmers will love, and to create a terrible one that embodies the idea about which you can publish an article.
Open issues
1. How to organize large libraries?
Libraries are becoming an important part of programming languages. They become so big that it can be dangerous. If it takes more time to find a function in the library that does what you need, than to write this function yourself, then all the code does nothing but thicken your manual. (Symbolics manuals were an example.) So we have to solve the problem of organizing libraries. Ideally, design them so that the programmer can guess which library function is suitable.
2. Are people really scared by the prefix syntax?
This is an open problem in the sense that I have been thinking about it for several years and still do not know the answer. The prefix syntax seems completely natural to me, perhaps besides using it in mathematics. But it may be that most of Lisp’s unpopularity is simply due to an unfamiliar syntax ... Is there anything to do with this, if it is true, this is another question.
3. What do you need for server software?
I think that most of the applications that will be written in the next twenty years will be web applications, in the sense that the programs will be located on the server and will communicate with you through a web browser. And to write such applications we need new things.
One of these things is supporting a new way to release server applications. Instead of one or two major releases per year, like desktop software, server software will be released in a series of small changes. You can have five or ten releases per day. And everyone will always have the latest version.
Do you know how to design programs to be supported? Server software must be designed to be adaptable. You should be able to change it easily, or at least know what a minor change means and what is important.
Another thing that can be useful in server software is, suddenly, continuity of delivery. In a web application, you can use something like CPS to get the effect of routines in the stateless world of web sessions. May have continuity of delivery is worth it if this opportunity is not too expensive.
4. What new abstractions are left to open?
I’m not sure how reasonable this hope is, but personally I would really like to discover a new abstraction - something that could be as important as the first-class functions or recursion, or at least the default parameters. Maybe this is an impossible dream. Such things often do not open. But I do not lose hope.
Little-known secrets
1. You can use any language you want
Previously, the creation of applications meant the creation of desktop software. And in desktop software there is a big bias towards writing applications in the same language as the operating system. So ten years ago, writing software as a whole meant writing software in C. In the end, tradition evolved: applications should not be written in unusual languages. And this tradition has evolved so long that non-technical people, such as managers and venture capitalists, have also learned this.
Server software destroys this model completely. With server software, you can take any language you want. Almost no one else understands this (especially managers and venture capitalists). But some hackers understand this, which is why we have heard about indy languages like Perl and Python. We don’t hear about Perl and Python because people use them to write Windows applications.
What does this mean for us, people interested in designing programming languages, that there is a potential audience for our work.
2. Speed comes from profilers
Language developers, or at least its implementers, love to write compilers that generate fast code. But I think this is not what makes languages fast for users. Knut has long noticed that speed depends on just a few bottlenecks. And anyone who tried to speed up the program knows that you cannot guess where the bottleneck is. The profiler is the answer.
Language developers are solving the wrong problem. Users do not need benchmarks to work quickly. They need a language that can show which parts of their program should be rewritten. At this point, speed is needed in practice. So it might be better if language implementers spend half the time they spend optimizing the compiler and spend it writing a good profiler.
3. You need an application that makes your language grow
Maybe this is not the ultimate truth, but it seems the best languages have developed along with the applications in which they were used. C was written by people who needed system programming. Lisp was designed in part for symbolic differentiation; McCarthy was so eager to start that he began to write differentiation programs even in the first Lisp document in 1960.
This is especially good if your application solves some new problems. This encourages your language to have new features that programmers need. Personally, I am interested in writing a language that is good for server applications.
[During the discussion, Guy Steele also expressed this idea, adding that the application should not consist of writing a compiler for your language, unless your language is intended for writing compilers.]
4. The language should be suitable for writing one-time programs.
You know what a one-time program means: this is when you need to quickly solve some limited problem. I believe that if you look around, you will find many serious programs that started as one-time ones. I would not be surprised if most programs started as one-off. Thus, if you want to create a language that is suitable for writing software in general, then it should be suitable for writing one-time programs, because this is the initial stage of many programs.
5. Syntax associated with semantics
It is traditionally believed that syntax and semantics are very different things. It may sound shocking, but it is not. I think that what you want to get in your program is related to how you express it.
I recently spoke with Robert Morris, and he noticed that operator overloading is a big plus in winning languages with infix syntax. In languages with prefix syntax, any function that you define is actually an operator. If you want to add up the new type of number you made up, you can simply define a new function to add it. If you do this in a language with infix syntax, you will see that there is a big difference between using an overloaded operator and calling a function.
Ideas that come back over time
1. New programming languages
Looking back in the 1970s, it was fashionable to develop new programming languages. Now this is not so. But I believe that server software will again return the fashion for creating new languages. With server software, you can use any language you want, so if someone creates a language that seems better than the rest, there will be people who decide to use it.
2. Time sharing
Richard Kelsey has come up with this idea, the time of which has come again, and I fully support it. My guess (and Microsoft, too) is that many calculations will move from the desktop to remote servers. In other words, the division of time has returned. I think you will need support at the language level. For example, Richard and Jonathan Reeves did a lot of work to implement process planning in Scheme 48.
3. Efficiency
Recently it seemed that computers were already quite fast. More and more we hear about bytecode, which at least for me means that we have the power in stock. But I think that with server software, we do not have it. Someone will have to pay for the servers running the software, and the number of users that the server can withstand per machine will be a divisor of their capital costs.
I think that efficiency will matter, at least in the bottlenecks of computing. This will be especially important for I / O operations, because server applications perform many such operations.
In the end, it might turn out that bytecode is not an option. Sun and Microsoft currently seem to be fighting face to face on the bytecode field. But they do it because bytecode is a convenient place to embed themselves in the process, and not because bytecode alone is a good idea. It may turn out that this whole battle will go unnoticed. It would be funny.
Traps and traps
1. Customers
This is only an assumption, but it is that only those applications that will be fully server-side will benefit. Designing software that works on the assumption that everyone will have your client is like creating a society based on the assumption that everyone will be honest. It would definitely be convenient, but you have to assume that it will never happen.
I think there will be a rapid increase in devices with access to the web, and we can assume that they will support basic html and forms. Do you have a browser on your phone? Will there be a phone in your PalmPilot? Will your blackberry have a bigger screen? Will you be able to go online from your gameboy? From your watch? I dont know. And I don’t have to find out if I bet that everything will be on the server. It is simply much more reliable to have all the brains on the server. .
2. Object Oriented Programming
I understand that this is a controversial statement, but I do not believe that OOP is something important. I think this is a suitable paradigm for specific applications that need specific data structures, such as window systems, simulations, CAD systems. But I do not understand why it should be suitable for all programs.
I think that people in large companies love OOP, in part because it provides a lot of what looks like work. What, of course, can be represented as, say, a list of integers, can now be represented as a class with all kinds of scaffolding, with noise and bustle.
Another attractive feature of OOP is that methods give you a certain effect of first-class functions. But this is not news for Lisp programmers. When you have real functions of the first class, you can simply use them in any way that corresponds to the task, instead of pushing everything into a template from classes and methods.
I think this means for language design that you should not embed OOP too deeply into it. Maybe the answer is to offer more general, fundamental things, and allow people to design any object systems in the form of libraries.
3. Design by committee
If your language is being drafted by a committee, then you are trapped, and not only for reasons that everyone knows. Everyone knows that committees tend to create a lumpy, inconsistent language design. But I think that the big danger is that they do not take risks. When one person is at the head, he takes risks that the committee will never agree to take on.
Does it take risks to create a good language? Many people may suspect that designing a language is where you should stay pretty close to traditional wisdom. I bet it's not like that. In everything else that people do, reward is proportional to risk. So why should language design be any different?