Stephen Cleary is in the top 100 Stack Overflow users. Mostly thanks to its asynchronous answers in .NET. His life is not limited to programming: on Twitter, he first writes about himself “Christian”, and only then “developer”.
Now, in connection with the appearance of async streams, his knowledge is especially relevant: as a rapporteur on this topic, Stephen’s figure just begs it. And very soon on DotNext he will tell about it. In the meantime, we asked him about all this: religion, Stack Overflow, and asynchrony.
Biography
- The first question is simple: tell us about your professional biography.
- I became interested in programming quite early, I was 7-8 years old. Then at school I was able to write a small program in the Logo language, and this was my first programming experience. Later, when I was 12-13, I saved up on my first computer. And when I chose Computer Science when I entered college, it was a pretty obvious decision: I was already interested in computers for a tangible time.
In 1998, graduating from college, I started working for a local company that dealt with conveyors automatically controlled by vehicles for industry and other factory automation. Seven years later, the company moved to Detroit, but I did not want to move there, and since then I managed to work in different companies. Since 2013, I work remotely, the last year and a half - in Faithlife.
- It's funny that you started with Logo, and now in a company whose main product is called Logos. And what exactly are you doing there?
- Backend: web services accessed by Logos and other products of the company. Faithlife has a range of church related products. Logos - for Bible study. And there is also Proclaim, designed for those who give sermons (helps with slides and the like). Faithlife has historically made products for pastors, but now they are adding things that help ordinary people.
I do backend, I often have to deal with ASP.NET Web API, and right now I'm working with Docker. Right now we have part in the cloud, and part on-premise, and something should not be transferred to the cloud at all, but we want to transfer some things there - and I'm doing it.
Religion
- It’s not often that you meet a company that makes “church” software, so I’d like to clarify: does work on the backend look like everywhere else, or does it have its own specifics? And developers are usually believers and themselves use the company's products, or not?
- There are no fundamental differences in the work: you need to think about the proper design of the API as much as everywhere else. Some of our products (especially desktop ones) have been around for a very long time, and you have to deal with old APIs. And when starting to use Docker, it's important to think about backward compatibility. In general, we have about the same concerns as others.
Although Faithlife makes products for the church, it is not a Christian company. To get here, you do not need to be a Christian: firstly, such a selection would violate American law (would be religious discrimination in hiring), and secondly, the company would not want this anyway.
But as for developers to use the company's products, it is encouraged in every way. For example, recently spent a training day on the use of Logos. We also have many internal tools - in this case, we are encouraged to go to work the day in the department that uses your tool to personally see how this happens.
- On your Twitter in the bio field there are the words “Christian” and “developer”. Do these two parts of your identity intersect somehow?
- I would say that they are separate. I always tried to separate the working life and the home, not doing work in my free time (although it did not always work out). The church is a big part of my personality, most of my friends are from there. At work, I made few friends.
“Well, your bio still says“ addicted to writing OSS, ”and giving people free software sounds altruistic and Christian. In this case, there is no correlation?
- Hmm, maybe. I think, from a philosophical point of view, you can really see the relationship between open source and Christianity: in both cases, the emphasis is on the “give to people” you mentioned.
There is also a “humanitarian” type of open source. I myself am not involved in this; I have general purpose libraries. But I know Christian developers who have worked for free on something that directly affects other lives, especially in places with poor people. For example, software that allows you to make sure that fire alarms are working properly. Here, of course, there is a correlation.
- You are also active on Stack Overflow - is this also a way for you to give something unselfishly to the community?
- I would not say that, for me it feels more like "teaching." Although “teaching” can also be connected with Christianity ... My family had many teachers, and I continue this in my own way - not as a profession, but through Stack Overflow and conferences. So I satisfy my need for this.
Stack overflow
- We still have a number of questions about Stack Overflow, the first wacky one. You recently received a badge there for answers on Windows Phone 8:
How did it happen that in 2019 you received a badge for answers about almost dead technology? Is someone still asking her questions?
- To get a badge, you usually need at least a certain number of answers with a certain number of votes for these answers. I suppose I got this badge, because now someone has voted for some of my answer, published years ago, and with it finally got the right number of votes. This is funny because I haven’t seen Windows Phone 8 questions for a very long time! (laughs)
- On SO, your reputation is 320,000 - and this is more than a quarter of the reputation of the legendary John Skeet, although you gave answers an order of magnitude smaller than him. How did you get this?
- I’m not completely sure. It’s easier to get a reputation for those who came to the site before everyone else, and although I was among the early adopters, I still came later than many. I began to answer questions - at first, 1-2 per day, which is insanely far from John Skeet. I focused on the topic of async / await, as well as concurrency and multithreading - simply because it then, in fact, became my specialty. Many years ago I was already engaged in asynchrony, and then it was painful. So when async / await appeared, I immediately saw what their meaning was. So I was at the right time in the right place, and my “teacher's blood” helped explain everything in a language that people understood. And in the case of Stack Overflow, I turned out to be a local async / await specialist. And John Skeet - well, he didn’t say it directly, but I suppose it - leaves most of the questions about async / await to me. But he, of course, answers much more than me, he can’t catch up!
- Looking at the numbers of reputation, it’s easy to think “a person takes half his life to answer” - but how much time do you actually spend on SO?
- Not so much. I check Stack Overflow a couple of times a day, and usually answer 1-3 questions. And I get the majority of positive votes for the answers left years ago. This is how SO helps those who came earlier: they were the first to answer questions years ago, received votes, and the voices make these answers more visible, and in the end they get new votes. This is an avalanche effect encouraging old answers.
It seems to me that this causes some problems, because sometimes there are new answers that refer to more modern technologies and are therefore better, but the old answers do not update. But, in general, I get the majority of votes for the old answers, but I still try to answer new ones every day. I do not spend much time on this, I just do it constantly for many years.
- When John Skeet flew to us on DotNext, we asked him about the current state of Stack Overflow and what he considered to be the problems of the site. It is interesting to compare: what do you think about this?
“I think SO is improving, especially in the last year.” They are now trying very hard to improve the quality of the questions. Often, people first asking something on this site are unfamiliar with how to ask technical questions correctly.
And this is a problem that has always been present on the Internet. In the 90s, when everyone was in the newsgroups, she was there too. Ten years later, everything happened on the mailing lists and Google Groups. Now a new resource for Stack Overflow programming questions. But there have always been questions about programming, and how to ask a good question has always been a problem, and how to maintain a friendly community, too. Perhaps these problems can never be completely resolved. I am not saying that we should not work on it - we should definitely. But if you look into the past, it turns out that even in the 90s they already wrote tutorials on how to ask technical questions.
There are a couple of issues that are new to Stack Overflow. You can start with the fact that many of them never asked someone else before asking their first question. Therefore, in many cases, they simply do not realize all those things that must be presented in the question so that it can be answered.
And then, imagine that you are working on a task. You are head over heels in code, your head is filled with all this. And you look at some specific thing that does not lend itself in any way. This is a small concrete detail of a large system, and usually you ask, “How do I do this small thing?” Without realizing that you have all this context that you know, but you don’t put it into the question. Often the question "How do I make X" the correct answer is "Do not do X, do Y". This is a trap that people often fall into when writing the first questions. They do not realize that their answer in its current form is “unresponsive”.
And besides the problem of the quality of questions, there is also a tendency - especially on Stack Overflow, where everyone struggles for points, trying to answer as quickly as possible - quickly close the question or quickly scribble not the most pleasant answers. I do not mean “spiteful”, I have seen very few frankly spiteful comments - only a few for all the years. Rather, they are harsh, and for the author of the question this is read as unfriendliness.
Stack Overflow has taken some simple steps to fix this. Now, when people first ask a question, the site tells you what you need to include in it. They previously added “look at these similar questions,” which was a good first step. And now there is a whole system that needs to be completed for the first question, which helps to compose it well.
They also add reminders for those who write the answers. It’s like “This is a newbie, be friendly”, and this is a good way to remind that most people don’t write questions well the first time, and this is completely normal. In general, they are working on it. Will this problem ever be completely resolved? I doubt. But progress is possible.
Asynchrony
- In a post dedicated to the 10th anniversary of the iPhone, you write that its appearance affected asynchronous programming, because mobile applications must be responsive. Can you give a general historical background on the development of asynchrony for those who have recently started developing and have not seen the world without async / await?
- Well, asynchronous programming has always been possible. I think in the 60s they were already doing it. And he had the same advantages for a long time: a more responsive UI and a more scalable server part. And support has always been a problem: it was very difficult to maintain asynchronous code. Initially, it was built on callbacks.
I think the big step was the advent of Promise. That's what they called it in JavaScript, and in C # it's Task or ValueTask. And this was a big step forward, because now we have an object representing the operation. You can monitor its progress and so on. And async / await in any language is essentially just syntactic sugar around Promise.
I would say that the appearance of Promise is the event that has had the most impact on asynchronous code. And in the case of async / await, it’s important that the state machine is created for you. In the old days, when we worked with callbacks, we had to make our own state machines ourselves. And it was always difficult, because you inevitably forgot some condition or transition, and nothing worked. And it was very difficult to debug. Async / await did not bring the very ability to write asynchronous code, but the ability to write code that can be supported.
- And now do you see the situation settled down, or will new changes soon be waiting for us?
- I think now everything is pretty stable. Async / await looked revolutionary - but only for those who had not done asynchronous programming before. And for those who worked, it felt very natural. But, in general, this was a big event.
And now another event comes with .NET Core 3. They do what everyone calls async streams, although they are actually asynchronous enumerables. I think this will confuse people, because there is already a Stream type that has nothing to do with async streams. In general, there will be more incremental improvements. Will we see something as massive as when async / await first announced themselves? I don’t think so.
If you want a new paradigm shift, there is always the possibility of an actor-based system. Or something like goroutine, where all asynchrony is implicit. In my opinion, these things are somewhat similar. The problem is that in .NET this is not so easy to add. I think there are too many restrictions for .NET to do this, so this is unlikely to happen. If we see a large-scale transition to the Actor-world or the Goroutine world, where the approach to concurrency will not be the same as with today's multithreading, this will require completely new languages and runtimes. And .NET is not worth going for. And I do not think that programming as a whole will take such a leap. Maybe I'm wrong, but my current position is as follows.
- More to the question of how much everything changes over time. Many programming books are rapidly becoming obsolete, but where changes are less, they last longer. What about your Concurrency in C # Cookbook , how much does it require updates?
- Good question. I just recently published the second edition, and the first came out in 2014. That is, five years have passed, and we are already on the second edition - for me it looks like a fast development.
I think it still depends on how the book is written. I tried to write so that it would not become out of date as long as possible. You just need to try not to refer to things like Windows Phone 8. But it will inevitably become obsolete anyway. Async streams and the like appear, and I wanted to include such things in the book. As a result, something was outdated, but most of the material migrated to the new edition without changes.
And, of course, it all depends on who the book is for. Of course, the book on using Visual Studio 2008 will expire very quickly. But I think that there is a place for genuine classics. I consider Code Complete to be one of the best programming books in the world. And how long was it written? I do not even know. Decades ago. And this is still a fantastic book! Something in it is outdated, but overall it is still great.
- Recently, there have been tweaks about async / await on Twitter, which began with a tweet from Vaughn Vernon, the author of several books on DTD and actor models:
He does not like async / await importunity - in his opinion, this spreads throughout the code and can lead to thread blocking. What do you think about this? Was it worth it to design something else?
- Yes, perhaps the fact that async / await is spreading around the project is the most common complaint. I would like to mention a couple of things here.
First, asynchronous code, in order to be truly asynchronous, requires asynchrony from everyone who calls it. And it does not get around. Whatever paradigm you use (even one of the old ones), in the end you run into it. I had a report where I chronologically go through all the paradigms and show how the code became cleaner and better.
There are a couple of different approaches to avoiding the widespread dominance of async / await. The first is to isolate all input / output into separate objects. You can use design patterns like Ports & Adapters: this allows you to contain all the I / O outside your business logic, and then it does not require any async / await. Recently, I saw a whole report on how to prevent “async everywhere” by refactoring the project so that business logic never deals with I / O. I would recommend this approach.
There is another approach to dealing with the dominance of async / await, but it is not feasible in .NET. In my opinion, Go tried to do this with goroutines. In fact, everything is there at all - it's a solid async / await. You can remove async / await from the language by adding it there to everything and making it implicit.
There are no other ways. When async / await appeared, someone (I don’t remember exactly who) said that it was like a zombie virus: as soon as part of your code is infected, distribution begins.
- Some developers use the actor model, explaining this by the simplicity of flow control (actors, in fact, are single-threaded). Do you think that with the right choice of libraries, developers can get rid of the complexity of programming concurrency?
“Well, not quite.” Because even with the model of actors, problems remain. You will not encounter such race conditions as in a multi-threaded program, but you will have friendship difficulties.
For example, problems with message delay or with asynchronous messages. They are usually necessary to prevent deadlocks in the actor model. But any actor model that uses asynchronous messages can also cause deadlocks. Asynchronous messages can also create their own coordination problems. Usually at this level one also has to deal with idempotency.
Also, using actors with asynchronous messages can cause state management to be quite confusing, unless it happens entirely in messages. And even so, you will have difficulty with eventual consistency. In general, from my point of view, the actor model cannot solve all the problems. I would describe it this way: it can only change where exactly the problems will arise.
“You are known as a donor, but you use other languages like TypeScript.” When you try them, how does it make you look at C # and .NET? Do you meet something there that you would like to see in C #?
- Great question. C # took a lot from other languages. One of them from which he took a lot is Python. And I like it. I haven’t written in Python for years, but I think this is a very well-designed language. I really appreciate the enumerator blocks that C # borrowed from Python. And Python was one of the first where Async streams appeared, so we can say that C # took it from there.
In different languages I like different things. I generally prefer static typing, so I haven’t been using Python lately, and for the same reason I’m using TypeScript instead of JavaScript. JavaScript has its advantages simply because of its one-threading. For example, if you look at the relevance of Reactive Extensions, you will see that in .NET this is not particularly accepted. And in JavaScript, you can see Rx everywhere.
- The last couple of questions is about your report on DotNext. You are going to talk about Asynchronous streams there - can you briefly tell .NET developers what to expect and why this report will be useful to them?
- So, my report on asyncronous streams: why they were added to the language, and what is the main scenario for their use. I approach this very pragmatically, and do not go into all the details of what is happening under the hood. There are things you need to know in order to properly use async streams yourself, and there are things you need to know in order to consume async streams in your code, and you will have to do this as they enter the scene.
- And the last question. It is very difficult to get you to the conference - why did you decide to come to DotNext?
- Yes, I'm not one of those who constantly travels to conferences, I am primarily a developer. I think it helps me as a speaker: I am very pragmatic and in the reports I give what I myself would be useful.
Your program committee contacted me a year and a half ago, and initially I was not opposed to coming, but it was necessary that this fit my schedule. And this year, due to the release of async streams, all conferences want a report on them, and I decided to speak not at two conferences a year, as usual, but a little more. I'm glad that thanks to this I can visit DotNext.
DotNext 2019 Moscow will be held November 6-7. In addition to the speech of Stephen Cleary, there will be dozens of other reports for donors - on the site you can see the full program .