Here's what we learned: at the conference, we are waiting for a preacher of functional programming, four English-language reports, a new tool that will solve the problems of data engineers, a report on why Scala Plugin is incompatible with the Scala compiler, Professor Haskell, and a lot of functional programming. We also talked about the developmental boundaries of the Scala developer, why there are no jones among them, the “holivar” between Scala and Haskell, and the informational “bubble” of the community. Details under the cut.
Concept
Hi Vadim! Zero question - tell us about the conference concept?
I’m disappointing - there is no general concept. We are not satiated with Scala conferences in Russia. Such events are not so many that before the organization think about how unusual to surprise. This is a Scala conference, and there is plenty of it.
Does this mean that the community attending the conference has no expectations?
There are expectations. These are the expectations that there will be cool reports and speakers, and it is according to Scala. These are the expectations that you will not need to go to related reports on Haskell or Java to attend the two and a half presentations for which I came.
Is it because there were few Scala conferences in Russia?
We can say that the conferences were zero . When I wrote investigative articles about the Scala movement, I found information that a couple of events took place in St. Petersburg. People called it a conference, but it was a mitap - one track for 3-4 speakers. In September, a meeting was held, at which 400 people were declared and this was considered a conference. There is nothing else. The rest is mitaps in St. Petersburg, Moscow, something in Novosibirsk and close to zero in other cities.
Scala conference in Russia is already an event.
Note. The first and second part of the investigation. The final one is coming soon - stay tuned.
It turns out that ScalaConf 2019 is the first professional conference on Scala?
Yes, with some assumptions. The peculiarity is that before that there were no sites specifically for Scala. Usually these were Java conferences, where the rockies were a black sheep, mixed events on FP, and mitaps. If this is not Moscow or St. Petersburg, then the mitaps are also mixed.
ScalaConf 2019 is the first rocky platform for rockies.
At rallies or mixed conferences, Scala speakers have to adjust to the audience, drop a lot of topics and reduce the complexity of the report. Hardcore presentations appear at FP conferences, but you can’t tell about Scala only.
There will be no restrictions on ScalaConf 2019. Only Scala, only hardcore.
Scala.js
Scala dominates two areas: Big Data and backend. Therefore, there was no fundamental task to prepare N sections with a certain number of speakers in each. We proceeded from the reports that were, selected cool ones and labeled: something more about the backend, something about the FP. The division into sections is nominal, and the reports from the Language / Tuning block will be as clear as from Big Data. Speakers will speak for everyone.
The only exception is Scala.js, which few people write on, as well as on the frontend. More often this is an experiment, less often a meaningful decision in order to achieve certain advantages. Scala.js is suitable for complex Scala codebase if you want to use it inside the browser. I saw a good example of a compiler for the blockchain language using this tool. The developers were able to interactively add it directly to the browser: you write in the language, the compiler checks the validity and highlights.
We added the report “ Scala.js - is the tool suitable for you? "So that people see an example of use. Fullstack developers who write both backend and frontend equally well do not exist. The average developer rarely writes front-end and he does not want to take JS or TypeScriprt. He thinks: “Can I try Scala.js? Yes, not that it's me, no one will understand me. ”
There is no need to use Scala.js when there are tools and libraries from the world of frontend. Therefore, few people are experimenting with Scala.js, and the topic is extremely interesting. Alexander Sharikhin will just tell about his experience of using Scala.js in eLama.
Scala.js niche technology is mature, working and has a specific community. There are some problems with the integration of popular JS libraries, but everyone who used Scala.js in real projects responds positively.
Evgeny Tokarev, member of the PC ScalaConf 2019 .
Backend
There are two reports in this block - about search and indexing and about Akka. The first report of Andrei Korzinev - will there be something like a “case” on the method of working with indexes in Auto.ru?
Historically, they first used Apache Lucene in Java, then threw it out and wrote their YoctoDB in Java. Now they are doing the third iteration on Scala for more control over the query results.
Scala is used for distribution, in multi-threaded applications and very rarely for similar projects. Scala developers are more likely to focus on correctness and worry little about performance. For an average task, the developer already gets high performance without making any effort.
Avto.ru project is one of those that combines both correctness and performance. Therefore, Andrei's report is unique. He will talk about indexes for which performance is important: byte buffers, varhandless. Reports of this format are rare. The search engine is a task of a different level and is very different from the average.
Few write their own search engine, even fewer do it on Scala.
Nikita Melkozerkov will tell "in general" about one of the approaches for working with Akka?
He will talk about Akka Persistence and FoundationDB . Akka Persistence is a library that is configured for Akka and allows you to conveniently store all events, save, pick up and not worry about it. For her, additional libraries are written that link Akka Persistence to any database.
Akka Persistance can only work out of the box with LevelDB. All other storage backends are plugins from the community. They are not often updated, and there are fears of taking them into a serious project. Therefore, it is difficult to choose what to use Akka Persistance with.
A year ago, Apple opened the Foundation Source OpenDB library, and everyone was happy. It is cool, distributed, super-correct and reliable due to the fact that every night the developers simulate one trillion tests. Everyone was inspired by the event, and Nikita wrote the Akka Persistance plugin for FoundationDB.
In 40 minutes Nikita will talk about FoundationDB and its advantages, why Cassandra did not work, about Event Sourcing, about the main stages and features of writing plugins in Akka Persistance. I hope Nikita will be able to convince everyone that to fix or write a plugin for the necessary storage is not scary and not difficult, and the problem of stability concerns will disappear.
Scala Developer Development and Functional Programming
The report of Olga Makhasoeva is the only one in soft skills and it is in the FP block. How did it happen: soft skills are not as important as technical skills or the community is not ready for such reports?
At the first Scala conference, we cannot afford to make many lectures on soft skills. It will be a waste of time and resources. It turns out that we can replace people who can tell us about something technical and useful with reports that we can listen to at other conferences.
At the same time, we wanted to add keynote to the program and Olga helped us out. Her report on soft skills - “The Art of Asking Questions ” —was perfect. He stands with reports on AF, because there is no way to create a separate block for it.
I would like to propose a hypothesis that Scala developers are more technically oriented.
Scala has a lot of technical things to talk about and discover. In mainstream languages, you can quickly learn a language, get acquainted with the ecosystem and reach the technical limit. Further develop only in management, in soft skills or to study work at a low level. In Scala and FP, there is a lot of room for deepening: how to achieve more guarantees from the code, how to write it more correctly, in short, faster and more reliably, to express it more concisely.
Scala has many techniques and approaches that never end.
You can study related fields, read other languages and add the best to Scala. We are expanding our horizons to be more likely to find something useful and interesting. There was no conference last year wherever they talked about Tagless Final. A year has passed, everyone has cooled down and are looking at the approach that is used in ZIO. The flow of new and fresh to study in the FP and Scala never ends. If you look deep enough, even within the framework of one pull request Scala, you can dig for the whole day.
The main news resource is specialized chats, public Twitter accounts like the Scala Center , and individual blogs, like the Akka or Scala-lang team, which publish posts on important language solutions. However, there is an incredible amount of information, it's hard to follow everything. The only resource that saves is the Scala Times . This is a weekly newsletter with a digest of posts, releases, tweets and news in the Scala world and community. I have been subscribed to the Scala Times for three years now. But I go very rarely, I look through the headers and a brief description. When needed, I can remember that I once saw this and search, but no more.
At one time, I tried to develop the Scalanews.org digest. We tried to work with information as journalists: we studied various sources, and not only the Scala Times, read, disassembled, wrote eyeliners. We wanted the reader to benefit from the article and remember something. But the information is sea, and the project is gradually fading.
In Scala, you can study everything indefinitely, but are there any steps for growth?
There has long been a gradation system for Scala. It is not known who invented it, but it describes the levels of the developer: a confident user, a library writer, and more. On a special page of the John LambdaConf conference, levels of mastery of concepts from 6 to 8 levels were painted. There are levels, but I'm not sure that they are in use.
I also had in mind the gradation in the June, Middle, Senior. Is there such a thing?
Yes, but not common. If in vacancies they write a tag, then this is a senior. Rarely is a middle or even a junior. Often, developers who already wrote in other languages go to Scala. They switched to Scala with experience in development, and this is clearly not a june.
Advanced developers move to Scala.
Hence the high average age of the developer.
English-language reports: scandals, recursion, API
There are four foreign speakers in the functional programming block. Let's start with John A De Hus.
Everyone knows John.
John is very active: attends all events, is engaged in his ZIO library and is actively promoting it, he recently performed at Scala Meetup. John promotes AF and he succeeds. He is a master of attention grabbing.
As a result, he participated in many scandals. For example, in 2019, several world dramas happened: ZIO and Scalaz were removed from the Scala build, Typelevel.scala forbade John to participate in their projects, and Skills Matter blocked the entrance to all their events. Travis Brown, who is associated with Typelevel.scala, released a post in which he accused John of all sins. These events were discussed even by people far from Scala, for example, in the Radio-T podcast.
In a sense, John is doing everything right - is in full view and can preach AF to the masses. He is currently organizing his Scala conference in London at the end of 2019 and has already invited many cool speakers. At ScalaConf 2019, he will perform with the Snake Oils & Elixirs keynote .
Interesting story of John with Scalaz Sparta. He launched a marathon in which he offered mentoring, organizing and helping create ecosystems for Scalaz to developers who are not involved in Open Source. 300 people responded, working groups of interests were formed from them, 30 repositories were opened. Almost nothing fired. It is unlikely that people will just get together and will remain motivated for a long time to write libraries that may not be needed in their current work.
Valentin Kassas also participated in the marathon. Perhaps his project will be released successfully - he is still involved in it. Valentine's “ report ” will deal with recursion patterns. This is a topic for advanced, but few people understand them. For years, developers have been trying to overcome schemes. Perhaps Valentine will finally help to understand this topic.
Functional programming conference without mentioning recursion schemes - money down the drain.
The third speaker is Adam Warski . He participates in both Open Source projects and commercial ones, as a co-founder of a software development company. The last year has been releasing many libraries, by API, for example. He is working on them qualitatively, promoting in the community. Everyone who follows the life of the Scala community should know him.
His talk will be about the Tapir library. It helps a lot with the Swagger documentation. It usually hurts to write Swagger. The best thing we had from this was a terrible monstrous construction on a heap of annotations that might still not match what you wrote there.
Tapir solves this issue and gives the Swagger description for free. The only similar project is for Oleg Nizhnikov from ScalaConf 2019 PC. But Tapir is more PR, because Adam is proficient in this.
But the quality of the product was not affected. You have to make big compromises, it’s convenient to organize an API to cover all cases and your fingers will not hook. It seems that Adam succeeded.
After learning the language and solving practical problems, the Scala developer looks towards the FP. Over the past 4 years, most reports at foreign conferences have been related to the FI and its approaches to solving practical problems. Therefore, we are interested in foreign speakers - they talk about FP, mainly.
Evgeny Tokarev, member of PC ScalaConf 2019
The last foreign speaker is Wojtek Pitula . He works for Sony in the information security department. In the report “ Magic of integrations - ecosystem better than any framework ”, he will tell how he created an application in which he placed all working libraries.
Foreign reports will be in English. Will there be a translation, subtitles?
No, there will be nothing. The developer is constantly confronted with technical terms, reads documentation, manuals, studies in English - there are all the primary sources.
We assume that all of us understand English.
If it is completely incomprehensible, then there will be a code on the screen and from it it is possible to understand what is happening. If not at all, then there is always an alternative report in Russian in the next room.
Just about the alternative. Will Daniil Smirnov also talk about schemes, like Valentin?
Nearly. They both write libraries that are related to schemas. Both use the same source of inspiration - Xenomorph . But libraries are completely different, like a set of features. The library of Daniel is simpler and in working order - take and use. Valentine has more bells and whistles, her own specific DSL, more features and she went much further from Xenomorph.
Daniel will show his Morphling project. But the project can scare away an unprepared person: spreading types with holes, strange phrases like “HMutu” - horror. Therefore, the purpose of the report is to overcome this fear and discover the simplicity and ease of use of schemes, even in this form.
For experimental purposes, we deliberately reduced the number of reports to 20 minutes. Daniel's report is one of them. I attended a recent run of Daniel, and I can say that he did a good job with the report due to time constraints. The themes of Daniel and Valentine intersect, but go at different times to cover the whole topic of recursions and schemas.
Alexandra Golubitskaya compares ZIO and Future. This is John's product. Are they coordinated?
Oh sure. Alexandra is actively involved in writing ZIO, they help each other with reports. When John shows some feature at conferences, he thanks Alexander for the preparation. She is also one of the organizers of the event in London. Now they are preparing a new feature in ZIO - chunks. If it is ready for the date of the conference, then Alexandra will talk about them .
Next up is Artem Seleznev and Scala Without JDBC.
JDBC is an old standard that is used in JVM languages for working with relational databases. A specific implementation is written for a specific database. JDBC is not developing, it is a fixed standard and no one can refuse it. Everyone spits, gets upset, but continues to chew on a cactus - there is no alternative. There are libraries that give asynchronous drivers, but they are not popular. Rob Norris has heard Skunk library , but it has been in development for more than a year and is unclear when it will be released.
Artyom will tell how to get down with JDBC. To do this, he wrote his drivers "on the knee" - the tools are enough to do it yourself and in a functional style. In the report, he will show how to write drivers, and that “on the knee” is no more difficult and no worse than everyone is used to thinking.
JetBrains, Holivars and Big Data
In the next block “Language / Tuning” is the first report by Andrey Kozlov and Mikhail Mutsianko. Will they talk about one of the JetBrains products? In the program, I counted 3 reports from 18 from speakers from JetBrains. Is it connected with something?
I also noticed this when I started making the schedule grid. This happened by chance and no one intentionally promoted JetBrains. The company employs many people who are close to AF, although their main products are not about that. Andrey Kozlov and Mikhail Mutsianko are working on the Scala Plugin, which we all use. They are definitely not superfluous here.
There is an old problem that the Scala Plugin does not quite match the Scala compiler. An alternative Scala code compiler is written in the plugin for their IDEs to work. Therefore, there is a discrepancy in the product - in some places the code is practically valid, correct, compiled by the Scala compiler, but in the IDE it is red. This is frustrating, but over time you get used to it, there are fewer red zones.
Obviously, the situation is not good, but the Scala compiler is not the easiest thing. Most developers are not aware of the issues and challenges facing IDE developers. Most likely, this situation will continue further, but Andrey Kozlov and Mikhail Mutsianko just want to tell their point of view. They want to explain why this happens, why it should be, what are the requirements for the compiler, why write it and why the plugin is written that way. There were attempts to uncover this secret, they came to us to record a podcast, but it’s still not clear.
Continuing on JetBrains and a little jump to Big Data. Vitaly Khudobakhshov will present a certain “harvester” for data engineers - in one tool you can immediately switch between tabs, terminal and IDE. What is this tool?
This is a new product for those who work with Big Data. Traditionally, this area is not very convenient. You have to work with an incredible amount of software: databases, remote servers, visualization, machine layout, synchronization. Improving developer convenience is a space field to work with. Vitaly's report will be about a product that should seriously reduce this level of pain for data engineers and solve many problems.
The third speaker from JetBrains is Vitaly Bragilevsky. At AppsConf, he made a historical breakthrough in functional programming. At ScalaConf 2019, is the theory awaiting us too?
Yes, Vitaliy is known as “Professor Haskell” and will talk about the DOT theory - a type system with dependent objects. What is the DOT model, elements and general principles of type theory. We can say that he is the only "saboteur" from Haskell at a conference about Scala. But, in a sense, he was preparing for this - for an unfriendly reception.
Will the reception be unfriendly?
No, everyone will be happy to see him :)
How so, is there a “holivar” between Haskell and Scala?
No, they just don’t like each other. Scala from Java is a breath of fresh air. It will also be if you look at Haskell from the side of Scala. Haskell has a more powerful compiler (not everywhere), and Scala has places to write ugly to maintain functionality.
Between languages there is a circulation of developers. A lot of functionality in Scala is brought from Haskell and is successfully used.
But the "holivar" has developed historically. People need to be divided into groups in order to hate someone together :)
Interestingly, only Bragilevsky, an adherent of Haskell, is probably the only person in Russia who has the competence to talk about the theory that they are trying to bring under the Scala language.
Will Vladimir Polushin also have a theory on static analysis in Scala?
No, it will not be a theory. Vladimir is perfect for this topic. He participated in many projects related to static analysis, was engaged in ensime, scalafix , fastparse. Scalafix is just more about analysis and Vladimir has gained good experience in this regard.
Vladimir wrote his analyzer, which at the time of the application was not in Open Source, but now is in the public domain . Vladimir will talk about the current problems of static analyzers, about his analyzer and how he came to write it.
Roman Grebennikov will "carry" GraalVM. You used this tool, what do you say?
GraalVM is a tool for writing native code. Now this is impossible with us - there is a Scala Native in an incomprehensible state, but in Java there is nothing at all. In recent years, this topic has been actively promoted at all conferences that are associated with the JVM.
We wanted to invite someone who is associated with GraalVM - Oleg Shilaev . Oleg did not succeed and he offered his colleagues, but they are English-speaking. Such reports are already enough. At that moment, Roman just offered us his report. He had experience with GraalVM, but if you look at the description , you can see that the experience is not entirely positive. Therefore, we are not getting the report that we could get from GraalVM developers. However, it is no less interesting, and perhaps even more. The novel is an experienced speaker and his reports are always fire.
If Roman makes a good presentation, will GraalVM developers have to make excuses next year?
Maybe. But the point is not only that GraalVM is evil in the flesh. It has certain limitations and they are declared by the developers. Some things just don't work due to limitations - GraalVM is still experimental. The programs themselves must be adapted to run it.
Everyone expects someone to try GraalVM and the code will run incredibly fast and magically. But few people try and talk about their experiences. Judging by Roman’s report, GraalVM was not the Holy Grail. But perhaps Roman’s conditions did not fit and he tried to do what was impossible on him. But you will learn this already at the conference.
A report by Mikhail Limansky from the company First PFD on the same topic as a report by Andrey Korzinev? They both talk about data mining, but in the Big Data block?
No, these are completely different reports and different tasks. Report by Michael from the Big Data field, he works in the first OFD and processes the data that comes from cash registers. So their data size is orders of magnitude higher than in the case of Auto.ru.
They wrote Yupana , a solution on top of HBase for working with time-series data to cover their specific analytic building issues. The tool is written for a common platform, it allows you to connect many fiscal data operators and get information between them all. In addition to technological tasks, there are also business ones - to make friends with other OFDs with Yupana so that each operator can build useful reports for customers, getting more information. The solution is already in Open Source and now you can collect all OFD into one large network.
At ScalaConf 2019 there will be technical hardcore and a feast of functional programming. The conference is only three weeks old, but the schedule is ready and you can choose your route of visiting the holiday with lunch and a coffee break. November 12 - Deadline for raising conference prices. Book your tickets, and pay later. Meet me at ScalaConf 2019!