There will be no immutable collections in Java - neither now nor ever
Hello!
Today, your attention is invited to the translation of a thoughtfully written article about one of the basic problems of Java - mutability, and how it affects the structure of data structures and how to work with them. The material is taken from the blog of Nicolai Parlog, whose brilliant literary style we really tried to keep in translation. Nicholas himself is remarkably characterized by an excerpt from the blog of the company JUG.ru on Habré; let us cite this passage in its entirety:
We read and vote. Who especially likes the post - we also recommend that you look at the readers' comments on the original post.
Volatility is bad, right? Accordingly, immutability is good . The main data structures in which immutability is especially fruitful are collections: in Java it is a list (
), a set (
) and a dictionary (
). However, although the JDK comes with immutable (or non-modifiable?) Collections, the type system knows nothing about this. There is no
in the JDK, and this type from Guava seems completely useless to me. But why? Why not just add
to this mix and not say that it should be?
In JDK terminology, the meanings of the words “ immutable ” and “ unmodifiable ” have changed over the past few years. Initially, “non-modifiable” was called an instance that did not allow mutability (mutability): in response to changing methods, he threw
. However, it could be changed in a different way - maybe because it was just a wrapper around a mutable collection. These views are reflected in the methods
,
and
, as well as in their JavaDoc .
Initially, the term " immutable " refers to collections returned by the factory methods of Java 9 collections . The collections themselves could not be changed in any way (yes, there is reflection , but it does not count), therefore, it seems that they justify their name. Alas, confusion often arises because of this. Suppose there is a method that displays all the elements from an immutable collection on the screen - will it always give the same result? Yes? Or not?
If you didn’t immediately answer no , it means that you’ve just been dawned on what kind of confusion is possible here. An “ immutable collection of secret agents ” - it would seem to sound pretty damn similar to a “ immutable collection of immutable secret agents, ” but these two entities may not be identical. An immutable collection cannot be edited using insert / delete / clean operations, etc., but if secret agents are mutable (though character traits in spy movies are so bad that they don’t really believe it), then that doesn’t mean it that the entire collection of secret agents is unchangeable. Therefore, now there is a shift in the direction of naming such collections non-modifiable , not immutable , which is enshrined in the new edition of JavaDoc .
The immutable collections discussed in this article may contain mutable elements.
Personally, I do not like such a revision of terminology. In my opinion, the term “unchangeable collection” should only mean that the collection itself is not amenable to change, but should not characterize the elements contained in it. In this case, there is one more positive point: the term “immutability” in the Java ecosystem does not turn into complete nonsense.
One way or another, in this article we will talk about immutable collections , where ...
Let's dwell on the fact that now we will practice in adding immutable collections. To be precise - an immutable list. Everything that will be said about lists is equally applicable to collections of other types.
Getting started adding immutable collections!
Let's create the
interface and make it, relative to
, uh ... what? Supertype or subtype? Let's dwell on the first option.
Beautifully,
no
methods, so using it is always safe, right? So?! No with.
This example shows that it is possible to transfer such a not entirely immutable list to the API, the operation of which can be built on immutability and, accordingly, nullify all guarantees that a name of this type may hint at. Here's a recipe for you that could lead to disaster.
OK, then
extends
. May be?
Now, if the API expects an immutable list, then it will receive such a list, but there are two drawbacks:
Thus, it turns out that you can only use the
locally, because it passes the API boundaries as a
, which requires you to superhuman level of precaution, or explodes at runtime. It's not as bad as a
extending an
, but such a solution is still far from ideal.
This is exactly what I had in mind when I said that the
type from Guava is practically useless. This is a great code sample, very reliable when working with local immutable lists (that's why I use it actively), but, resorting to it, it is very easy to go beyond the impregnable, guaranteed compiled citadel, the walls of which were composed of immutable types - and only in this immutable types can fully reveal their potential. This is better than nothing, but inefficient as a JDK solution.
If the
cannot extend the
, and the workaround still does not work, then how is it supposed to make it all work?
The problem that we encountered in the first two attempts to add immutable types was our misconception that immutability is simply the absence of something: we take a
, remove the changing code from it, we get an
. But, in fact, all this does not work that way.
If we simply remove the modifying methods from the
, we get a read-only list. Or, adhering to the terminology formulated above, it can be called
- it can still change, just you will not change it.
Now we can add two more things to this picture:
Immutability is not the absence of mutability, but a guarantee that there will be no changes
In this case, it is important to understand that in both cases we are talking about full features - immutability is not the absence of changes, but a guarantee that there will be no changes. An existing feature may not necessarily be something that is used for good, it can also guarantee that something bad will not happen in the code - in this case, think, for example, about thread safety.
Obviously, mutability and immutability conflict with each other, and therefore we cannot simultaneously use the two above-mentioned inheritance hierarchies. Types inherit capabilities from other types, therefore, no matter how you cut them, if one of the types inherits from the other, it will contain both features.
So well,
and
cannot extend each other. But we were brought here by working with
, and it really turns out that both types have the same API, read-only, which means they should extend it.
Although, I would not call things exactly these names, the hierarchy of this kind is reasonable. In Scala, for example, this is practically what is being done . The difference is that the shared supertype, which we called
, defines mutable methods that return a modified collection and leave the original one untouched. Thus, an immutable list turns out to be persistent and gives a mutable variant two sets of modifying methods - inherited to receive modified copies and its own for changes in place.
What about Java? Is it possible to modernize such a hierarchy by adding new supertypes and siblings to it?
Is it possible to improve non-modifiable and unchangeable collections?
Of course, there is no problem in adding the
and
and creating the inheritance hierarchy described above. The problem is that in the short and medium term it will be practically useless. Let me explain.
The
to have
,
and
as types - in which case the APIs will be able to clearly express what they need and what they offer.
However, unless you start the project from scratch, you will likely have such functionality, and it will look something like this:
This is not good, because in order for the new collections just introduced by us to be useful, we need to work with them! (phew). What is shown above resembles the application code, therefore, refactoring
and
, and you can implement it, as was shown in the above listing. This can be a big chunk of work, coupled with confusion when you need to organize the interaction of old and updated code, but at least it seems feasible.
What about frameworks, libraries, and the JDK itself? Everything here looks bleak. Attempting to change a parameter or return type from
to
will result in incompatibility with the source code , i.e. existing source code will not compile with the new version, since these types are not related to each other. Similarly, changing the return type from
to a new supertype
will result in compilation errors.
With the introduction of new types, it will be necessary to make changes and recompile throughout the ecosystem.
However, even if we expand the parameter type from
to
, we will run into a problem, since such a change causes incompatibility at the bytecode level . When the source code calls the method, the compiler converts this call to bytecode that references the target method by:
Any change in the parameter or return type of the method will cause the bytecode to indicate an invalid signature when referring to the method; as a result, a
error will occur during execution. If the change you make is compatible with the source code — for example, if you are talking about narrowing the return type or expanding the type of the parameter — then recompilation should be sufficient. However, with far-reaching changes, for example, with the introduction of new collections, everything is not so simple: in order to consolidate such changes, you need to recompile the entire Java ecosystem. This is a lost business.
The only conceivable way to take advantage of such new collections without breaking compatibility is to duplicate existing methods each with a new name, change the API, and then mark the old version as undesirable. Can you imagine how monumental and virtually infinite such a task would be ?!
Of course, immutable collection types are a great thing I'd love to have, but we are unlikely to see anything like this in the JDK. Competent implementations of
and
never expand each other (in fact, they both extend the same list type
, read-only), which makes it difficult to implement such types in existing APIs.
Outside the context of any specific relationships between types, changing existing method signatures is always fraught with problems, since such changes are incompatible with bytecode. When they are introduced, at least recompilation is required, and this is a devastating change that will affect the entire Java ecosystem.
Therefore, I believe that nothing like this will happen - never and never.
Today, your attention is invited to the translation of a thoughtfully written article about one of the basic problems of Java - mutability, and how it affects the structure of data structures and how to work with them. The material is taken from the blog of Nicolai Parlog, whose brilliant literary style we really tried to keep in translation. Nicholas himself is remarkably characterized by an excerpt from the blog of the company JUG.ru on Habré; let us cite this passage in its entirety:
Nikolay Parlog is such a media dude who does reviews on Java features. But he is not from Oracle, so the reviews are surprisingly frank and understandable. Sometimes after them someone is fired, but rarely. Nikolay will talk about the future of Java, what will be in the new version. He is good at talking about trends and generally about the big world. He is a very well-read and erudite companion. Even simple presentations are pleasant to listen to, all the time you learn something new. Moreover, Nicholas knows beyond what he is telling. That is, you can come to any report and just enjoy it, even if this is not your topic at all. He is teaching. He wrote “The Java Module System” for Manning Publishing House, maintains blogs on software development at codefx.org, and has long been involved in several open source projects. He can be hired right at the conference, he’s a freelancer. True, a very expensive freelancer. Here is the report .
We read and vote. Who especially likes the post - we also recommend that you look at the readers' comments on the original post.
Volatility is bad, right? Accordingly, immutability is good . The main data structures in which immutability is especially fruitful are collections: in Java it is a list (
List
), a set (
Set
) and a dictionary (
Map
). However, although the JDK comes with immutable (or non-modifiable?) Collections, the type system knows nothing about this. There is no
ImmutableList
in the JDK, and this type from Guava seems completely useless to me. But why? Why not just add
Immutable...
to this mix and not say that it should be?
What is an immutable collection?
In JDK terminology, the meanings of the words “ immutable ” and “ unmodifiable ” have changed over the past few years. Initially, “non-modifiable” was called an instance that did not allow mutability (mutability): in response to changing methods, he threw
UnsupportedOperationException
. However, it could be changed in a different way - maybe because it was just a wrapper around a mutable collection. These views are reflected in the methods
Collections::unmodifiableList
,
unmodifiableSet
and
unmodifiableMap
, as well as in their JavaDoc .
Initially, the term " immutable " refers to collections returned by the factory methods of Java 9 collections . The collections themselves could not be changed in any way (yes, there is reflection , but it does not count), therefore, it seems that they justify their name. Alas, confusion often arises because of this. Suppose there is a method that displays all the elements from an immutable collection on the screen - will it always give the same result? Yes? Or not?
If you didn’t immediately answer no , it means that you’ve just been dawned on what kind of confusion is possible here. An “ immutable collection of secret agents ” - it would seem to sound pretty damn similar to a “ immutable collection of immutable secret agents, ” but these two entities may not be identical. An immutable collection cannot be edited using insert / delete / clean operations, etc., but if secret agents are mutable (though character traits in spy movies are so bad that they don’t really believe it), then that doesn’t mean it that the entire collection of secret agents is unchangeable. Therefore, now there is a shift in the direction of naming such collections non-modifiable , not immutable , which is enshrined in the new edition of JavaDoc .
The immutable collections discussed in this article may contain mutable elements.
Personally, I do not like such a revision of terminology. In my opinion, the term “unchangeable collection” should only mean that the collection itself is not amenable to change, but should not characterize the elements contained in it. In this case, there is one more positive point: the term “immutability” in the Java ecosystem does not turn into complete nonsense.
One way or another, in this article we will talk about immutable collections , where ...
- The instances contained in the collection are determined at the design stage
- These copies - an even amount, neither reduce nor add
- No statements are made regarding the mutability of these elements.
Let's dwell on the fact that now we will practice in adding immutable collections. To be precise - an immutable list. Everything that will be said about lists is equally applicable to collections of other types.
Getting started adding immutable collections!
Let's create the
ImmutableList
interface and make it, relative to
List
, uh ... what? Supertype or subtype? Let's dwell on the first option.
Beautifully,
ImmutableList
no
ImmutableList
methods, so using it is always safe, right? So?! No with.
List<Agent> agents = new ArrayList<>(); // , `List` `ImmutableList` ImmutableList<Agent> section4 = agents; // section4.forEach(System.out::println); // `section4` agents.add(new Agent("Motoko"); // "Motoko" – , ?! section4.forEach(System.out::println);
This example shows that it is possible to transfer such a not entirely immutable list to the API, the operation of which can be built on immutability and, accordingly, nullify all guarantees that a name of this type may hint at. Here's a recipe for you that could lead to disaster.
OK, then
ImmutableList
extends
List
. May be?
Now, if the API expects an immutable list, then it will receive such a list, but there are two drawbacks:
- Immutable lists should still offer modifying methods (as they are defined in the supertype), and the only possible implementation will throw an exception
-
ImmutableList
instancesImmutableList
alsoList
instances, and when assigned to such a variable, passed as such an argument, or returned of this type, it is logical to assume that mutability is allowed.
Thus, it turns out that you can only use the
ImmutableList
locally, because it passes the API boundaries as a
List
, which requires you to superhuman level of precaution, or explodes at runtime. It's not as bad as a
List
extending an
ImmutableList
, but such a solution is still far from ideal.
This is exactly what I had in mind when I said that the
ImmutableList
type from Guava is practically useless. This is a great code sample, very reliable when working with local immutable lists (that's why I use it actively), but, resorting to it, it is very easy to go beyond the impregnable, guaranteed compiled citadel, the walls of which were composed of immutable types - and only in this immutable types can fully reveal their potential. This is better than nothing, but inefficient as a JDK solution.
If the
ImmutableList
cannot extend the
List
, and the workaround still does not work, then how is it supposed to make it all work?
Immutability is a feature
The problem that we encountered in the first two attempts to add immutable types was our misconception that immutability is simply the absence of something: we take a
List
, remove the changing code from it, we get an
ImmutableList
. But, in fact, all this does not work that way.
If we simply remove the modifying methods from the
List
, we get a read-only list. Or, adhering to the terminology formulated above, it can be called
UnmodifiableList
- it can still change, just you will not change it.
Now we can add two more things to this picture:
- We can make it mutable by adding appropriate methods
- We can make it immutable by adding appropriate warranties.
Immutability is not the absence of mutability, but a guarantee that there will be no changes
In this case, it is important to understand that in both cases we are talking about full features - immutability is not the absence of changes, but a guarantee that there will be no changes. An existing feature may not necessarily be something that is used for good, it can also guarantee that something bad will not happen in the code - in this case, think, for example, about thread safety.
Obviously, mutability and immutability conflict with each other, and therefore we cannot simultaneously use the two above-mentioned inheritance hierarchies. Types inherit capabilities from other types, therefore, no matter how you cut them, if one of the types inherits from the other, it will contain both features.
So well,
List
and
ImmutableList
cannot extend each other. But we were brought here by working with
UnmodifiableList
, and it really turns out that both types have the same API, read-only, which means they should extend it.
Although, I would not call things exactly these names, the hierarchy of this kind is reasonable. In Scala, for example, this is practically what is being done . The difference is that the shared supertype, which we called
UnmodifiableList
, defines mutable methods that return a modified collection and leave the original one untouched. Thus, an immutable list turns out to be persistent and gives a mutable variant two sets of modifying methods - inherited to receive modified copies and its own for changes in place.
What about Java? Is it possible to modernize such a hierarchy by adding new supertypes and siblings to it?
Is it possible to improve non-modifiable and unchangeable collections?
Of course, there is no problem in adding the
UnmodifiableList
and
ImmutableList
and creating the inheritance hierarchy described above. The problem is that in the short and medium term it will be practically useless. Let me explain.
The
ImmutableList
to have
UnmodifiableList
,
ImmutableList
and
List
as types - in which case the APIs will be able to clearly express what they need and what they offer.
public void payAgents(UnmodifiableList<Agent> agents) { // , // } public void sendOnMission(ImmutableList<Agent> agents) { // ( ), // , } public void downtime(List<Agent> agents) { // , // , } public UnmodifiableList<Agent> teamRoster() { // , , // , - } public ImmutableList<Agent> teamOnMission() { // , } public List<Agent> team() { // , , // }
However, unless you start the project from scratch, you will likely have such functionality, and it will look something like this:
// , `Iterable<Agent>` // , , public void payAgents(List<Agent> agents) { } public void sendOnMission(List<Agent> agents) { } public void downtime(List<Agent> agents) { } // , // , `List` public List<Agent> teamRoster() { } // , `Stream<Agent>` public List<Agent> teamOnMission() { } public List<Agent> team() { }
This is not good, because in order for the new collections just introduced by us to be useful, we need to work with them! (phew). What is shown above resembles the application code, therefore, refactoring
ImmutableList
UnmodifiableList
and
ImmutableList
, and you can implement it, as was shown in the above listing. This can be a big chunk of work, coupled with confusion when you need to organize the interaction of old and updated code, but at least it seems feasible.
What about frameworks, libraries, and the JDK itself? Everything here looks bleak. Attempting to change a parameter or return type from
List
to
ImmutableList
will result in incompatibility with the source code , i.e. existing source code will not compile with the new version, since these types are not related to each other. Similarly, changing the return type from
List
to a new supertype
UnmodifiableList
will result in compilation errors.
With the introduction of new types, it will be necessary to make changes and recompile throughout the ecosystem.
However, even if we expand the parameter type from
List
to
UnmodifiableList
, we will run into a problem, since such a change causes incompatibility at the bytecode level . When the source code calls the method, the compiler converts this call to bytecode that references the target method by:
- The name of the class whose instance the target is declared
- Method name
- Method parameter types
- Method return type
Any change in the parameter or return type of the method will cause the bytecode to indicate an invalid signature when referring to the method; as a result, a
NoSuchMethodError
error will occur during execution. If the change you make is compatible with the source code — for example, if you are talking about narrowing the return type or expanding the type of the parameter — then recompilation should be sufficient. However, with far-reaching changes, for example, with the introduction of new collections, everything is not so simple: in order to consolidate such changes, you need to recompile the entire Java ecosystem. This is a lost business.
The only conceivable way to take advantage of such new collections without breaking compatibility is to duplicate existing methods each with a new name, change the API, and then mark the old version as undesirable. Can you imagine how monumental and virtually infinite such a task would be ?!
Reflection
Of course, immutable collection types are a great thing I'd love to have, but we are unlikely to see anything like this in the JDK. Competent implementations of
List
and
ImmutableList
never expand each other (in fact, they both extend the same list type
UnmodifiableList
, read-only), which makes it difficult to implement such types in existing APIs.
Outside the context of any specific relationships between types, changing existing method signatures is always fraught with problems, since such changes are incompatible with bytecode. When they are introduced, at least recompilation is required, and this is a devastating change that will affect the entire Java ecosystem.
Therefore, I believe that nothing like this will happen - never and never.
All Articles