So all the same RAML or OAS (Swagger)?
In the dynamic world of microservices, anything can change - any component can be rewritten in another language using different frameworks and architecture. Only contracts should remain unchanged, so that it is possible to interact with the microservice from the outside on some permanent basis, regardless of internal metamorphoses. And today we will talk about our problem of choosing a format for describing contracts and share found artifacts.
The post was prepared by Anna Melekhova and Vladimir Lapatin
Microservices. When developing Acronis Cyber Cloud, we realized that we could not get away from them. And microservice design is impossible without formalizing a contract, which is a microservice interface.
But when a product contains more than one component, and the development of a contract becomes regular activity, you involuntarily begin to think about optimizing the process. It becomes obvious that the interface (contract) and implementation (microservice) must correspond to each other, that different components must do the same things the same, and that without a centralized adoption of all these decisions, each team will be forced to spend time getting them again and again .
Amazon microservice layout from Werner Vogelis tweet , Amazon CTO
What is the dilemma? De facto, there are two ways that microservices interact - HTTP Rest and gRPC from Google. Not wanting to get involved in the Google technology stack, we chose HTTP Rest. Annotations to HTTP REST contracts are most often described in one of two formats: RAML and OAS, formerly known as Swagger. Therefore, each development team is faced with the need to choose one of the standards. But, as it turned out, making this choice can be very difficult.
Annotation is needed so that an external user can easily figure out what can be done with your service through its HTTP interface. That is, at a basic level, the annotation should contain at least a list of available resources, their HTTP methods, request bodies, an enumeration of parameters, an indication of the necessary and supported headers, as well as return codes and response formats. An extremely important element of the contract annotation is their verbal description (“what happens if you add this query parameter to the request?”, “In which case will the 400 code return?”)
Nevertheless, when it comes to developing a large number of microservices, I want to derive additional benefit from the written annotations. For example, based on RAML / Swagger, you can generate both client and server code in a huge number of programming languages. You can also automatically receive documentation for the microservice and upload it to your developer-portal :).
An example of a structured contract description
Less common is the practice of testing microservices based on contract descriptions. If you wrote both an annotation and a component, then you can create an autotest that checks the adequacy of the service with various types of input data. Does the service return a response code that is not described in the annotation? Will it be able to correctly process knowingly incorrect data?
Moreover, the high-quality implementation of not only the contracts themselves, but also the tools for visualizing annotations makes it easier to work with the microservice. That is, if the architect qualitatively described the contract, on its basis designers and developers will implement the service in other products without additional time costs.
For the operation of additional tools, both RAML and OAS have the ability to add metadata that are not provided by the standard ( for example, this is done in OAS ).
In general, the field for creativity in the application of contracts for microservices is huge ... at least theoretically
Currently, Acronis' priority development area is the development of the Acronis Cyber Platform. Acronis Cyber Platform is a new point of integration of third-party services with Acronis Cyber Cloud and the agent part. Although our internal APIs described in RAML were fine with us, the need to publish the API again raised the question of choice: which standard of annotations is better to use for our work?
Initially, it seemed that there were two solutions - these were the most common developments of RAML and Swagger (or OAS). But in fact it turned out that the alternatives are at least not 2, but 3 or more.
On the one hand there is RAML - a powerful and effective language. It implements hierarchy and inheritance well, so this format is more suitable for large companies that need a lot of descriptions - that is, not one product, but many microservices that have common parts of contracts - authentication schemes, the same data types, error bodies.
But the RAML developer, Mulesoft, has joined the Open API consortium that is developing Swagger . Therefore, RAML suspended its development. To imagine the format of the event, imagine that the maintainers of the main Linux components have gone to work at Microsoft. This situation creates the prerequisites for using Swagger, which is developing dynamically and in the latest - third version - practically catches up with RAML in terms of flexibility and functionality.
As it turned out, not all open-source utilities have been updated to OAS 3.0. For microservices on Go, the most critical will be the lack of adaptation of go-swagger to the latest version of the standard. However, the difference between Swagger 2 and Swagger 3 is huge . For example, in the third version, the developers:
The situation is funny: when choosing a standard, you need to consider RAML, Swagger 2 and Swagger 3 as separate alternatives. However, only Swagger 2 has good support for the OpenSource toolkit. RAML is very flexible ... and complex, and Swagger 3 is poorly supported by the community, so you have to use proprietary tools or commercial solutions, which are usually very expensive.
At the same time, if there are many nice features in Swagger, such as the ready-made portal editor.swagger.io , on which you can download the annotation and get its visualization with a detailed description, links and links, then there is no such possibility for a more fundamental and less friendly RAML. Yes, you can search for something among the projects on GitHub, find an analog there and deploy it yourself. However, in any case, someone will have to support the portal, which is not so convenient for basic use or test needs. In addition, swagger is more “unprincipled”, well, or liberal - it can be generated from comments in the code, which, of course, runs counter to the principle of API first and is not supported by any of the RAML utilities,
At one time, we started working with RAML as a more flexible language, and as a result we had to do a lot with our own hands. For example, one of the projects uses the ramlfications utility in unit tests, which only supports RAML 0.8. So I had to add crutches so that the utility could “eat” RAML version 1.0.
Having gotten involved in adding an ecosystem of solutions for RAML, we came to the conclusion that we need to convert RAML to Swagger 2 and already carry out all the automation, verification, testing and subsequent optimization in it. This is a good way to leverage both the flexibility of RAML and community tool support from Swagger.
To solve this problem, there are two OpenSource tools that should ensure the conversion of contracts:
And this is not all the difficulties that we have encountered. One of the steps in our pipeline is to verify that the RAML from the repository is correct with respect to the specification. We tried several utilities. Surprisingly, they all cursed at our annotations in different places and with completely different bad words. And it’s not always the case :).
In the end, we settled on the now obsolete project, which also has a number of problems (sometimes it falls out of the blue, has problems when working with regular expressions). Thus, we did not find a way to solve the validation and conversion tasks based on free tools, and decided to use a commercial utility. In the future, when OpenSource tools become more developed, the solution to this problem may become easier. In the meantime, the time and labor involved in “finishing” seemed to us more significant than the cost of a commercial service.
After all this, we wanted to share our experience and note that before choosing a tool for describing contracts, you need to clearly determine what you want from it and what budget you are ready to invest. If you forget about OpenSource, now there are a large number of services and products that will help you to check, convert, validate. But they are expensive, and sometimes very expensive. For a large company, such costs are tolerable, but for a startup, it can become a big burden.
Define a set of tools that you will use later. For example, if you just need to display the contract, it will be easier to use Swagger 2, which has a beautiful API, because in RAML you have to lift and maintain the service yourself.
The more tasks you have, the wider the need for tools, and they will be different for different platforms, and it is better to immediately familiarize yourself with the available versions to make a choice that minimizes your costs in the future.
But it is worth recognizing that all ecosystems that exist today are imperfect. Therefore, if the company has fans who like to work in RAML, because “it allows you to express your thoughts more flexibly”, or, on the contrary, prefer Swagger, because “it is more understandable”, it is best to leave them to work in what they they’re used to and want to, because the toolkit of any of the formats requires file refinement.
As for our experience, in the following posts we will talk about what kind of static and dynamic checks we carry out based on our RAML-Swagger architecture, as well as what documentation we generate from contracts, and how it all works.
The post was prepared by Anna Melekhova and Vladimir Lapatin
Microservices. When developing Acronis Cyber Cloud, we realized that we could not get away from them. And microservice design is impossible without formalizing a contract, which is a microservice interface.
But when a product contains more than one component, and the development of a contract becomes regular activity, you involuntarily begin to think about optimizing the process. It becomes obvious that the interface (contract) and implementation (microservice) must correspond to each other, that different components must do the same things the same, and that without a centralized adoption of all these decisions, each team will be forced to spend time getting them again and again .
Amazon microservice layout from Werner Vogelis tweet , Amazon CTO
What is the dilemma? De facto, there are two ways that microservices interact - HTTP Rest and gRPC from Google. Not wanting to get involved in the Google technology stack, we chose HTTP Rest. Annotations to HTTP REST contracts are most often described in one of two formats: RAML and OAS, formerly known as Swagger. Therefore, each development team is faced with the need to choose one of the standards. But, as it turned out, making this choice can be very difficult.
Why annotations?
Annotation is needed so that an external user can easily figure out what can be done with your service through its HTTP interface. That is, at a basic level, the annotation should contain at least a list of available resources, their HTTP methods, request bodies, an enumeration of parameters, an indication of the necessary and supported headers, as well as return codes and response formats. An extremely important element of the contract annotation is their verbal description (“what happens if you add this query parameter to the request?”, “In which case will the 400 code return?”)
Nevertheless, when it comes to developing a large number of microservices, I want to derive additional benefit from the written annotations. For example, based on RAML / Swagger, you can generate both client and server code in a huge number of programming languages. You can also automatically receive documentation for the microservice and upload it to your developer-portal :).
An example of a structured contract description
Less common is the practice of testing microservices based on contract descriptions. If you wrote both an annotation and a component, then you can create an autotest that checks the adequacy of the service with various types of input data. Does the service return a response code that is not described in the annotation? Will it be able to correctly process knowingly incorrect data?
Moreover, the high-quality implementation of not only the contracts themselves, but also the tools for visualizing annotations makes it easier to work with the microservice. That is, if the architect qualitatively described the contract, on its basis designers and developers will implement the service in other products without additional time costs.
For the operation of additional tools, both RAML and OAS have the ability to add metadata that are not provided by the standard ( for example, this is done in OAS ).
In general, the field for creativity in the application of contracts for microservices is huge ... at least theoretically
Hedgehog comparison with snake
Currently, Acronis' priority development area is the development of the Acronis Cyber Platform. Acronis Cyber Platform is a new point of integration of third-party services with Acronis Cyber Cloud and the agent part. Although our internal APIs described in RAML were fine with us, the need to publish the API again raised the question of choice: which standard of annotations is better to use for our work?
Initially, it seemed that there were two solutions - these were the most common developments of RAML and Swagger (or OAS). But in fact it turned out that the alternatives are at least not 2, but 3 or more.
On the one hand there is RAML - a powerful and effective language. It implements hierarchy and inheritance well, so this format is more suitable for large companies that need a lot of descriptions - that is, not one product, but many microservices that have common parts of contracts - authentication schemes, the same data types, error bodies.
But the RAML developer, Mulesoft, has joined the Open API consortium that is developing Swagger . Therefore, RAML suspended its development. To imagine the format of the event, imagine that the maintainers of the main Linux components have gone to work at Microsoft. This situation creates the prerequisites for using Swagger, which is developing dynamically and in the latest - third version - practically catches up with RAML in terms of flexibility and functionality.
If not for one but ...
As it turned out, not all open-source utilities have been updated to OAS 3.0. For microservices on Go, the most critical will be the lack of adaptation of go-swagger to the latest version of the standard. However, the difference between Swagger 2 and Swagger 3 is huge . For example, in the third version, the developers:
- improved description of authentication schemes
- completed support for JSON Schema
- pumped the ability to add examples
The situation is funny: when choosing a standard, you need to consider RAML, Swagger 2 and Swagger 3 as separate alternatives. However, only Swagger 2 has good support for the OpenSource toolkit. RAML is very flexible ... and complex, and Swagger 3 is poorly supported by the community, so you have to use proprietary tools or commercial solutions, which are usually very expensive.
At the same time, if there are many nice features in Swagger, such as the ready-made portal editor.swagger.io , on which you can download the annotation and get its visualization with a detailed description, links and links, then there is no such possibility for a more fundamental and less friendly RAML. Yes, you can search for something among the projects on GitHub, find an analog there and deploy it yourself. However, in any case, someone will have to support the portal, which is not so convenient for basic use or test needs. In addition, swagger is more “unprincipled”, well, or liberal - it can be generated from comments in the code, which, of course, runs counter to the principle of API first and is not supported by any of the RAML utilities,
At one time, we started working with RAML as a more flexible language, and as a result we had to do a lot with our own hands. For example, one of the projects uses the ramlfications utility in unit tests, which only supports RAML 0.8. So I had to add crutches so that the utility could “eat” RAML version 1.0.
Do I need to choose?
Having gotten involved in adding an ecosystem of solutions for RAML, we came to the conclusion that we need to convert RAML to Swagger 2 and already carry out all the automation, verification, testing and subsequent optimization in it. This is a good way to leverage both the flexibility of RAML and community tool support from Swagger.
To solve this problem, there are two OpenSource tools that should ensure the conversion of contracts:
- oas-raml-converter is now an unsupported utility. In the process of working with her, we found that she has a number of problems with complex RAMLs that are “spread out” over a large number of files. This program is written in JavaScript and performs a recursive traversal of the syntax tree. Due to dynamic typing, it becomes difficult to understand this code, so we decided not to waste time writing patches for a dying utility.
- webapi-parser is a tool from the same company, which claims to be ready to convert everything and everything, and in any direction. To date, support for RAML 0.8, RAML 1.0, and Swagger 2.0 has been announced. However, at the time of our study, the utility was EXTREMELY crude and unusable. Developers create a kind of IR , which will allow them to quickly add new standards in the future. But for now, all this just doesn't work.
And this is not all the difficulties that we have encountered. One of the steps in our pipeline is to verify that the RAML from the repository is correct with respect to the specification. We tried several utilities. Surprisingly, they all cursed at our annotations in different places and with completely different bad words. And it’s not always the case :).
In the end, we settled on the now obsolete project, which also has a number of problems (sometimes it falls out of the blue, has problems when working with regular expressions). Thus, we did not find a way to solve the validation and conversion tasks based on free tools, and decided to use a commercial utility. In the future, when OpenSource tools become more developed, the solution to this problem may become easier. In the meantime, the time and labor involved in “finishing” seemed to us more significant than the cost of a commercial service.
Conclusion
After all this, we wanted to share our experience and note that before choosing a tool for describing contracts, you need to clearly determine what you want from it and what budget you are ready to invest. If you forget about OpenSource, now there are a large number of services and products that will help you to check, convert, validate. But they are expensive, and sometimes very expensive. For a large company, such costs are tolerable, but for a startup, it can become a big burden.
Define a set of tools that you will use later. For example, if you just need to display the contract, it will be easier to use Swagger 2, which has a beautiful API, because in RAML you have to lift and maintain the service yourself.
The more tasks you have, the wider the need for tools, and they will be different for different platforms, and it is better to immediately familiarize yourself with the available versions to make a choice that minimizes your costs in the future.
But it is worth recognizing that all ecosystems that exist today are imperfect. Therefore, if the company has fans who like to work in RAML, because “it allows you to express your thoughts more flexibly”, or, on the contrary, prefer Swagger, because “it is more understandable”, it is best to leave them to work in what they they’re used to and want to, because the toolkit of any of the formats requires file refinement.
As for our experience, in the following posts we will talk about what kind of static and dynamic checks we carry out based on our RAML-Swagger architecture, as well as what documentation we generate from contracts, and how it all works.
All Articles