Continuous delivery for your Kotlin Multiplatform library

Hello! My name is Yuri Vlad, I am an Android developer at Badoo and take part in creating the Reaktive library - Reactive Extensions on pure Kotlin.

In the process, we are faced with the fact that in the case of Kotlin Multiplatform continuous integration and continuous delivery require additional configuration. It is necessary to have several virtual machines on various operating systems in order to assemble the library completely. In this article, I will show how to configure continuous delivery for your Kotlin Multiplatform library.

Continuous integration and continuous delivery for open-source libraries

Continuous integration and continuous delivery have long been a part of the open-source community thanks to various services. Many of them provide their services to open-source projects for free: Travis CI, JitPack, CircleCI, Microsoft Azure Pipelines, the recently launched GitHub Actions.

In Badoo's open-source projects for Android, we use Travis CI for continuous integration and JitPack for continuous delivery.

After implementing iOS support in our multi-platform library, I found that we cannot build the library using JitPack, because it does not provide virtual machines on macOS (iOS can only be built on macOS).

Therefore, for further publication of the library, Bintray , more familiar to everyone, was chosen . It supports finer tuning of published artifacts, unlike JitPack, which simply took all the results of the publishToMavenLocal



call.

For publication, it is recommended to use the Gradle Bintray Plugin, which I later customized to our needs. And to build the project, I continued to use Travis CI for several reasons: firstly, I was already familiar with it and used it in almost all of my pet projects; secondly, it provides the macOS virtual machines needed to build on iOS.

Parallel assembly of a multi-platform library

If you delve into the bowels of the Kotlin documentation, you can find a section on publishing multi-platform libraries.

Kotlin Multiplatform developers are aware of the problems of multi-platform assembly (not everything can be assembled on any operating system) and offer to assemble the library separately on different operating systems.

 kotlin { jvm() js() mingwX64() linuxX64() // Note that the Kotlin metadata is here, too. // The mingwx64() target is automatically skipped as incompatible in Linux builds. configure([targets["metadata"], jvm(), js()]) { mavenPublication { targetPublication -> tasks.withType(AbstractPublishToMaven) .matching { it.publication == targetPublication } .all { onlyIf { findProperty("isLinux") == "true" } } } } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

As you can see from the code above, depending on the isLinux



property passed to Gradle, we isLinux



publication of certain targets. Under the targets in the future I will mean the assembly for a specific platform. On Windows, only the Windows target will be collected, while on other operating systems metadata and other targets will be collected.

A very beautiful and concise solution that works only for publishToMavenLocal



or publish



from the maven-publish



plugin, which is not suitable for us due to the use of the Gradle Bintray Plugin .

I decided to use the environment variable to select the target, since this code was previously written in Groovy, lay in a separate Groovy Gradle script, and access to the environment variables is from a static context.

 enum class Target { ALL, COMMON, IOS, META; val common: Boolean @JvmName("isCommon") get() = this == ALL || this == COMMON val ios: Boolean @JvmName("isIos") get() = this == ALL || this == IOS val meta: Boolean @JvmName("isMeta") get() = this == ALL || this == META companion object { @JvmStatic fun currentTarget(): Target { val value = System.getProperty("MP_TARGET") return values().find { it.name.equals(value, ignoreCase = true) } ?: ALL } } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

As part of our project, I identified four groups of targets:

1. ALL - all targets are connected and collected, used for development and as the default value.
2. COMMON - only Linux-compatible targets are connected and collected. In our case, these are JavaScript, JVM, Android JVM, Linux x64, and Linux ARM x32.
3. IOS - only iOS targets are connected and assembled, used to build on macOS.
4. META - all targets are connected, but only the module with meta-information for Gradle Metadata is assembled.

With this set of target groups, we can parallelize the assembly of the project into three different virtual machines (COMMON - Linux, IOS - macOS, META - Linux).

At the moment, you can build everything on macOS, but my solution has two advantages. First, if we decide to implement support for Windows, we just need to add a new target group and a new virtual machine on Windows to build it. Secondly, there is no need to waste virtual machine resources on macOS on what you can build on Linux. CPU time on such virtual machines is usually twice as expensive.

Gradle metadata

What is Gradle Metadata and what is it for?

Maven currently uses POM (Project Object Model) to resolve dependencies.

 <?xml version="1.0" encoding="UTF-8"?> <project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <modelVersion>4.0.0</modelVersion> <groupId>com.jakewharton.rxbinding2</groupId> <artifactId>rxbinding-leanback-v17-kotlin</artifactId> <version>2.2.0</version> <packaging>aar</packaging> <name>RxBinding Kotlin (leanback-v17)</name> <description>RxJava binding APIs for Android's UI widgets.</description> <url>https://github.com/JakeWharton/RxBinding/</url> <licenses> <license> <name>The Apache Software License, Version 2.0</name> <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url> <distribution>repo</distribution> </license> </licenses> <developers> <developer> <id>jakewharton</id> <name>Jake Wharton</name> </developer> </developers> <scm> <connection>scm:git:git://github.com/JakeWharton/RxBinding.git</connection> <developerConnection>scm:git:ssh://git@github.com/JakeWharton/RxBinding.git</developerConnection> <url>https://github.com/JakeWharton/RxBinding/</url> </scm> <dependencies> <dependency> <groupId>com.android.support</groupId> <artifactId>support-annotations</artifactId> <version>28.0.0</version> <scope>compile</scope> </dependency> </dependencies> </project>
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The POM file contains information about the version of the library, its creator and the necessary dependencies.

But what if we want to have two versions of the library for different JDKs? For example, kotlin-stdlib



has two versions: kotlin-stdlib-jdk8



and kotlin-stdlib-jdk7



. Users need to connect the desired version.

When upgrading the JDK version, it's easy to forget about external dependencies. It was to solve this problem that Gradle Metadata was created, which allows you to add additional conditions for connecting a particular library.

One of the supported Gradle Metadata attributes is org.gradle.jvm.version



, which indicates the version of the JDK. Therefore, for kotlin-stdlib



a simplified form of a metadata file might look like this:

 { "formatVersion": "1.0", "component": { "group": "org.jetbrains.kotlin", "module": "kotlin-stdlib", "version": "1.3.0" }, "variants": [ { "name": "apiElements", "attributes": { "org.gradle.jvm.version": 8 }, "available-at": { "url": "../../kotlin-stdlib-jdk8/1.3.0/kotlin-stdlib-jdk8.module", "group": "org.jetbrains.kotlin", "module": "kotlin-stdlib-jdk8", "version": "1.3.0" } }, { "name": "apiElements", "attributes": { "org.gradle.jvm.version": 7 }, "available-at": { "url": "../../kotlin-stdlib-jdk7/1.3.0/kotlin-stdlib-jdk7.module", "group": "org.jetbrains.kotlin", "module": "kotlin-stdlib-jdk7", "version": "1.3.0" } } ] }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Specifically, in our case, reaktive-1.0.0-rc1.module



in a simplified form looks like this:

 { "formatVersion": "1.0", "component": { "group": "com.badoo.reaktive", "module": "reaktive", "version": "1.0.0-rc1", "attributes": { "org.gradle.status": "release" } }, "createdBy": { "gradle": { "version": "5.4.1", "buildId": "tv44qntk2zhitm23bbnqdngjam" } }, "variants": [ { "name": "android-releaseRuntimeElements", "attributes": { "com.android.build.api.attributes.BuildTypeAttr": "release", "com.android.build.api.attributes.VariantAttr": "release", "com.android.build.gradle.internal.dependency.AndroidTypeAttr": "Aar", "org.gradle.usage": "java-runtime", "org.jetbrains.kotlin.platform.type": "androidJvm" }, "available-at": { "url": "../../reaktive-android/1.0.0-rc1/reaktive-android-1.0.0-rc1.module", "group": "com.badoo.reaktive", "module": "reaktive-android", "version": "1.0.0-rc1" } }, { "name": "ios64-api", "attributes": { "org.gradle.usage": "kotlin-api", "org.jetbrains.kotlin.native.target": "ios_arm64", "org.jetbrains.kotlin.platform.type": "native" }, "available-at": { "url": "../../reaktive-ios64/1.0.0-rc1/reaktive-ios64-1.0.0-rc1.module", "group": "com.badoo.reaktive", "module": "reaktive-ios64", "version": "1.0.0-rc1" } }, { "name": "linuxX64-api", "attributes": { "org.gradle.usage": "kotlin-api", "org.jetbrains.kotlin.native.target": "linux_x64", "org.jetbrains.kotlin.platform.type": "native" }, "available-at": { "url": "../../reaktive-linuxx64/1.0.0-rc1/reaktive-linuxx64-1.0.0-rc1.module", "group": "com.badoo.reaktive", "module": "reaktive-linuxx64", "version": "1.0.0-rc1" } }, ] }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Thanks to the attributes org.jetbrains.kotlin



Gradle understands in which case which dependency to pull into the desired source set.

You can enable metadata using:

 enableFeaturePreview("GRADLE_METADATA")
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

You can find detailed information in the documentation .

Publish Setting

After we figured out the targets and parallelization of the assembly, we need to configure what exactly and how we will publish.

For publishing, we use the Gradle Bintray Plugin, so the first thing to do is turn to its README and set up information about our repository and credentials for publication.

We will perform the entire configuration in our own plugin in the buildSrc



folder.

Using buildSrc



provides several advantages, including an always-working autocomplete (in the case of Kotlin scripts, it still does not always work and often requires a call to apply dependencies), the ability to reuse classes declared in it and access them from Groovy and Kotlin scripts. You can see an example of using buildSrc



from the latest Google I / O (Gradle section).

 private fun setupBintrayPublishingInformation(target: Project) { //  Bintray Plugin   target.plugins.apply(BintrayPlugin::class) //    target.extensions.getByType(BintrayExtension::class).apply { user = target.findProperty("bintray_user")?.toString() key = target.findProperty("bintray_key")?.toString() pkg.apply { repo = "maven" name = "reaktive" userOrg = "badoo" vcsUrl = "https://github.com/badoo/Reaktive.git" setLicenses("Apache-2.0") version.name = target.property("reaktive_version")?.toString() } } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

I use the three dynamic properties of the project: bintray_user



and bintray_key



, which can be obtained from the personal profile settings on Bintray , and reaktive_version



, which is set in the root build.gradle



file.

For each target, the Kotlin Multiplatform Plugin creates a MavenPublication , which is available in PublishingExtension .

Using the sample code from the Kotlin documentation that I provided above, we can create this configuration:

 private fun createConfigurationMap(): Map<String, Boolean> { val mppTarget = Target.currentTarget() return mapOf( "kotlinMultiplatform" to mppTarget.meta, KotlinMultiplatformPlugin.METADATA_TARGET_NAME to mppTarget.meta, "jvm" to mppTarget.common, "js" to mppTarget.common, "androidDebug" to mppTarget.common, "androidRelease" to mppTarget.common, "linuxX64" to mppTarget.common, "linuxArm32Hfp" to mppTarget.common, "iosArm32" to mppTarget.ios, "iosArm64" to mppTarget.ios, "iosX64" to mppTarget.ios ) }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

In this simple map, we describe which publications should be released on a particular virtual machine. The publication name is the name of the target. This configuration is fully consistent with the description of target groups that I cited above.

Let's set up publishing in Bintray. The Bintray plugin creates a BintrayUploadTask



, which we will customize to our needs.

 private fun setupBintrayPublishing( target: Project, taskConfigurationMap: Map<String, Boolean> ) { target.tasks.named(BintrayUploadTask.getTASK_NAME(), BintrayUploadTask::class) { doFirst { //   } } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Everyone who starts working with the Bintray plugin quickly discovers that his repository has been covered with moss for a long time (the last update was about six months ago), and that all problems are solved by all sorts of hacks and crutches in the Issues tab. We did not implement support for such a new technology as Gradle Metadata, but in the corresponding issue you can find a solution that we use.

 val publishing = project.extensions.getByType(PublishingExtension::class) publishing.publications .filterIsInstance<MavenPublication>() .forEach { publication -> val moduleFile = project.buildDir.resolve("publications/${publication.name}/module.json") if (moduleFile.exists()) { publication.artifact(object : FileBasedMavenArtifact(moduleFile) { override fun getDefaultExtension() = "module" }) } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Using this code, we add the module.json



file to the list of artifacts for publication, due to which Gradle Metadata works.

But our problems do not end there. When you try to run bintrayPublish



nothing happens.

For regular Java and Kotlin libraries, Bintray automatically pulls up available publications and publishes them. However, in the case of Kotlin Multiplatform, when automatically pulling up publications, the plugin simply crashes with an error. And yes, there’s also an issue on GitHub for this. And we will again use the solution from there, only by filtering the publications we need.

 val publications = publishing.publications .filterIsInstance<MavenPublication>() .filter { val res = taskConfigurationMap[it.name] == true logger.warn("Artifact '${it.groupId}:${it.artifactId}:${it.version}' from publication '${it.name}' should be published: $res") res } .map { logger.warn("Uploading artifact '${it.groupId}:${it.artifactId}:${it.version}' from publication '${it.name}'") it.name } .toTypedArray() setPublications(*publications)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

But this code doesn't work either!

This is because bintrayUpload



does not have a task in the dependencies that would assemble the project and create the files necessary for publication. The most obvious solution is to set publishToMavenLocal



as a publishToMavenLocal



dependency, but not so simple.

When collecting metadata, we connect all the targets to the project, which means publishToMavenLocal



will compile all the targets, since the dependencies for this task include publishToMavenLocalAndroidDebug



, publishToMavenLocalAndroiRelase



, publishToMavenLocalJvm



, etc.

Therefore, we will create a separate proxy task, depending on which we put only those publishToMavenLocalX



that we need, and we will put this task itself in bintrayPublish



dependencies.

 private fun setupLocalPublishing( target: Project, taskConfigurationMap: Map<String, Boolean> ) { target.project.tasks.withType(AbstractPublishToMaven::class).configureEach { val configuration = publication?.name ?: run { // Android-       PublishToMaven,        val configuration = taskConfigurationMap.keys.find { name.contains(it, ignoreCase = true) } logger.warn("Found $configuration for $name") configuration } //          enabled = taskConfigurationMap[configuration] == true } } private fun createFilteredPublishToMavenLocalTask(target: Project) { //  -         publishToMavenLocal target.tasks.register(TASK_FILTERED_PUBLISH_TO_MAVEN_LOCAL) { dependsOn(project.tasks.matching { it is AbstractPublishToMaven && it.enabled }) } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

All that remains is to collect all the code together and apply the resulting plug-in to a project in which publication is required.

 abstract class PublishPlugin : Plugin<Project> { override fun apply(target: Project) { val taskConfigurationMap = createConfigurationMap() createFilteredPublishToMavenLocalTask(target) setupLocalPublishing(target, taskConfigurationMap) setupBintrayPublishingInformation(target) setupBintrayPublishing(target, taskConfigurationMap) }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

 apply plugin: PublishPlugin
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The full PublishPlugin



code can be found in our repository here .

Configure Travis CI

The hardest part is already behind. It remains to configure Travis CI so that it parallelizes the assembly and publishes artifacts to Bintray when a new version is released.

We will designate the release of the new version by creating a tag on the commit.

 #    ( ) matrix: include: #  Linux  Android  Chrome    JS, JVM, Android JVM  Linux - os: linux dist: trusty addons: chrome: stable language: android android: components: - build-tools-28.0.3 - android-28 #  MP_TARGET,        env: MP_TARGET=COMMON #     install — Gradle     install: true #    JVM       RxJava2 script: ./gradlew reaktive:check reaktive-test:check rxjava2-interop:check -DMP_TARGET=$MP_TARGET #  macOS   iOS- - os: osx osx_image: xcode10.2 language: java env: MP_TARGET=IOS install: true script: ./gradlew reaktive:check reaktive-test:check -DMP_TARGET=$MP_TARGET #  Linux    - os: linux language: android android: components: - build-tools-28.0.3 - android-28 env: MP_TARGET=META #     -  install: true script: true #    Gradle (           ) before_cache: - rm -f $HOME/.gradle/caches/modules-2/modules-2.lock - rm -fr $HOME/.gradle/caches/*/plugin-resolution/ cache: directories: - $HOME/.gradle/caches/ - $HOME/.gradle/wrapper/ #  ,   Kotlin/Native    - $HOME/.konan/ #     Bintray    ,           deploy: skip_cleanup: true provider: script script: ./gradlew bintrayUpload -DMP_TARGET=$MP_TARGET -Pbintray_user=$BINTRAY_USER -Pbintray_key=$BINTRAY_KEY on: tags: true
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

If for some reason the assembly on one of the virtual machines failed, then the metadata and other targets will still be uploaded to the Bintray server. That is why we do not add a block with automatic release of the library on Bintray through their API.

When releasing the version, you need to make sure that everything is in order, and just click on the button to publish the new version on the site, since all artifacts are already uploaded.

Conclusion

So we set up continuous integration and continuous delivery in our Kotlin Multiplatform project.

Having parallelized the tasks of assembling, running tests, and publishing artifacts, we effectively use the resources provided to us on a free basis.

And if you use Linux (like Arkady Ivanov arkivanov , the author of the Reaktive library), then you no longer need to ask the person using macOS (me) to publish the library each time a new version is released.

I hope that after the release of this article, more projects will begin to use this approach to automate routine activities.

Thanks for your attention!