I would like to talk about continuous integration and delivery for mobile applications using fastlane. How we implement CI / CD on all mobile applications, how we went about it and what happened in the end.
The network already has enough material on the instrument, which we so lacked at the start, so I will not intentionally describe the instrument in detail, but only refer to what we had then:
- Official fastlane documentation
- Examples of other companies
- We automate the assembly of iOS applications using Fastlane
The article consists of two parts:
- Background of the emergence of mobile CI / CD in the company
- Technical solution for rolling CI / CD on N-applications
The first part is more nostalgia for the old days, and the second is experience that you can apply at home.
So historically
Year 2015
We just started developing mobile applications, then we still did not know anything about continuous integration, about DevOps and other fashionable things. Each update of the application was rolled out by the developer from his machine. And if for Android it is quite simple - assembled, signed .apk
and uploaded it to the Google Developer Console, then for iOS the distribution tool through Xcode left us gorgeous evenings - attempts to download the archive often ended in errors and had to be tried again. It turned out that the most pumped-up developer several times a month does not write code, but is engaged in the release of the application.
Year 2016
We grew up, we already had thoughts on how to free developers from the whole day for release, and a second application also appeared, which only pushed us towards automation. In the same year, we first installed Jenkins and wrote a bunch of ugly scripts, very similar to those shown by fastlane in our documentation.
$ xcodebuild clean archive -archivePath build/MyApp \ -scheme MyApp $ xcodebuild -exportArchive \ -exportFormat ipa \ -archivePath "build/MyApp.xcarchive" \ -exportPath "build/MyApp.ipa" \ -exportProvisioningProfile "ProvisioningProfileName" $ cd /Applications/Xcode.app/Contents/Applications/Application\ Loader.app/Contents/Frameworks/ITunesSoftwareService.framework/Versions/A/Support/ $ ./altool —upload-app \ -f {abs path to your project}/build/{release scheme}.ipa \ -u "appleId@example.com" \ -p "PASS_APPLE_ID"
Unfortunately, only our developers so far knew how these scripts work and why this endless bundle of keys is needed, and when something broke down again, they got "chic evenings" for analyzing logs.
Year 2017
This year we learned that there is such a thing as fastlane. There was not as much information as it is now - how to get it, how to use it. And the instrument itself was still damp at that time: constant errors only disappointed us and it was hard to believe in the magic automation that they promised.
However, the main utilities included in the fastlane core, gym
and pilot
, we managed to get.
Our scripts are slightly ennobled.
$ fastlane gym —-workspace "Example.xcworkspace" --scheme "AppName" —-buildlog_path "/tmp" -—clean
Ennobled, if only because not all the parameters necessary for xcodebuild
need to be specified - gym
will independently understand where and what lies. And for finer tuning, you can specify the same keys as in xcodebuild
, only key naming is more understandable.
This time, thanks to the gym and the built-in xcpretty formatter, the build logs are much more legible. This began to save time on fixing broken assemblies, and sometimes the release team could figure it out on their own.
Unfortunately, we did not take measurements on the build speed of xcodebuild
and gym
, but we will believe the documentation - up to 30% acceleration.
A single process for all applications
Year 2018 and Present
By 2018, the process of assembling and rolling out applications completely moved to Jenkins, the developers stopped releasing from their machines, only the release team had the right to release.
We already wanted to tighten up the launch of tests and static analysis, and our scripts grew and grew. Grew and changed with our applications. There were about 10 applications at that time. Considering that we have two platforms, these are about 20 “living” scripts.
Each time we wanted to add a new step to the script, we had to copy-paste the pieces into all shell scripts. Perhaps it was possible to work more carefully, but often such changes ended with typos, which already turned into evenings of the release team to fix the scripts and find out which of the wise men added this command and what it does at all. In general, it cannot be said that the scripts for assembly under one platform were at least somewhat similar. Although they certainly did the same thing.
In order to start the process for a new application, you had to spend a day to select the “fresh” version from these scripts, debug it and say “yes, it works”.
In the summer of 2018, we once again looked towards the still developing fastlane.
Task number 1: summarize all the steps of the scripts and rewrite them in Fastfile
When we started, our scripts looked like a footcloth from all the steps and crutches in one shell script in Jenkins. We have not yet switched to pipeline and dividing by stage.
We looked at what is and highlighted 4 steps that fit the description of our CI / CD:
- build - install dependencies, build the archive,
- test - launch unit developer tests, coverage calculation,
- sonar - launch all linters and send reports to SonarQube,
- deploy - sending an artifact to alpha (TestFlight).
And if you don’t go into details, omit the keys used by actions, you get such a Fastfile:
default_platform(:ios) platform :ios do before_all do unlock end desc "Build stage" lane :build do match prepare_build gym end desc "Prepare build stage: carthage and cocoapods" lane :prepare_build do pathCartfile = "" Dir.chdir("..") do pathCartfile = File.join(Dir.pwd, "/Cartfile") end if File.exist?(pathCartfile) carthage end pathPodfile = "" Dir.chdir("..") do pathPodfile = File.join(Dir.pwd, "/Podfile") end if File.exist?(pathPodfile) cocoapods end end desc "Test stage" lane :test do scan xcov end desc "Sonar stage (after run test!)" lane :run_sonar do slather lizard swiftlint sonar end desc "Deploy to testflight stage" lane :deploy do pilot end desc "Unlock keychain" private_lane :unlock do pass = ENV['KEYCHAIN_PASSWORD'] unlock_keychain( password: pass ) end end
In fact, the first Fastfile turned out to be monstrous, taking into account some crutches that we still needed, and the number of parameters that we substituted:
lane :build do carthage( command: "update", use_binaries: false, platform: "ios", cache_builds: true) cocoapods( clean: true, podfile: "./Podfile", use_bundle_exec: false) gym( workspace: "MyApp.xcworkspace", configuration: "Release", scheme: "MyApp", clean: true, output_directory: "/build", output_name: "my-app.ipa") end lane :deploy do pilot( username: "appleId@example.com", app_identifier: "com.example.app", dev_portal_team_id: "TEAM_ID_NUMBER_DEV", team_id: "ITS_TEAM_ID") end
In the example above, only a part of the parameters we need to specify: these are the assembly parameters - the scheme, configuration, Provision Profile names, as well as distribution parameters - the Apple developer account ID, password, application identifier and so on. In a first approximation, we put all these keys in special files - Gymfile
, Matchfile
and Appfile
.
Now in Jenkins you can call short commands that do not "blur" the look and are well read by the eye:
# fastlane ios <lane_name> $ fastlane ios build $ fastlane ios test $ fastlane ios run_sonar $ fastlane ios deploy
Hooray, we are great
What did you get? Clear commands for every step. Combed scripts neatly laid out in fastlane files. Rejoicing, we ran to the developers with a request to add everything we needed to our repositories.
But in time we realized that we would face the same difficulties - we will still have 20 build scripts that will somehow begin to live their own lives, it will be more difficult to edit them, as the scripts will move to the repositories, and we don’t have access there. And, in general, to solve our pain in this way will not work.
Problem number 2: get a single Fastfile for N-applications
Now it seems that solving the problem is not so difficult - set the variables, and let's go. Yes, actually, that’s how the problem was solved. But at that moment when we screwed it in, we had neither expertise in fastlane itself, nor in Ruby, on which fastlane is written, nor useful examples on the network - everyone who wrote about fastlane then was limited to an example for one application for one the developer.
Fastlane can do environment variables, and we already tried this by setting a password for Keychain:
ENV['KEYCHAIN_PASSWORD']
Looking at our scripts, we highlighted the common parts:
#for build, test and deploy APPLICATION_SCHEME_NAME=appScheme APPLICATION_PROJECT_NAME=app.xcodeproj APPLICATION_WORKSPACE_NAME=app.xcworkspace APPLICATION_NAME=appName OUTPUT_IPA_NAME=appName.ipa #app info APP_BUNDLE_IDENTIFIER=com.example.appName APPLE_ID=appleID@example.com TEAM_ID=ABCD1234 FASTLANE_ITC_TEAM_ID=123456789
Now, in order to start using these keys in fastlane's files, you had to figure out how to get them there. Fastlane has a solution for this: loading variables through dotenv . The documentation says that if it is important for you to load keys for different purposes, spawn several .env
, .env.default
, .env.development
configuration files in the .env
.env.development
.
And then we decided to use this library a little differently. We will not place the fastlane scripts and its meta information in the developer repository, but the unique keys of this application in the .env.appName
file.
Fastfile
, Appfile
, Matchfile
and Gymfile
, we hid in a separate repository. They also hid an additional file with passwords from other services - .env
.
An example can be seen here .
On CI, the call has not changed much, a configuration key for a specific application has been added:
# fastlane ios <lane_name> --env appName $ fastlane ios build --env appName $ fastlane ios test --env appName $ fastlane ios run_sonar --env appName $ fastlane ios deploy --env appName
Before running the commands, we load our repository with scripts. It doesn't look so pretty:
git clone git@repository.com/FastlaneCICD.git fastlane_temp cp ./fastlane_temp/fastlane/* ./fastlane/ cp ./fastlane_temp/fastlane/.env fastlane/.env
So far they have left this solution, although Fastlane has a solution for loading Fastfile via action import_from_git
, but it works only for Fastfile, but for the rest of the files it doesn’t. If you want to "direct quite beautifully", you can write your action
.
A similar set was made for Android applications and ReactNative, the files are in the same repository, but in different branches of iOS
, android
and react_native
.
When the release team wants to add some new step, changes to the script are fixed through MR in git, you no longer need to look for the culprits of the broken scripts, and in general - break now, it should be tried.
Now everything is for sure
Previously, we spent time supporting all scripts, updating them, and fixing all the consequences of updates. It was very disappointing when the causes of the errors and downtime of the releases were simple typos, which are so difficult to keep track of in the hash of the shell script. Now, such errors are minimized. Changes are rolled immediately to all applications. And it takes 15 minutes to get a new application into the process - set up a template pipeline on CI and add keys to the developer's repository.
It seems that the item with Fastfile for Android and the signature of applications remained unlit, if the article is interesting, I will write a sequel. I will be glad to your questions or suggestions "how would you solve this problem" in the comments or in the Telegram bashkirova .