Article about how to automate android app size regression detection on CI.

We, as developers, want to make our application available to many users so that they can download and install our applications without any problems. And one of the metrics related to this goal is the size of our application — a large app size can cause problems when downloading and installing on devices with limited memory or slow internet. And the more we reduce the app size, the higher the install conversion becomes, as detailed in a separate article. Therefore, we would like to not only optimize the size, but also be able to monitor the size change and be able to detect the increase in size before it hit production.

When it comes to a large team, it is very difficult to control the changes taking place in the project, as there are many feature commands that add cool features that can later affect the size. Therefore, it is best to make this control automated, and our special flow on CI will help us with this.

App size regression detection in development branch

We will start with the simplest, we will check app size in development branch, since it is the only source of truth for us, since we create our release branches from development.

Our first idea was — what if we collect a release build for each commit (pull request merge) in development and collect size metrics. But we immediately realized that this is not the best for a quick solution to this issue, since if 2 commits are simultaneously pushed to development, then it will be difficult to compare these 2 apks with each other, since the release build takes a decent amount of time. Therefore, we found an easier way.

1. Suppose we select a specific commit at a specific time and run a release build from it using the familiar gradle command:

./gradlew assembleRelease

As a result, from the built apk, we collect and upload metrics (apk size, current used commit hash, app version, date, etc., in json format) to any storage (gcloud), the main thing is that we can upload our artifacts there and upload back when needed. We also download the assembled release apk to storage, we will need it in the subsequent app size check. Using the gsutil command line tool:

gsutil cp "$build_size_info.json" "gs://custom_folder/build_size_info.json"
gsutil cp "$current-release.apk" "gs://custom_folder/current-release.apk"

2. Select the next specific commit in development, after some time, for example, after 3 hours (up to you). We also collect a release build on this commit, and also upload apk, metrics. But now we need to download the previous build from the storage, which we collected 3 hours earlier, and compare the previous and current apks. But just comparing apk sizes is not enough, we would like a detailed comparison in which we can find out which files have been added or removed. The diffuse tool by Jake Wharton can compare apk, aab, jar, aar files and show detailed comparison information.

java -jar diffuse-binary.jar diff $prev.apk $current.apk > $report.txt
gsutil cp "$report.txt" "gs://custom_folder/report.txt"

Write the result of the comparison to the report.txt file and upload it to storage so that we can check it later by clicking on the link. If you show the contents of report.txt, it looks like:

The file contains more details about the changed resources (added, removed), but information may be enough to understand what served as a regression. We are interested in the diff column, which shows how much the files inside the apk have been changed. In this case, we see that the arsc(resources.arsc) file has been significantly changed, which stores all compiled resources (drawable/.xml, layout/, raw/, values/etc.). And we understand that someone merged pull request in development, in which compiled resources were added, and we can find this commit in git history.

3. To make it easier to find the commit that served as a regression, we can collect all involved commits between commits when the app size has been checked. To do this, we call git command:

git rev-list --ancestry-path "$prev_commit_hash".."$current_commit_hash"

Result

As a result, our CI workflow will look like:

To run a check every 3 hours, you can assign a task in your CI, or run it depending on the time (depends on the capabilities of your CI).

If size regression is found, you can notify the team with a link (report.txt in gcloud) to detailed comparison information, in the consequences, the responsible developer can identify the cause and find the specific commit in development that served as the regression and fix or revert the changes.

Drawbacks

1. Late regression detection — we catch regression only after the changes have already been merged into development

2. Requires manual work — even so, this is not a fully automated process, since as soon as a regression occurs, the developer has to check the result of comparison and look for a specific commit, inside all involved commits.

3. We build a universal apk — which contains resources for all device configurations, which affects app size. This problem is solved inside Google Play, where we need to upload the App Bundle, and apk is built inside depending on the configuration of a specific device.

We will fix all these drawbacks in the next part of the article.

Size does matter… (part 1)