Mobile applications testing

Theory of testing is usually differs from practice. Today we want to talk about our practical experience in testing application's code which is used by millions of iOS users and about the difficult path that our team finished in order to achieve stable code.

Let’s imagine: the developers successfully convinced themselves and the business of the need to cover the code with tests. Over time, in the project were created more than a dozen thousand unit- and more than a thousand UI-tests. Such a large test base araised several problems, and we successfully found the solution for them.

We have been working on our current QA process for about two years, and we still keep improving it. This process may seem obvious, but when we started to implement it in a new team that consisted entirely of new developers, we realized that it was difficult to do right away. Many people are used to working differently and need to make a lot of changes at once to switch, which is difficult. At the same time, it is ill-advised to implement this process in parts, because it can negatively affect the quality.

What do we do? We need to do preliminary preparation for each block of the development process: task decomposition, evaluation and planning, development itself, investigative testing, and release. This preparation does not consist of simply throwing old parts out of the process but of their adequate replacement, which increases quality.

In this article, I will talk in detail about our testing process at each stage of creating a new feature and also about the introduced changes that have increased the quality and speed of development.

The ITMO Accelerator continues to empower entrepreneurs. We’ve already covered some of its most successful graduates — the Laeneco staethoscope and the GoROBO robotics club. Today we’ll be discussing Hamstand — a modular hub that simplifies the process of mobile app testing. Let’s talk about how it came to be, why it’s a welcome addition to many developers’ toolkits, and what awaits the project in the near future.

In our previous article, we discussed the reasons of unit-tests’ instability and how to make them stable. Now let’s look through a new tools for debugging and profiling which were introduced by Apple in iOS 12 — the framework os_log and instrument for performance analysis os_signpost.



In one of the sprints, we were tasked with implementing the generation of a pdf-document on the client-side. We completed the task. But we wanted to make sure the effectiveness of the technical nuances of the decision. Signpost helped us with this. Using it we increased he document’s displaying speed several times.

To learn more about os_signpost application technology, see where it can help you and how it has already helped us, go further forward.

Instruments for Apple's Xcode is a tool for performance analysis of an iOS application. In 2018 Apple introduced Custom Instruments — an opportunity to expand the standard set of instruments for application profiling. When existing tools are not enough, you can create new ones yourself — it will collect, analyze and display the data the way you need.

The explosion of open source and issues related to it

The amount of open source or other third party code used in a software project is estimated as 60-90% of a codebase. Components, such as libraries, frameworks, and other software modules, almost always run with full privileges. If a vulnerable component is exploited, such an attack can facilitate serious data loss or server takeover. Applications using components with known vulnerabilities may undermine application defences and enable a range of possible attacks and impacts.



Conclusion: even if you perform constant security code reviews, you still might be vulnerable because of third-party components.

Some have tried to do this manually, but the sheer amount of work and data is growing and is time consuming, difficult, and error prone to manage. It would require several full time employees and skilled security analysts to constantly monitor all sources to stay on top.

You can’t properly test and debug mobile apps without test devices, which there should be plenty of considering how the same code may behave differently on different models. So how do we keep track of these devices? How do we quickly provide developers and testers with the smartphones they need, configured the way they need, and without much red tape?

I’m Alexey Lavrenuke. Over the years, I’ve worn many hats: one of the authors behind Yandex.Tank, a speaker on load testing, and the guy who calculated energy consumption by mobile phones. Now I’m a Yandex.Rover developer on the self-driving car team.

After the phones and before Yandex.Rover, there was Hypercube.

A few years ago, the head of mobile development popped in to the load testing department and mentioned a problem they were having with test devices: phones had a tendency to inexplicably migrate from one desk to another. Picking the right device and then finding it had become a challenge. We already experienced working with mobile devices from building a digital ammeter to calculate energy consumption, so we decided to help our coworkers out and quickly rig up a handy contraption. We figured the whole thing wouldn’t take more than three months. Oh how wrong we were. Let me tell you what we were really in for.


^{_{''Dallas cube''}}