IT systems testing *

Disclaimer: This is a translation of an article. All rights belongs to author of original article and Miro company.

I'm a QA Engineer in Miro. Let me tell about our experiment of transferring partially testing tasks to developers and of transforming Test Engineer role into QA (Quality assurance).

First briefly about our development process. We have daily releases for client side and 3 to 5 weekly releases of server side. Team have 60+ people spitted onto 10 Functional Scrum Teams.

I'm working in Integration team. Our tasks are:

• Integration of our service into external products
• Integration of external products into our service
  For example we have integrated Jira. Jira Cards — visual representation of tasks so it's useful to work with tasks not opening Jira at all.
  
  

How the experiment starts

All starts with trivial issue. When someone of Test Engineers had sick leave then team performance was degraded significantly. Team was continued working on tasks. However when code was reached testing phase task was hold on. As a result new functionality didn't reach production in time.

Going onto vacation by Test Engineer is a more complex story. He/she needs to find another Test Engineer who ready to take extra tasks and conduct knowledge sharing. Going onto vacation by two Test Engineers at the sane time is not an applicable luxury.

This article is an authorized translation of the original post. The translation was made with the kind help of the guys from PVS-Studio. Thank you, guys!

What encouraged me to write this article is considerable quantity of materials on static analysis, which recently has been increasingly coming up. Firstly, this is a blog of PVS-Studio, which actively promotes itself on Habr posting reviews of errors, found by their tool in open source projects. PVS-Studio has recently implemented Java support, and, of course, developers from IntelliJ IDEA, whose built-in analyzer is probably the most advanced for Java today, could not stay away.

When reading these reviews, I get a feeling that we are talking about a magic elixir: click the button, and here it is — the list of defects right in front of your eyes. It seems that as analyzers get more advanced, more and more bugs will be found, and products, scanned by these robots, will become better and better without any effort on our part.

Well, but there are no magic elixirs. I would like to talk about what is usually not spoken in posts like «here are things that our robot can find»: what analyzers are not able to do, what's their real part and place in the process of software delivery, and how to implement the analysis properly.


Ratchet (source: Wikipedia).

Ability to write good unit tests is an important feature of any developer. But how to understand that your unit tests are correct? Good unit test is like a good chess game. In our case chessmen are the approaches which we are going to discuss in this post. There is no best chessman in a chess game because everything depends on the positions (and a player). Likewise, in unit testing you don't have to distinguish only one approach. In other words, you should use all approaches together to get the best result. So, if you want to win this game, then welcome under the cut.

It is text version of the presentation 2018-04-25 at Saint-Petersburg Linux User Group. Configuration example locates at https://github.com/ultral/ansible-role-testing

I suppose that that you make configuration management, not bash. It means that you have to test it some how. Have you ever tested ansible roles? How do you do it?

Russian version

Let's imagine that you are developing software and hardware appliance. The appliance consists of custom OS distributive, upscale servers, a lot of business logic, as a result, it has to use real hardware. If you release broken appliance, your users will not be happy. How to do stable releases?

I'd like to share my story how we dealt with it.