Solebit (MimeCast)

Solebit (MimeCast) provides identification and prevention of zero-day malware and unknown threats.

From the moment when we joined the project, there were automation tests designed and developed by the in-house development team. Tests were not useful because every time a developer triggered the tests, it was by hand in a terminal. As the result, the team had a big and complicated file with the results.

Also, the tests took more than 20 hours for completion, and the history of the build was also unavailable.

The test suite architecture was unscalable, making it difficult to maintain a large number of test machines.

The task was to create simple runner for tests and increase their speed. We learned some valuable lessons. We should have redesigned the architecture to support cloud platform integration, made the tests run much more easily, and generated a clean report with all the necessary details.

At the same time, we should propose the improvements for current testing process and made some manual testing, too.

DeviQA created complex and detailed automated scenarios for testing REST APIs using the Faraday library. We built a software development kit application that worked on various machines, including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC containers. The QA team created complex methods for these integrations using SSH and SFTP connections. Altogether, we designed and developed more than 25,000 tests. Every test ran 500,000 files which were then uploaded.

We reduced the build time by a factor of 10 and configured how to run tests in simple way without the involvement of a tech person. We also created a reporting system that allowed the team to view clean reports even if the results contained 10k rows. Finally, we configured file processing selection on the number machines (Azure Cloud, Google Cloud).