Home

Case studies

Cloud-based email protection software

QA for cybersecurity software platform

Optimizing automated testing and an entire QA process for a cutting-edge cyber protection software platform.

>150

Autotests developed

Platforms covered

10x

Faster testing time

Parallel threads

Machines used

500k+

Files uploaded

300+

Bugs reported

200+

Bugs reported by autotests

About project

SoleGATE™ (Mimecast) is a disruptive cyber protection software platform developed by Solebit. It challenges the norms of existing technologies that usually rely on slow, costly, mostly outdated, and ineffective methods of sandboxing, signatures, and behavioral inspection.

Before DeviQA

20 hours to run regression testing
10% test coverage
1 machine for running tests
1 thread for running tests
1 report for all tests
~15 UI test cases
1 platform was covered by autotests
Tests were triggered by a developer manually via a terminal
A team stored test results in a large and complicated file
The history of test runs was unavailable
An unscalable test suite architecture
There were no UI tests
All test results got lost if some tests failed due to technical reasons

With DeviQA

2 hours to run regression testing
80% test coverage
30 different machines for running autotests
10 threads for running tests on each machine
30 reports with test results of each machine
250+ UI test cases
6+ platforms were covered by autotests
Simplified test runs without involving a tech person
An efficient reporting system was created to allow the team to view clean reports even if the results contained 10k rows
The history of test runs can be easily reviewed in the reporting system
Configured file processing selection on several machines (Azure Cloud, Google Cloud)
We added >50 e2e UI tests that were triggered automatically after each deployment
We simultaneously saved a main test result file and test results of each machine

DeviQA automated numerous testing processes to reduce manual effort and resolved file scanning issues. While the build was projected to take 15 hours, they completed it in just three. The team was highly professional, kept detailed records, and maintained an open dialogue and synergistic workflow.

Yuval Or

QA manager, Solebit (Mimecast)

Our contribution

Team

4 senior SDETs

Project length

5 years

Technologies and tools

Jira

Ruby

Faraday

Cucumber

Linux

REST API

Azure Cloud

Google Cloud

Docker

LCX

Jenkins

RubyMine

Postman

TestRail

Our engagement

DeviQA created complex and detailed automated scenarios for testing REST APIs using the Faraday library. We built a software development kit application that worked on various machines, including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC (Linux containers). The QA team created complex methods for these integrations using SSH and SFTP connections. Altogether, we designed and developed more than 250 tests. Every test ran 500,000 files which were then uploaded.

We reduced build time by a factor of 10 and configured test runs in a way that didn’t need the involvement of a tech person. We also created a reporting system that allowed the team to view clean reports even if test results contained 10k rows. Finally, we configured file processing selection on several machines (Azure Cloud, Google Cloud).

Services provided

Web testing

We tested a UI part of the application and prepared comprehensive test documentation. Our test cases covered 80% of the application. Also, we created test suites for each release for different customers.

Dedicated QA team

Before starting a long-term journey with the client, we proved our expertise by performing a QA audit. We identified thin places in their infrastructure, including non-flexible machines, an insufficient number of machines for checking, a necessity to clean up previous application versions, and too time-consuming machine creation before build. We provided clear and simple ways to fix these issues.

Performance testing

Our team designed and developed performance/load and stress tests and then integrated them into CI. In addition to checking the operation of the system under high pressure, we also checked how fast it could be restored after a crash.

API testing

Our work was mostly focused on automating REST API tests, auto-sending emails through SMTP, implementing DevOps practices, and performing configuration activities. Autotests for file uploading were run in parallel on 30 different machines, with 10 threads on each machine.