DeviQA Case Study: Solegate Mimecast
DeviQA Logo

Cloud-based email protection software

Optimizing automated testing and an entire QA process for a cutting-edge cyber protection software platform.

>150

Autotests developed

6+

Platforms covered

10x

Faster testing time

10

Parallel threads

30

Machines used

500k+

Files uploaded

300+

Bugs reported

200+

Bugs reported by autotests

About project

SoleGATE™ (Mimecast) is a disruptive cyber protection software platform developed by Solebit. It challenges the norms of existing technologies that usually rely on slow, costly, mostly outdated, and ineffective methods of sandboxing, signatures, and behavioral inspection.

Before DeviQA

  • 20 hours to run regression testing

  • 10% test coverage

  • 1 machine for running tests

  • 1 thread for running tests

  • 1 report for all tests

  • ~15 UI test cases

  • 1 platform was covered by autotests

  • Tests were triggered by a developer manually via a terminal

  • A team stored test results in a large and complicated file

  • The history of test runs was unavailable

  • An unscalable test suite architecture

  • There were no UI tests

  • All test results got lost if some tests failed due to technical reasons

With DeviQA

  • 2 hours to run regression testing

  • 80% test coverage

  • 30 different machines for running autotests

  • 10 threads for running tests on each machine

  • 30 reports with test results of each machine

  • 250+ UI test cases

  • 6+ platforms were covered by autotests

  • Simplified test runs without involving a tech person

  • An efficient reporting system was created to allow the team to view clean reports even if the results contained 10k rows

  • The history of test runs can be easily reviewed in the reporting system

  • Configured file processing selection on several machines (Azure Cloud, Google Cloud)

  • We added >50 e2e UI tests that were triggered automatically after each deployment

  • We simultaneously saved a main test result file and test results of each machine

DeviQA automated numerous testing processes to reduce manual effort and resolved file scanning issues. While the build was projected to take 15 hours, they completed it in just three. The team was highly professional, kept detailed records, and maintained an open dialogue and synergistic workflow.

Yuval Or

QA Manager, Solebit (Mimecast)

Our contribution

Team

4 Senior SDET

Project length

From 2013 to 2018

Technologies and tools

Jira

Ruby

Faraday

Cucumber

Linux

REST API

Azure Cloud

Google Cloud

Docker

LCX

Jenkins

RubyMine

Postman

TestRail

Our engagement

DeviQA created complex and detailed automated scenarios for testing REST APIs using the Faraday library. We built a software development kit application that worked on various machines, including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC (Linux containers). The QA team created complex methods for these integrations using SSH and SFTP connections. Altogether, we designed and developed more than 250 tests. Every test ran 500,000 files which were then uploaded.

We reduced build time by a factor of 10 and configured test runs in a way that didn’t need the involvement of a tech person. We also created a reporting system that allowed the team to view clean reports even if test results contained 10k rows. Finally, we configured file processing selection on several machines (Azure Cloud, Google Cloud).

Services provided

Web Testing

We tested a UI part of the application and prepared comprehensive test documentation. Our test cases covered 80% of the application. Also, we created test suites for each release for different customers.

Dedicated QA Team

Before starting a long-term journey with the client, we proved our expertise by performing a QA audit. We identified thin places in their infrastructure, including non-flexible machines, an insufficient number of machines for checking, a necessity to clean up previous application versions, and too time-consuming machine creation before build. We provided clear and simple ways to fix these issues.

Performance Testing

Our team designed and developed performance/load and stress tests and then integrated them into CI. In addition to checking the operation of the system under high pressure, we also checked how fast it could be restored after a crash.

API Testing

Our work was mostly focused on automating REST API tests, auto-sending emails through SMTP, implementing DevOps practices, and performing configuration activities. Autotests for file uploading were run in parallel on 30 different machines, with 10 threads on each machine.

More projects

Test coverage:

90%

  • Web App Testing
  • Test Automation
  • API Testing
  • Dedicated QA Team

Abbott Laboratories

A global healthcare giant relies on DeviQA for end-to-end test automation.

  • 1 day to run regression testing

  • 1 day to run smoke testing

  • 60% increase in the number of regression tests and relevant test cases

  • >1500 automated test scenarios created

Read more

Dev capacity:

+15%

  • Web App Testing
  • Automation Testing
  • E2E Testing
  • Load Testing
  • Mobile Testing
  • API Testing
  • Dedicated QA Team

Compass

DeviQA's team takes full responsibility for testing a real estate solution consisting of a web app and mobile apps.

  • 2 days to run regression testing

  • 10 minutes to run automated E2E testing on each pull request

  • 85% of the application is covered with test cases

Read more

Test coverage:

95%

  • Web App Testing
  • API Testing
  • Dedicated QA Team
  • Mobile Testing
  • Performance Testing

Arklign

Setting up and fine-tuning a holistic QA process for a dental lab management platform.

  • A comprehensive test plan designed

  • 5000+ test cases created for the web app

  • 2000+ test cases created for the mobile apps

  • >20 various devices with different OS, screen resolutions, and browsers used to run regression testing

Read more