DeviQA Case Study: DCMS
DeviQA LogoDeviQA is the finalist of the Software Testing Award 2019

DCMS

Test Automation, Mobile Testing, Performance Testing, Web App Testing, API Testing

Project Overview

What DCMS had in the testing processes when they came to us and what they got after they started working with us.

Before improvement

The lack of test documentation
Existing workflow in Jira didn't allow to understand how many bugs were left unfixed before the release and their priorities.

After improvement

Integrated TestRail with Jira to have full access to related test results on Jira side
Developed an architecture of the automation test suites
Added multithreading to run tests in parallel on 10 different threads, which reduced the time of the test run in 7 times.
Used Appium and ran tests on AWS Device Farm using real devices as well as on our local mobile physical devices.
Covered more than 90 % of the application with the test cases
Covered more than 80% of web/mobile apps by automated tests.
Several rounds of performance testing were done. Detailed report with the bottlenecks and our recommendations for the improvements were provided.
Tests were integrated into an existing CI/CD process
Security Testing was performed quarterly
8000
Bugs reported
90%
Test Cases coverage
3300
Auto tests developed
7000
Test Cases designed
700
Critical defects found
3000
Major issues reported
QA Team:
6 QA engineers
Project length:
3 years
Technologies & Tools
Ruby
Capybara
Selenium
Faraday
Appium
Cucumber
Jenkins
JMeter
Jira

The Challenge

A technology company that integrating software into dental lab fabrication, logistics, and data to transform the dental service experience.

The client was looking for a reliable, QA partner who will be responsible for the quality of the project. The product consists of Web and Mobile applications for iOS and Android.

The requirement was to build a robust and formal QA process to speed up the release cycle.

The lack of test documentation didn't allow us to make structural testing; that's why we started with familiarization with the product and the requirements. We designed the test plan and test cases. Test cases allowed us to avoid chaotical testing and prevented us from missing the bugs.

Another challenge was the existing workflow in Jira, which complicated bug's lifecycle and didn't allow to understand how many bugs left unfixed before the release and their priorities.

The testing should be made on all major browsers, more than 40 different mobile devices and various platforms.

The client wanted to invest in automated testing once the QA process is formalized and integrated auto-tests into CI/CD process. The automated test should be running on the real devices as well as on the emulators in the cloud. All the team should have access to clear and nice-view reports.

Additionally, there were needs in the performance and load testing services while the project was growing.

DevOps also was a part of our responsibility for the project.

We were continuously adding the new test cases, automated tests, and made manual testing activities for the new features.

Achievements

After familiarization with the project's requirements, we decided to store test documentation in the TestRail. We integrated TestRail with Jira to have full access to related test results on Jira side, as well as test case links and TestRail metrics & dashboards.

For Web automated testing, we decided to use Ruby + Capybara + Selenium + Cucumber stack and to add tests in Jenkins to integrate them into the CI/CD process. We ran tests on SauceLabs and stored the results on Jenkins side. To speed up the tests, we added multithreading to run tests in parallel on 10 different threads, which reduced the time of the build in 7 times.

For mobile automated testing, we used Appium and ran tests on AWS Device Farm as well as on our local mobile physical devices. Here we used 3 threads, which means that 3 different tests were running in parallel on mobile devices.

Within the timescale of this project, we covered more than 90% of the application with the test cases and 80% of web/mobile apps with the automated tests.

Performance testing was also a part of QA strategy designed by DeviQA. The suite and all scenarios were created from scratch using JMeter. We set up monitors on the backend to review how CPU, RAM, I/O changes under the load. As a result of performance testing, we prepared a detailed report with the bottlenecks and our recommendations for the improvements.

Also, we provided API Security Testing to secure it from external threats.

As a result of work, our customer successfully launched a project with a minimum number of issues. Our customer praised our solutions.

Services Provided

We were responsible for creating an architecture of autotests, increasing its stability and speed, reduce the build time, simplify the regression testing. We designed and developed automated tests for Web, Mobile(iOS, Android) applications as well as API. Tests were integrated into the CI/CD process and were running on 10 different threads in parallel.

Test Automation

We were also responsible for mobile testing. Our QA engineers developed and created test documentation, conducted manual testing, designed mobile test suite architecture, developed automated tests, configured environment to run tests in the cloud as well as on local devices. We tested on both IOS and Android platforms and used around 40 different mobile devices.

Mobile Testing

We conducted a performance and load testing, to understand the capacity of the application. We used JMeter as the primary tool for this goal. During the testing process, we set up monitors on the backend to get the information about the CPU, RAM and I/O usage on the backend.

Performance Testing

QA team from DeviQA were responsible for manual testing, bugs verification, updating, and creating new test scenarios. QA engineers designed and created test documentation which covered almost the entire project with tricky cases, and they were involved in the manual web testing process.

Web App Testing

We automated API testing process too and used Faraday to achieve that. Tests were integrated into an existing test suite and added to the CI/CD process along with other tests.

API Testing