DeviQA Case Study: SoftNAS
DeviQA Logo

SoftNAS

Mobile Automation Testing, Dedicated QA Team, Web Apps, Software Performance Testing

Project Overview

What SoftNAS had in the testing processes when they came to us and what they got after they started working with us.

Before improvement

Test suite was developed without maintenance and scalability support in mind
Bugs were poorly described

After improvement

SoftNAS benefited by getting test results faster and receiving stable automated tests that led to them being able to deploy
Significantly improved the quality of products
Integrated performance testing suite into Continuous Integration process.
Low level API automation test suite was developed from scratch
The number of issues reduced to 25 % after automation was properly implemented
1,000+
auto tests developed
6
parallel threads
30%
"critical" bugs reported
80%
coverage of application tests
200+
bugs reported
QA Team:
4 QA Engineers
Project length:
1 years
Technologies & Tools
Python
Selenium
Robot
API
CLI
Jenkins
Linux
Azure
AWS
VMware
NFS
iSCSI
CIFS
DevOps

The Challenge

Software-defined Cloud NAS for AWS, Microsoft Azure, VMware vSphere and CenturyLink Cloud supporting NFS, CIFS/SMB, iSCSI and AFP.

DeviQA started collaboration with Softnas with 2 automation QA engineers. We joined the large in-house automation team which included 8 employees. We had to deal with huge test suite that supported various cloud storage platforms, operating systems (Linux and Windows) and file transfer protocols (NFS, CIFS, ISCSI). Furthermore, the test suite was developed by a different team which made it hard to gather information about test's code base.

After some time, two more manual QA engineers from DeviQA joined the project. The main task was to perform pre-release testing on different cloud platforms and to report all found issues. Besides that, our manual QA engineers were verifying fixes for bugs reported by other team members that were poorly described.

We faced complex test scenarios that usually required 2-3 connected virtual machines with shared data between them.

Achievements

4 QA Engineers from DeviQA were involved in the testing process for SoftNas products. We performed functional, API, and automated testing. We also implemented methods that use VMware and Azure CLI tools into tests, made test cases, and code review. Also, we integrated automated tests with a complex structure of input variables into Jenkins CI and maintained multiple parallel test flows on CI server side. We worked with CIFS, NFS, and ISCSI protocols on Linux and Windows.

As a result of our activities, SoftNAS benefited by getting test results faster and receiving stable automated tests that led to them being able to deploy. We increased test suite coverage by more than 1000 automated tests and significantly improved the quality of their products.

Our manual QA team reported more than 200 bugs, 30% of which were critical or blocker issues that prevented releasing new versions of the application.

DeviQA testing team successfully became part of large QA team and helped with improving product quality

Services Provided

We developed and maintained functional tests using "Robot" framework on Python. We also added API tests and implemented methods that use API calls to speed up functional tests. Our engineers worked as a part of the client's in-house automated testing team and participated in code review / merge process.

Mobile Automation Testing

4 Senior Test Engineer from DeviQA worked as a dedicated team and performed manual functional and nonfunctional testing using Microsoft Azure, Amazon AWS, and VMWare platforms, updating test documentation (test cases, checklists). All bugs were tracked and reported in Assembla.

Dedicated QA Team

2 Manual QA Engineers from DeviQA were responsible for manual testing, bugs verification, backlog clean up, updating and creating new test scenarios, communication with dev and automation QA team.

Web Apps

During the automated testing process, we also tested the speed, scalability, and stability of the application. We integrated performance testing suite into Continuous Integration process.

Software Performance Testing