Home

Case studies

SoftNAS

QA for cloud storage solution

Applying a comprehensive approach to testing a NAS solution for achieving unmatched quality.

About project

SoftNAS is a software-defined cloud Network Attached Storage (NAS) that integrates seamlessly with AWS, Microsoft Azure, VMware vSphere, and CenturyLink Cloud. It supports a variety of protocols, including NFS, CIFS/SMB, iSCSI, and AFP, covering diverse storage needs.

SoftNAS functions as a Linux-based virtual NAS appliance, deployed on modern hypervisor-based systems. Operating as a virtual machine, it provides a comprehensive range of software-defined capabilities. These include optimizing data performance, managing costs, ensuring high availability, maintaining control, providing robust backup solutions, and implementing advanced security measures.

Before DeviQA

There was an improperly configured automated testing structure
Performance testing was not executed
The results of automated test runs were delivered very slowly
API testing was not executed
QA services and support were insufficient

With DeviQA

25% fewer issues were spotted after the proper implementation of test automation
>1,000 test scripts were developed
A performance testing suite was created and integrated into a CI process
45% faster delivery of automation test results was achieved
A low-level API automation test suite was developed from scratch
>200 bugs were reported
30% of logged bugs were critical issues
80% of the app was covered with tests

Our contribution

Team

4 QA engineers

Project length

1 year

Technologies and tools

Python

Selenium

Robot

API

CLI

Jenkins

Linux

Azure

AWS

VMware

NFS

iSCSI

CIFS

DevOps

Our engagement

Four QA engineers from DeviQA were engaged in testing SoftNas products, carrying out functional, non-functional, API, and automated testing.

Our team introduced the use of VMware and Azure CLI tools for testing, wrote test cases, and conducted code reviews. Additionally, we integrated automated tests with a complex structure of input variables into Jenkins CI and maintained multiple parallel test flows on the CI server. We worked with CIFS, NFS, and ISCSI protocols on both Linux and Windows.

Thanks to our efforts, SoftNAS received stable automated tests and fast test results, enabling confident deployments. We increased test coverage by writing more than 1,000 automated tests. Overall, 80% of the app was covered with tests. This significantly contributed to better product quality.

Our manual QA team reported over 200 bugs, 30% of which were critical or blocker issues preventing the release of new app versions.

DeviQA’s QA engineers were valuable members of a large SoftNAS team and helped them propel the quality of the developed software.

Services provided

Web testing

Two of our QA engineers were responsible for manual testing, bug verification, backlog cleanup, updating and creating new test scenarios, and communication with development and automation QA teams.

Mobile testing

We developed and maintained functional tests using the Robot framework on Python. Also, we created API tests and implemented methods that used API calls to speed up functional tests. Our engineers worked as members of the in-house automated testing team and participated in the code review/merge process.

Performance testing

Our automated testing process also suggested assessing the speed, scalability, and stability of the application. We integrated a performance testing suite into a continuous integration process.

Dedicated QA team

Four senior QA engineers from DeviQA worked as a dedicated team. They updated test documentation (test cases, checklists) and executed manual functional and non-functional testing using Microsoft Azure, Amazon AWS, and VMWare platforms. All bugs were tracked and reported in Assembla. Throughout our collaboration with SoftNAS, we strived to be on the same page with all team members and stakeholders, ensuring excellent product quality.