Logo

SoftNAS

QA Team: 4 QA Engineers
Project length: 1 years
1,000+
auto tests developed
6
parallel threads
30%
“critical” bugs reported
80%
coverage of application tests
200+
bugs reported

The Challenge

Software-defined Cloud NAS for AWS, Microsoft Azure, VMware vSphere and CenturyLink Cloud supporting NFS, CIFS/SMB, iSCSI and AFP.

DeviQA started collaboration with Softnas with 2 automation QA engineers. We joined the large in-house automation team which included 8 employees. We had to deal with huge test suite that supported various cloud storage platforms, operating systems (Linux and Windows) and file transfer protocols (NFS, CIFS, ISCSI). Furthermore, the test suite was developed by a different team which made it hard to gather information about test’s code base.

After some time, two more manual QA engineers from DeviQA joined the project. The main task was to perform pre-release testing on different cloud platforms and to report all found issues. Besides that, our manual QA engineers were verifying fixes for bugs reported by other team members that were poorly described.

We faced complex test scenarios that usually required 2-3 connected virtual machines with shared data between them.

Achievements

A team of 4 from DeviQA were involved in the testing process for SoftNas products. We performed functional, API, and automated testing. We also implemented methods that use VMware and Azure CLI tools into tests, made test cases, and code review. Also, we integrated automated tests with a complex structure of input variables into Jenkins CI and maintained multiple parallel test flows on CI server side. We worked with CIFS, NFS, and ISCSI protocols on Linux and Windows.

As a result of our activities, SoftNAS benefited by getting test results faster and receiving stable automated tests that led to them being able to deploy. We increased test suite coverage by more than 1000 automated tests and significantly improved the quality of their products.

Our manual QA team reported more than 200 bugs, 30% of which were critical or blocker issues that prevented releasing new versions of the application.

DeviQA testing team successfully became part of large QA team and helped with improving product quality

Services Provided

Mobile Automation Testing

We developed and maintained functional tests using “Robot” framework on Python. We also added API tests and implemented methods that use API calls to speed up functional tests. Our engineers worked as a part of the client’s in-house automated testing team and participated in code review / merge process.

Dedicated QA Team

3 Senior Test Engineer from DeviQA worked as a dedicated team and performed manual functional and nonfunctional testing using Microsoft Azure, Amazon AWS, and VMWare platforms, updating test documentation (test cases, checklists). All bugs were tracked and reported in Assembla.

Web Testing

2 of the QA team from DeviQA were responsible for manual testing, bugs verification, backlog clean up, updating and creating new test scenarios, communication with dev and automation QA team.

Performance Testing

During the automated testing process, we also tested the speed, scalability, and stability of the application. We integrated performance testing suite into Continuous Integration process.

Client's feedback

DeviQA is a great QA automation developers team. It was a pleasure to work with them. They did a great job. We would work with them again in the future.
Eric Olson
VP of Development, SoftNas