API Automation at busuu
There is a very important part of the software testing lifecycle that is not usually considered in most of companies, and that is API testing. A valid question to ask is why we should test our APIs when we have a functional testing? The superficial summary is that with functional testing you’re testing how the application behaves; with API testing, you’re testing the internal mechanisms that drive application behaviour.
Why we need API tests
At the base of the test automation pyramid is unit testing. Unit testing should be the foundation of a solid test automation strategy and as such represents the largest part of the pyramid. Then there is user interface testing, placed at the top of the pyramid because we want to do as little of it as possible. At busuu, we had both Unit tests and UI tests but we were lacking API tests which come between the two.
Where many organizations have gone wrong in their test automation efforts over the years has been in ignoring this middle layer of service testing. Although automated unit testing is wonderful, it can cover only so much of an application’s testing needs. Without service-level testing to fill the gap between unit and user interface testing, all other testing ends up being performed through the user interface, resulting in tests that are expensive to run, expensive to write, and brittle.
What we wanted to achieve
- Confidence before releasing
- Easy to add to and maintain
- Simulate requests from different clients
- Able to run in different environments (production, integration, staging)
- Maintainable by both API and Test teams
- Good reporting with proper alerting
Keeping in mind what we would like to achieve, we had faced lots of difficulties in finalising the tools and technology to be used to write API tests and then we discovered “Codeception”.
Why Codeception?
Codeception is a BDD-styled PHP testing framework, whose only requirements are basic knowledge of PHP and the theory of Automated Testing.
Codeception is kept as simple as possible to allow many kinds of users to use it effectively.
- It’s easy to read / write / debug
- Enables users with different roles to share common test languages
- Describes actions from a users point of view
- Allows continuous integration with systems like Jenkins or Bamboo
- Results of the tests are readable to non-technical stakeholders.
- Makes testing fun
What Does Codeception Do?
It helps in testing all the below:
- Unit Tests
- Functional Tests
- Acceptance Tests
- API Tests
Test Structure
The above code example explains the following:
1) BDD style definition that appears in the report
2) Checking the response code after sending a request
3) Validating the response with the schema
What is a schema?
- A schema is the basic definition for the structure of the returned JSON as shown above
- It also defines which fields should be present by using the “required” key
- Defines the content type of each field, ensuring types such as integers, strings, etc
- Allows for easy readability and compartmentalisation of definitions
Output and Test Results
With Codeception we get a nice report stating the success and failure. When you expand the failed result then you can get error messages with the reason for the failure.
Let’s look into some of the error messages:
The above error message clearly explains the problem. The [data.language] field requires a string value but we have received an integer.
And in this example, response should be 200 but we are receiving a 400.
Integration with Jenkins
As shown, we have integrated API tests with Jenkins for all our integration, staging and production environments. When any build fails then you can see information about the server, branch, and time when the build failed and the person who committed the change.
Challenges
- There is overhead updating the Schema for API tests
- Testing parameter combinations and validating
- Managing the data and constants for integration and production environment
- Sequencing the API calls
I hope you enjoyed this post! If this sounds great, you’d like to progress in your tech career, and you’ve got a love for learning languages (tech or otherwise!), we’ve got lots of open roles in our Tech team!
