Introduction

Integration testing helps in testing applications in real-time, using actual dependencies without mocking. Unit Tests, on the other hand, mocks external dependencies using fake objects. Unit tests are great for testing systems involving logical functionalities, without external dependencies. For example algorithms or a unit involving logical functionalities. But for Web API Integration testing is much more beneficial compared to the Unit Tests. There is a great article written by Steven Sanderson which explains the cost and benefits of Unit Testing.

In this article, I am going to explain an approach for integration testing Web API using a snapshot-based technique.

The technique explained in this article is using ASP.NET Core Web API but the approach can be used for any language Java, Go, NodeJs, Ruby on Rails, etc.

What is Snapshot based testing?

You might have heard of snapshots in UI testing. In UI testing, a snapshot of UI is taken and compared before and after running a test using a tool such as a cypress.

Similar to how UI is tested using snapshots Web API can also be tested using this technique. The difference is in UI testing snapshots is an image file. In API snapshot denotes API response JSON output file. You can find an example here.

When a test runs for the first time, a JSON output response of API under test is generated (Snapshot). On the next runs, an old snapshot is compared with a new snapshot using a Json diff patch comparison technique, inspired by Jestjs (JavaScript test framework). If no mismatch is found, the test passes. If a mismatch is found, then a mismatched file is generated. This mismatch file needs to be manually verified whether the output is expected one, or not. If not expected then, recent code changes need to be reviewed to find the cause of mismatch and correct it. If the output is expected one, then the old snapshot file needs to be replaced by a mismatched file.

Some of the reason for mismatch in snapshots are:

  • API parameter changed
  • Field or its value is added, removed or modified

Why use a Docker-based approach and why not In Memory?

In ASP.NET Core, you can switch the data providers easily to use In-Memory during tests. In Memory provider was mainly introduced in the framework for integration testing, making tests faster to run with close to real database functionalities avoiding the need to mock dependencies.

But there are limitations of In-Memory Data provider, It is close to the real database but still lacks many features which actual database provides.

Running a database in Docker containers assures that tests are running under a real-time environment. We can perform all operations which actual real database provides. Like Foreign keys Integrity checks, Postgres specific functionalities Jsonb, text search, etc.,

Scenario

You have an ASP.NET Core Web API which interacts with Postgres and Redis. In this article Postgres is just used as a sample to explain this concept, you could use any external dependencies your system is using and run that in docker containers.

Steps to setup Integration Testing Environment

  • Install a Docker Desktop in your machine.
  • Write a docker and a docker-compose file for spinning up a docker container containing Postgres and Redis.
  • Create an integration testing project.
  • Write up the Test Setup method to run a docker container on the startup of the integration test session.
  • Configure Test Web host server.
  • Configure connection string to connect to docker Postgres and Redis container image.
  • Use the EF Core code-first approach for seeding database schema.
  • Seed Postgres Database with test data. You can maintain a file per Database table (JSON / YAML ) containing test data.
  • Write a method to read files and seed databases running under docker containers. (You can maintain a test data JSON file per Database table either JSON/YAML)

Writing Integration Tests

  • Now write an actual integration test. You can find a guide here by Microsoft. It uses in-memory instead of that you need to use a database running under a docker container.
  • Call a Web API URL under test. Behind the scene, a Web API call is made through the test server, which connects to the actual Postgres database.
  • API returns a JSON output.
  • Use the Snapshooter library assert method. On the first run, it will create a JSON snapshot file of output.
  • The first run test will pass since the new file will be created.
  • In the next run, the Snapshooter assert method will match the output returned from API with the existing snapshot file.
  • If no mismatch in output test will pass.
  • If mismatch in the output file, a mismatched file will be generated.
  • Manually validate mismatch file if valid, replace mismatch file with an actual file. If the output is not intended, cross-check functionality and correct.
  • Once all tests complete, delete the docker container. In this way, each test session runs in isolation.
  • Note: Snapshot files need to be committed to the GIT / SVN.

Continuous Integration

This process can be easily integrated into Continuous integration using Cake Script and Jenkins. A sample for the cake script can be found here. Some alternate method is using Azure DevOps, Gitlab, or GitHub pipelines to run tests. These tests can be run on each Merge Requests or on a schedule basis.

Flow Diagram

Integration Testing Flow Diagram

Advantages

  • Tests run in a real environment
  • No mocking, functionality is ensured with actual dependencies.

Disadvantages

  • Tests will be slow.
  • If you write more test cases, time for running may increase further. Which will cause Continuous Integration builds taking more time.

Overcoming disadvantages

  • Run tests on a scheduled basis like daily / weekly once.
  • Run tests before release or merging codes to release & master branches.
  • Don’t write more tests covering all edge cases, write smartly just enough tests to check the integrity of the system

References

Modern Web API Testing with Snapshots in .NET Core 3
Integration tests in ASP.NET Core

Libraries

snapshooter – A .NET Core library helpful for snapshot-based testing
CliWrap – A .NET Core Virtual shell library
JsonDiffPatch.Net – JSON object diffs and reversible patching (jsondiffpatch compatible)
jest-dotnet – Simple snapshot testing with inspiration from the amazing Jest library.

Conclusion

Nowadays, container technology has eased up the work, external dependencies can be easily set up and can be run anywhere. Integration tests are great to test the overall integrity of systems. Testing Web API involving database calls using integration testing is more beneficial compared to Unit Testing using Mocks.

Last modified: August 9, 2020

Author

Comments

Write a Reply or Comment

Your email address will not be published.