Testing Patterns for External Web API Access

Writing automated tests that entail external Web API access isn't an easy job. You may suffer various things such as API rate limits, slowness due to network overheads, little control over API's behaviour, and so on.

I'll introduce four patterns to write robust automated tests involving external Web API access, depending on your needs and restrictions.

I'll call external Web API just API hereunder since it's a sort of acceptable way to refer to Web API in some contexts.

Here are the four patterns:

1. Real API
2. Official or community-supported fake API
3. API request stub (with contract tests)
4. API wrapper and test doubles (with contract tests)

Roughly speaking, more reliable but costly (in some sense) one comes first in this list. Pattern 2 to 4 involve some sort of test doubles, which usually make your tests less resistant to refactoring. This list is also in order of the layer test doubles belong to - one with test doubles in the more outer layer comes first.

Resilience to refactoring and design changes

Before looking into the four patterns, I'd like to mention this important concept - resilience to refactoring and design changes.

When you use test doubles, your tests will rely on them, of course. That means the code you can refactor without breaking tests is partly determined by your test doubles. Let's look at this example:

class ApiClient {
  exec(params) { ... }
}

class UseCase {
  doSomeBussiness() {
    // Do some business using ApiClient#exec
  }
}

When you write tests for UseCase#doSomeBusiness, you can use a test double in place of ApiClinet#exec. But if you change the interface of ApiClinet#exec, your tests will be useless as it's going to rely on the old interface. But if you don't use any test doubles and rely on the real API, you have no such worries.

Testing a private method has a similar issue. Even though your refactoring of some public method doesn't change its behaviour at all, the tests for the private method may start failing if you change its interface.

This is also about white box testing vs black box testing. You tend to write white box like tests when you use test doubles in an inner layer because you need to know more implementation details to write such test doubles. The more outer layer where you use test doubles, the bigger black box you can conceptually build. For an extreme example, if you write only E2E tests, as long as your E2E tests pass, you can refactor everything without breaking any tests because your tests only cover the largest black boxes a.k.a. behaviours.

So far, it may have sounded like using test doubles, testing private methods, and white box testing were a bad idea - but not really, it depends. The point is tests and test doubles define the interfaces you can change without breaking tests. As long as the interfaces of test targets and test double targets are stable enough, they don't prevent you from refactoring.

This topic also reminds me of The Testing Trophy.

Real API

Just use real API. No test doubles are required. In a sense, this is the easiest and most robust way to write tests involving API.

When

• You have sandbox API you can safely consume
• Costs aren't problematic
• Network overheads are at an acceptable level

Pros

• Most robust (because it's the real API!)
• No test doubles required

Cons

• Sandbox API isn't always available
• API limit may prevent you from calling API that often
• It may be quite costly
• Network overheads may make your tests slow

Official or community-supported fake API

A fake is a type of test double that behaves as if it's a real one. In other words, with a fake, you can write tests as if you use the real one. What you need to do is just replace the real one with a fake somehow.

What I want to mean by a fake here is a fake server that responds to HTTP requests. One of the most popular fakes is LocalStack.

For instance, LocalStack S3 behaves as if it's a real S3. You can put, get, and delete an object against LocalStack S3.

What you need to do is:

• Get a fake server running
• Replace the endpoint with the fake server's one

You have to be careful to choose what fakes you'd like to use. Fakes have to conform to the real API. Otherwise, your tests will rely on something unreal.

That's why I explicitly mentioned "official or community-supported". They should be well-made and well-maintained as we need reliable and realistic fake servers.

When

• There are reliable fake servers
• You don't like to make real requests
• You don't like to use test doubles other than fake servers

Pros

• Robust depending on the reliability of your choice of fake servers
• No test doubles required other than fake servers

Cons

• Good reliable fake servers aren't always available
• It's a bit cumbersome to spin up a fake server every time you run your tests
• Sometimes, it's a bit hard to replace the endpoint with your fake server's one

API request stub (with contract tests)

This is another way to stub out HTTP requests to Web API. This is usually a bit easier to adopt than fake servers. In JavaScript, you can do this with Nock.

What's important is to write contract tests if possible. Contract tests usually run periodically to confirm the API works as expected. As long as contract tests pass, your HTTP request stubs should be reliable as there is no contract change.

When

• You don't like to use test doubles other than HTTP request stubs
• You can't seem to find good fake servers
• It's too cumbersome to set up fake servers

Props

• No test doubles required other than HTTP request stubs
• Easier to introduce than fake servers
• A bit more control over stubbed API's behaviour
• Tests run fast

Cons

• Writing HTTP request stubs can be cumbersome
• Without contract tests, you may not be able to detect API changes

API wrapper and test doubles (with contract tests)

This is the easiest to implement because you don't need to use extra libraries if you want. Let's have a look at the same example again:

class ApiClient {
  exec(params) { ... }
}

class UseCase {
  doSomeBussiness() {
    // Do some business using ApiClient#exec
  }
}

If you inject an instance of ApiClient into UseCase, you can also inject a test double instead with no hassle. Of course, you can use libraries - in JavaScript, Sinon is quite popular for this purpose.

Contract tests or tests for your wrappers play an important part as well to make your tests robust. As you stub your wrappers, your wrappers should be well tested.

When

• You can write well-designed wrappers
• You can't seem to find good fake servers
• It's too cumbersome to set up fake servers or HTTP request stubs

Props

• Really easy to introduce
• Very good control over API's behaviour
• Easier to understand what's stubbed when you read tests
• Tests run fast

Cons

• Your tests may not be very resilient to refactoring and design changes
• You may need to write more white box like tests
• Without contract tests, you may not be able to detect API changes