Scalability in ASP.NET web APIs with Cancellation Tokens
Posted 17 May, 2022
#Technology

Scalability in ASP.NET web APIs with Cancellation Tokens

When dealing with high volumes of data and traffic in web application APIs, it is imperative to avoid wasting resources so that end users won’t perceive any slow downs, as well as to reduce costs in the underlying infrastructure. The topic of today’s article is to examine under what circumstances it is possible to cancel already-running HTTP requests before their completion in an API implemented with ASP.NET, and how to implement these capabilities in a few common scenarios.

Motivation

Long running HTTP requests may occur as a consequence of having to deal with an ever-increasing amount of resources for their execution, which is a scenario that can be expected in an active API serving thousands of clients per minute. Under those circumstances, having a way of knowing when the results of such requests may end up being unused would be highly desirable in order to free up server resources, such as CPU, RAM, database locks, network sockets, etc., that other clients may need to use.

Common situations where this may arise are:

  • The user starts a search but closes or navigates away from the page before the server finishes serving its results.
  • The UI client has a typeahead component that makes requests to the server and updates the results in the page as the user types, replacing old results with newer ones.

What kind of requests can and cannot (or should not) be canceled?

Short lived requests that consume too little resources are better left alone, as the cost of implementing cancellation capabilities would be most surely greater than any real benefit that could be gained from that.

Requests that can have side effects, such as creating, updating, or deleting data, should also be left as they are, because there is a real risk of compromising data integrity of the system if the request is interrupted at the wrong moment, especially if there are multiple internal steps to the operation.

That leaves us only with nullipotent requests, that is, those that only fetch data but don’t modify anything after they finish, metrics and logs notwithstanding. In other words, if implemented correctly, HTTP GET requests.

Implementation

As an example, we’ll start with a basic ASP.NET controller written in C#:

using Microsoft.AspNetCore.Mvc;

namespace Kaizen.Blog.Examples.Controllers;

[ApiController]
[Route("api/[controller]")]
public class CancellationController : ControllerBase
{
    [HttpGet]
    public ActionResult<string> Get()
    {
        return "The endpoint was called.";
    }
}

The change to allow a request to be canceled is as simple as adding a parameter with type CancellationToken to the endpoint. This type has two useful members that can be leveraged for our purposes:

  • IsCancellationRequested: Returns true if the caller asked for the request to be canceled; otherwise, false.
  • ThrowIfCancellationRequested(): Throws an OperationCanceledException if the caller asked for the request to be canceled.

With this in mind, the endpoint can be updated to handle request cancellations manually:

using System;
using System.Threading;
using Microsoft.AspNetCore.Mvc;

namespace Kaizen.Blog.Examples.Controllers;

[ApiController]
[Route("api/[controller]")]
public class CancellationController : ControllerBase
{
    [HttpGet]
    public ActionResult<string> Get(CancellationToken cancellationToken)
    {
        try
        {
            cancellationToken.ThrowIfCancellationRequested();

            return "The request was not canceled.";
        }
        catch (OperationCanceledException)
        {
            return "The request was canceled.";
        }
    }
}

Many common asynchronous operations in web APIs can also be canceled using cancellation tokens, thus saving us from having to check the tokens ourselves. Some examples from the standard .NET APIs, as well as some popular libraries, are:

Task (System.Threading.Tasks):

// Non-cancellable version
await Task.Run(() => LongRunningSynchronousOperation());

// Cancellable version
await Task.Run(() => LongRunningSynchronousOperation(), cancellationToken);

IAsyncEnumerable (System.Linq):

var collection = GetExpensiveToCreateCollection();

// Non-cancellable version
await foreach(var item in collection)
{
    // do something with item...
}

// Cancellable version
await foreach(var item in collection.WithCancellation(cancellationToken))
{
    // do something with item...
}

ParallelEnumerable (System.Linq):

var collection = GetCollection();

// Non-cancellable version
var results = collection.AsParallel()
    .Where(Condition)
    .ToArray();

// Cancellable version
var results = collection.AsParallel().WithCancellation(cancellationToken)
    .Where(Condition)
    .ToArray();

IQueryable (System.Data.Entity; Entity Framework):

var usersQuery = DbContext.Users.GetAll().Where(Condition);

// Non-cancellable version
var results = usersQuery.ToList();

// Cancellable version
var results = usersQuery.ToListAsync(cancellationToken);

Dapper:

using var connection = new SqlConnection(connectionString);
await connection.OpenAsync();

// Non-cancellable version
var results = await connection.QueryAsync(sqlQuery);

// Cancellable version
var results = await connection.QueryAsync(
    new CommandDefinition(sqlQuery,
        cancellationToken: cancellationToken));

From the front-end perspective, web client applications can cancel API calls before their completion by making use of the so-called “Abort API” (AbortController and AbortSignal). Both the standard Fetch API, as well as popular HTTP library Axios, allow canceling requests this way. An example using the former can be found in Mozilla's dom-examples repository, while usage of the later can be found in Axios Docs.

Conclusions

It should be noted that canceling long running requests in cases where they’re caused by an inefficient implementation is only a stopgap measure. It’s no substitute for a good implementation, but a way of mitigating the damage a bad one can cause. Having said that, I’ve seen situations where the cost of reimplementing an existing piece of functionality is accompanied by a high risk of breaking something, so, as a short term patch to scalability-related problems, request cancellation is a good tool to have in your arsenal.

Further reading

Cancellation in Managed Threads | Microsoft Docs

Task Cancellation | Microsoft Docs

How to: Cancel a PLINQ Query | Microsoft Docs

Gonzalo Curbelo
Gonzalo Curbelo

Newest Posts

First Design Thinking Workshop at KAIZEN
Posted 16 August, 2022

First Design Thinking Workshop at KAIZEN

Carried out by Pablo Manzoni, leader of the UI/UX Design team, the 7-hour workshop was attended by 20 KAIZEN’s members from different departments such as design, software development, sales and marketing.

#Business
98% of our collaborators say we are a Great Place to Work
Posted 06 July, 2022

98% of our collaborators say we are a Great Place to Work

We are happy to be recognized as a Great Place to Work! The Certification™ Great Place to Work® is a program that recognizes the quality of the organizational culture. The methodology behind the Certification Program is based on 30 years of research to quantify organizational culture and compare it to The Best Places to Work. 98% of KAIZEN team members say we are Great Place to Work based on the implementation of a labor climate survey  (Trust Index©). The Trust Index is made up of 60 stateme

#Announcements

Book Us

You call it a challenge? We are ready, bring it on