Stateless services

  • Lesson 6 of 40
  • 10 min
  • Beginner
  • 10 steps

Follow Alice across three servers and see what breaks when one of them remembers things the others do not know.

Alice's requests that work
100%
Servers useful to Alice
3
Sessions lost if one fails
0%
Extra cost per request
1 ms
AliceLoadbalancerLoad balancerServer AServer AServer BServer BServer CServer CSession storeObject storagesession: Alice · cart 2cookie: session_idround robinsession: Aliceno session for Aliceno session for Alicesticky: Alice → Asession: Alice · cart 2downsticky: Alice → Bno session for Aliceempty cartAlice · cart 2round robindownAlice · cart 2cart: 2 productssigned token: Alicesignature validavatar.png on its diskavatar.png missingbroken imageavatar.pngwrites avatar.pngavatar showsAlice · cart 2Where each kind of state goessessionsshared store or tokenfilesobject storagedatadatabasethe server keeps nothing between requests
  • request
  • response

Step 1 of 10

Concept

Alice signs in and fills her cart

Alice signs in and puts two products in her cart. The load balancer sent that request to Server A, and A notes in its memory who Alice is and what she is buying.

In return, Alice's browser gets a cookie with a session_id: the key that lets the next request find that data.

Step 2 of 10

Problem

The next request lands on another server

The load balancer spreads traffic round robin: Alice's second request goes to B and the third to C. Neither has her session in memory.

From the outside it looks like a random bug: one page shows the cart and the next asks her to sign in again. Only 1 of her 3 requests works.

What changed

  • Requests 2 and 3 → Servers B and C
  • Alice's session can no longer be found

Step 3 of 10

Solution

Patch 1: sticky sessions

You ask the load balancer to remember which server each user hit first (usually with a cookie of its own) and to keep sending them there. Alice always goes back to A and her session is there.

It is a load balancer setting: you do not touch a single line of the app.

What changed

  • Load balancer rule: Alice → A
  • Her 3 requests work

Step 4 of 10

Problem

Until A goes down

Server A crashes. The load balancer does the right thing and pins Alice to B, but the session lived in A's memory and went with it.

Alice has to sign in again and her cart is empty. With three evenly loaded servers, about a third of all users just went through the same thing.

What changed

  • Server A out of service
  • Alice's session lost
  • Cart: 2 products → 0

Step 5 of 10

Solution

Patch 2: move the state out of the server

Sessions move to a shared session store: a very fast in-memory key-value database. Each server looks up Alice's session there with her session_id.

Servers no longer keep anything between requests: they are stateless. The load balancer goes back to round robin and Alice keeps her cart wherever she lands.

What changed

  • Shared session store
  • Sticky rule
  • Extra cost: +1 ms per request (example)

Step 6 of 10

Concept

Now losing a server is no big deal

A crashes again. The load balancer stops sending it traffic, B and C read the same session from the store and Alice does not even notice.

This is what makes autoscaling possible: create servers when traffic grows and destroy them when it drops, with no coordination.

What changed

  • Server A out of service
  • Alice keeps her session

Step 7 of 10

Technique

Or let the request carry its state

Another route: put the state in the request itself with a signed token (a JWT, for example) that says "I am Alice". Any server checks the signature with its key, without asking anyone.

No store and no extra lookup. The price: a valid token is hard to revoke before it expires, and it travels with every request, so keep it small.

What changed

  • Session store
  • Signed token on every request
  • Extra cost: 0 ms

Step 8 of 10

Problem

Another case: files on the local disk

Alice uploads a profile picture. Server A saves it as avatar.png on its own disk and shows it to her just fine.

The next page is served by B, which does not have that file: broken image. A file on disk is state too.

What changed

  • avatar.png on A's disk
  • B and C cannot find the image

Step 9 of 10

Solution

Files go to object storage

Files are uploaded to shared object storage: it keeps files by key, grows almost without limit, replicates the data and every server reads it the same way.

A writes the avatar there and B and C read the same file. Even better: the browser can download the image straight from there, through a CDN, without touching your servers.

What changed

  • Object storage
  • avatar.png on the local disk

Step 10 of 10

Summary

Summary

State in a server's memory ties each user to that server.

Sticky sessions hide the problem until a server goes down.

Move state to a shared store or carry it in a signed token; files go to object storage.

With stateless servers you can add, remove and lose machines without users noticing.

You will learn

  • What state is and why it ties a user to one server
  • How sticky sessions work and where they fail
  • Why a shared session store makes servers interchangeable
  • When the request should carry its state in a signed token
  • Why uploaded files belong in object storage