Stateless services
- Lesson 6 of 40
- 10 min
- Beginner
- 10 steps
Follow Alice across three servers and see what breaks when one of them remembers things the others do not know.
- Alice's requests that work
- 100%
- Servers useful to Alice
- 3
- Sessions lost if one fails
- 0%
- Extra cost per request
- 1 ms
- request
- response
Step 1 of 10
Concept
Alice signs in and fills her cart
Alice signs in and puts two products in her cart. The load balancer sent that request to Server A, and A notes in its memory who Alice is and what she is buying.
In return, Alice's browser gets a cookie with a session_id: the key that lets the next request find that data.
Step 2 of 10
Problem
The next request lands on another server
The load balancer spreads traffic round robin: Alice's second request goes to B and the third to C. Neither has her session in memory.
From the outside it looks like a random bug: one page shows the cart and the next asks her to sign in again. Only 1 of her 3 requests works.
What changed
- Requests 2 and 3 → Servers B and C
- Alice's session can no longer be found
Step 3 of 10
Solution
Patch 1: sticky sessions
You ask the load balancer to remember which server each user hit first (usually with a cookie of its own) and to keep sending them there. Alice always goes back to A and her session is there.
It is a load balancer setting: you do not touch a single line of the app.
What changed
- Load balancer rule: Alice → A
- Her 3 requests work
Step 4 of 10
Problem
Until A goes down
Server A crashes. The load balancer does the right thing and pins Alice to B, but the session lived in A's memory and went with it.
Alice has to sign in again and her cart is empty. With three evenly loaded servers, about a third of all users just went through the same thing.
What changed
- Server A out of service
- Alice's session lost
- Cart: 2 products → 0
Step 5 of 10
Solution
Patch 2: move the state out of the server
Sessions move to a shared session store: a very fast in-memory key-value database. Each server looks up Alice's session there with her session_id.
Servers no longer keep anything between requests: they are stateless. The load balancer goes back to round robin and Alice keeps her cart wherever she lands.
What changed
- Shared session store
- Sticky rule
- Extra cost: +1 ms per request (example)
Step 6 of 10
Concept
Now losing a server is no big deal
A crashes again. The load balancer stops sending it traffic, B and C read the same session from the store and Alice does not even notice.
This is what makes autoscaling possible: create servers when traffic grows and destroy them when it drops, with no coordination.
What changed
- Server A out of service
- Alice keeps her session
Step 7 of 10
Technique
Or let the request carry its state
Another route: put the state in the request itself with a signed token (a JWT, for example) that says "I am Alice". Any server checks the signature with its key, without asking anyone.
No store and no extra lookup. The price: a valid token is hard to revoke before it expires, and it travels with every request, so keep it small.
What changed
- Session store
- Signed token on every request
- Extra cost: 0 ms
Step 8 of 10
Problem
Another case: files on the local disk
Alice uploads a profile picture. Server A saves it as avatar.png on its own disk and shows it to her just fine.
The next page is served by B, which does not have that file: broken image. A file on disk is state too.
What changed
- avatar.png on A's disk
- B and C cannot find the image
Step 9 of 10
Solution
Files go to object storage
Files are uploaded to shared object storage: it keeps files by key, grows almost without limit, replicates the data and every server reads it the same way.
A writes the avatar there and B and C read the same file. Even better: the browser can download the image straight from there, through a CDN, without touching your servers.
What changed
- Object storage
- avatar.png on the local disk
Step 10 of 10
Summary
Summary
State in a server's memory ties each user to that server.
Sticky sessions hide the problem until a server goes down.
Move state to a shared store or carry it in a signed token; files go to object storage.
With stateless servers you can add, remove and lose machines without users noticing.
← → step · R replay · Space pause
You will learn
- What state is and why it ties a user to one server (covered)
- How sticky sessions work and where they fail (covered)
- Why a shared session store makes servers interchangeable (covered)
- When the request should carry its state in a signed token (covered)
- Why uploaded files belong in object storage (covered)