Redis 8.4's digest-based SET option cuts a 1MB compare-and-set from 4.3ms to 1.6ms

I keep a cached object in Redis that several workers read and occasionally rewrite. Before writing, each worker has to check that nobody else changed the object since it was last read, or the write clobbers someone else's update. Redis never had one command for that. You used WATCH/MULTI/EXEC, or a Lua script that reads and compares server-side, or you accepted the race and moved on.

Redis 8.4 adds four conditions to SET: IFEQ, IFNE, IFDEQ and IFDNE. The first two compare against a literal value. The last two compare against a digest, a 16-character hex hash the server computes with DIGEST. I pulled redis:8.4 and ran all four against the patterns they replace.

The headline number

The point of a digest is that you don't send the value you're comparing against, you send its hash. For a small string that saves nothing. For a large one it matters.

I set a 1MB string, kept a cached digest of it client-side, and timed three ways of doing a conditional overwrite 50 times each, taking the median:

Those are medians across three separate runs of 50 trials each at every size, against a redis:8.4 container on localhost. At 1MB, the digest-based set was roughly a third of the classic GET-then-SET and under a quarter of the Lua script, while the data crossing the wire for the comparison itself dropped from 1,000,000 bytes to 16.

The classic approach does a GET (full value back to the client) followed by a conditional SET (full value back to the server). The Lua script sends the full old value as an argument so the server can compare it internally. The digest option sends neither; the client already has the digest from the last time it fetched the object, and this SET just needs to check it still matches.

The number that argues against it

At 100 bytes to 10KB, none of the three were reliably faster than the others. The differences at that end were inside normal jitter between runs — sometimes the digest set won, sometimes the classic one did, by fractions of a millisecond either way. The benefit only shows up once the payload is large enough that transfer time, not command overhead, dominates the round trip.

More surprising: the Lua script was the slowest option at every single size I tested, including 100 bytes. I expected it to win, since it does the comparison server-side in one round trip, the same as IFEQ does natively. It didn't. EVAL carries script-lookup overhead that a plain SET with a condition flag doesn't, and that overhead outweighed the round-trip savings until the value got large, and even then it never caught up to the native digest option.

Separately, DIGEST itself isn't free. Its own cost scales with the value, because it has to read every byte to hash it:

So the win from IFDEQ is specifically a network-bytes win, not a CPU-avoidance win. If you have to call DIGEST fresh every time rather than reusing a digest you already hold from an earlier read, you've moved the cost from the wire to the server's CPU, not eliminated it.

What it refuses

A handful of error cases, with the exact messages:

127.0.0.1:6379> DEL mylist
127.0.0.1:6379> RPUSH mylist a b c
127.0.0.1:6379> SET mylist v IFEQ a
(error) WRONGTYPE Operation against a key holding the wrong kind of value

127.0.0.1:6379> SET anotherkey v NX IFEQ x
(error) ERR syntax error

NX and IFEQ are mutually exclusive, which makes sense once you think about it: NX means "only if the key doesn't exist", IFEQ means "only if it holds a specific value", and a key can't satisfy both at once except trivially.

A more useful refusal: IFEQ and IFDEQ never succeed against a key that doesn't exist, even though you might reasonably expect "the value matches what I think it should be" to include "there is no value yet". I deleted a key, then ran SET nosuch v IFEQ anything against it, and got a plain (nil) — no error, just a silent no-op, same as a normal failed condition. If your code checks IFEQ on a key that might legitimately not exist yet, you need NX as well, or a separate existence check; IFEQ alone won't create it.

And a digest with the wrong shape is a hard error, not a silent failure:

127.0.0.1:6379> SET cs1 c IFDEQ zz
(error) ERR must be exactly 16 hexadecimal characters

Under concurrency

The whole point of these flags is optimistic concurrency control, so I tested it directly. Twenty threads, each doing fifty read-increment-write cycles against the same ~950-byte JSON blob with a counter field inside it. Expected final count: 1,000.

Plain GET then unconditional SET, no retry:

naive GET+SET   trial 1: final=109  lost_updates=891
naive GET+SET   trial 2: final=105  lost_updates=895
naive GET+SET   trial 3: final=105  lost_updates=895

Roughly 89 to 90 percent of increments vanished every single run. That's not a surprising result in itself — it's the textbook lost-update problem — but the size of it, with twenty threads on commodity hardware, was bigger than I expected going in.

Same workload, SET ... IFEQ <old-blob> in a retry loop instead of an unconditional SET:

SET...IFEQ loop trial 1: final=1000  lost_updates=0  total_retries=4892
SET...IFEQ loop trial 2: final=1000  lost_updates=0  total_retries=5111
SET...IFEQ loop trial 3: final=1000  lost_updates=0  total_retries=5111

Zero lost updates every run, which backs up Redis's own description of these flags as enabling "optimistic concurrency control patterns". But correctness wasn't free: getting 1,000 successful writes took on the order of 5,000 extra attempts, roughly five tries per success, because twenty threads hammering the same key collide constantly. IFEQ fixes correctness. It doesn't fix contention, and under heavy contention on one hot key, you pay for that in retries, not in lost data.

What I got wrong on the way

I tried the new option through redis-py's low-level execute_command escape hatch first, the same way I'd call any command the client doesn't have a typed wrapper for yet:

r.execute_command('SET', 'gtest4', 'attempted-new', 'IFEQ', 'wrong-guess', 'GET')
# -> False

I read that False as "the server doesn't return the old value when the condition fails, only when it succeeds", and nearly wrote that up as a limitation. It isn't true. I opened a raw socket and sent the same command by hand:

$ python3 -c "..."
failed-condition with GET, raw reply: b'$8\r\noriginal\r\n'

The server was sending the old value back correctly the whole time. redis-py's execute_command path has a response-type table for known commands, and it doesn't yet know that SET with IFEQ and GET together returns a bulk string on a failed condition, so it falls back to treating the reply as a plain boolean and throws the actual value away. redis-py 8.1.0 does have a proper typed wrapper for this — r.set(key, value, ifeq=..., get=True) — and that one returns the real old value correctly. It's marked experimental and prints a UserWarning on every call, which is a fair trade for a command three months old. The bug is specifically in the generic low-level path, which gives no warning at all and just quietly hands you the wrong answer. If you're reaching for execute_command because your client doesn't have the new kwargs yet, check the actual reply type before you trust it.

Watching it in production

INFO commandstats tracks these like any other command, including failed conditional writes:

$ redis-cli INFO commandstats | grep -E "cmdstat_(set|digest)"
cmdstat_set:calls=3,usec=38,usec_per_call=12.67,rejected_calls=0,failed_calls=1
cmdstat_digest:calls=1,usec=13,usec_per_call=13.00,rejected_calls=0,failed_calls=0

failed_calls on set rises every time an IFEQ/IFNE/IFDEQ/IFDNE condition doesn't match. That's a cheap way to watch contention on a hot key in production without instrumenting your own retry loop: if that counter is climbing fast relative to successful set calls, you have the same problem my twenty threads had.

Run it yourself

docker run -d --name redistest -p 16379:6379 redis:8.4
pip install redis

python3 - <<'EOF'
import redis
r = redis.Redis(host='localhost', port=16379, decode_responses=True)
r.set('k', 'v1')
print(r.set('k', 'v2', ifeq='v1'))      # True, condition matched
print(r.get('k'))                        # v2
digest = r.execute_command('DIGEST', 'k')
print(r.set('k', 'v3', ifdeq=digest))    # True, digest matched
print(r.get('k'))                        # v3
EOF

The bandwidth/latency comparison and the twenty-worker concurrency test are both plain Python scripts against that same container; the core of the concurrency test is this loop, run on twenty threads at once:

while True:
    blob = rr.get(key)
    c = parse_counter(blob)
    ok = rr.execute_command('SET', key, make_blob(c + 1), 'IFEQ', blob)
    if ok:
        break

If you're already doing optimistic locking against Redis with WATCH/MULTI/EXEC or a Lua script, IFEQ replaces the common case in one round trip and one command. If the values you're comparing are large and you already keep a digest of the last version you read, reach for IFDEQ instead of IFEQ and skip sending the payload twice. Check whichever client library you use against its raw reply before trusting the boolean it hands back on a brand-new command — the server was right in my case; the convenience wrapper around it was not.