3 — Threads & concurrency

Waiting is overlapped; computing must be split

Emmanuel Pilliat — ENSAI 3A

Goals

• Concurrency: overlap waiting, one core — @async/@sync.
• Parallelism: split computing, many cores — @spawn.
• The trap: a parallel sum that lies (race condition).

▶ Script · Notebook · 🐍 Python · ✏️ Activity · ✏️ Activity (Colab)

⬅ Previous: 2 — Multiple dispatch vs OOP · Next: 4 — Memory & batching ➡

Parallelism starts here: compilation and dispatch made one core fast; now we use many. Script: predict, then run each cell.

Two problems, two tools

Concurrency ≠ parallelism

	problem	fix	cores
Concurrency	waiting (network, disk)	overlap the waits	1
Parallelism	computing	split the work	many

Waiting is overlapped; computing must be split.

Pick the wrong one: all the complexity, none of the speed. We will measure it.

Part A — async: overlap the waiting

Fetching 15 prices

• 15 HTTP requests to Binance, ~0.25 s wait each.
• One by one = 15 waits in a row, CPU idle.

@async expr     # start a task, return at once
@sync begin     # wait for every @async inside
    ...
end

One core is enough: only waiting to overlap.

Three ways to fetch

for i in 1:n                         # 1) sequential
    seq[i] = fetch_price(SYMBOLS[i])
end
for i in 1:n                         # 2) @async, no @sync
    @async bad[i] = fetch_price(SYMBOLS[i])
end
@sync for i in 1:n                   # 3) @sync + @async
    @async good[i] = fetch_price(SYMBOLS[i])
end

Bet first: how long does each one take?

▶ Script: cell “Three ways to fetch”

Three ways to fetch: the timings

	time	what happened
1) sequential	~4 s	sum of 15 waits
2) @async without @sync	~10 µs	⚠️ timed task creation only
3) @sync + @async	~0.3 s	≈ one round-trip

• Row 2: “400 000× faster” — wrong, data not there yet.
• @sync: honest timing and correct result.

~4 s → ~0.3 s on one core.

Network-dependent: your latency will differ. The shape (sum of latencies → one round-trip) is the lesson, not the exact seconds. Run it live if the wifi allows. The script prints “x/15 ready” for row 2, then again 2 s later: 15/15. Async overlaps waiting, not computing.

Julia didn’t wait; Python didn’t start

	Julia @async	Python async def
the task is…	eager: running	lazy: a coroutine object
ready at once	0 / 15	0 / 15
2 s later	15 / 15	0 / 15: never started

• Julia: you didn’t wait → @sync.
• Python: never awaited → await / asyncio.gather.

asyncio.gather = @sync + @async.

Same bug in both languages, from opposite causes.

Part B — @spawn: real parallelism

Splitting the computing

t = Threads.@spawn f()   # may run on another core
fetch(t)                 # wait, get the result

• Needs julia --threads=auto (else nthreads() = 1).
• Our machine: Threads.nthreads() = 22.

@async and @spawn look alike. Only one uses more cores.

First split: independent Markov chains

markov_seq(K, L) = [simulate_markov(L) for _ in 1:K]
markov_spawn(K, L) =
    fetch.([Threads.@spawn simulate_markov(L) for _ in 1:K])

• K = 8 chains, L = 2×10⁷ steps, no shared state.
• Bet first: speedup on 8 chains?

▶ Script: cell “Predict the speedup on 8 chains, then run”

Independent tasks: each returns, fetch collects.

Same CPU work, with @async

Bet first: π (n = 10⁸), @async × 22: speedup? ▶ Script: cell “Same CPU work with @async”

	time	speedup
pi_seq (one core)	0.357 s	—
@async × 22	0.332 s	1.1×
@spawn × 22	0.057 s	6.3×

Concurrency ≠ parallelism, as a number.

count_in_disk is the same count as in the compilation module; only the macro changes (who runs it). @async: one core, nothing to overlap. @spawn: the tasks land on different cores. Script cells “Same CPU work with @async” (seq, @async) and “Speedup vs nthreads” (@spawn). @async never passed ~1.3x at any task count tried (1 -> 22). No speedup-vs-ntasks curve here: in one session the OS may place few tasks on 5 GHz P-cores or 2.5 GHz LP-E cores (measured 1.3x at 4 tasks but 4.7x at 6). The honest curve needs one process per point: experiences/threads_scaling/.

Bet before you look 🎲

π on all 22 threads, 20 million points, one task per thread.

What does it print?

▶ Script: cell “π on all threads, one task per thread: what does it print?”

The parallel sum that lies

hits = 0
@sync for _ in 1:ntasks
    Threads.@spawn for _ in 1:per
        x = rand(); y = rand()
        hits += (x * x + y * y <= 1.0)   # ⚠️ shared
    end
end

	run 1	run 2	run 3	run 4
pi_race	0.219	0.361	0.354	0.393
pi_spawn	3.14126	3.14174	3.14172	3.14146

Why it lies

• 10× too small, different every run.
• hits += 1 = read, add, write: increments get lost.

Warning

No error. No warning. Just a wrong number.

Your values WILL differ — that is the point (non-determinism). Never quote a fixed number here; run it live and let the class watch it change. Two tasks read the same value → one increment is lost.

The cure: return and reduce

function pi_spawn(n; ntasks=Threads.nthreads())
    per = n ÷ ntasks
    tasks = [Threads.@spawn count_in_disk(per) for _ in 1:ntasks]
    return 4 * sum(fetch, tasks) / (per * ntasks)
end

No shared counter in the hot loop: correct and fast.

▶ Script: cell “The cure”

Each task returns its own count; we add them up once, at the end. Same pattern as the Markov chains.

Why not an atomic counter?

Threads.Atomic makes hits += 1 safe. Correct? Fast?

▶ Script: cell “Why not an atomic counter?”

22 threads, vs pi_seq (1 core)
pi_spawn (return + reduce)	6.3× faster
pi_atomic	~7× slower

Correct, but catastrophic. The cure is less sharing, not a bigger lock.

22 cores fight over one memory location, once per draw. Touch shared memory once per task instead. Ratio measured at n = 4e7 (7x; 8.8x in power-saver mode); it does not depend much on n.

Amdahl: speedup ≠ number of threads

Bet first: speedup on 2, 4, 22 threads? ▶ Script: cell “Speedup vs nthreads” (--threads=1, 2, 4)

threads	1	2	4	22
speedup	1×	1.8×	3.1×	6.3×

• Amdahl: the sequential part caps the speedup (10 % → ≤ 10×).
• SMT: 22 logical threads, 16 physical cores.
• Unequal cores (next slide).

Monte-Carlo π, @spawn, one Julia process per thread count. Sequential part here: splitting, spawning, the final reduce. SMT = simultaneous multithreading: two threads share one core. Never say “speedup ≈ number of threads”. 1/2/4 threads: experiences/threads_scaling/ (one process per point; @async stays ~1x on the same plot). 22 threads: the @async vs @spawn table above.

Equal work, unequal cores

6 fast (5.0 GHz), 8 medium (3.8 GHz), 2 slow (2.5 GHz) cores.

22 equal π tasks, each timed (demo, not in the script):

fastest task	slowest task
0.022 s	0.051 s (2.3×)

The slowest task sets the wall clock. Fix: more, smaller tasks.

More, smaller tasks so the scheduler can rebalance (work stealing). Same trap in BLAS on this box: 8 threads -> 2.7 ms, 22 threads -> 39 ms (14x slower).

And in Python? The GIL

• GIL: one thread runs Python bytecode at a time.
• Released while waiting on the network.

ThreadPoolExecutor on…	speedup
I/O (15 URLs)	~10×
CPU (pure-Python loop)	~1×

“Threads are pointless in Python”: false, only for CPU. Julia has no GIL.

Same threads, same GIL; what changed is waiting vs computing. Ageing caveat: true for default CPython. Free-threaded builds (PEP 703) are officially supported since 3.13/3.14 and remove the GIL — say “default CPython”.

When does what help?

	I/O-bound (waiting)	CPU-bound (computing)
Python	threads (GIL released), asyncio	multiprocessing
Julia	@async / @sync (one core)	@spawn / fetch (many cores)

Waiting is overlapped; computing must be split.

Going further

• ▶ Script: 3_threads_and_concurrency.jl
• ▶ Notebook: Threads & concurrency
• 🐍 Python: 3_threads_and_concurrency_python.ipynb
• 🔬 Amdahl curve, @spawn vs @async: experiences/threads_scaling/
• 🎯 A race condition does not crash. It lies.

⬅ Previous: 2 — Multiple dispatch vs OOP · Next: 4 — Memory & batching ➡