Parallel Computing — Overview

Why the same computation can be 100× faster

Emmanuel Pilliat — ENSAI 3A

Course outline

• 0 — Julia for Python users: a short syntax bridge — 1-based arrays, end, broadcasting, structs. No performance yet.
• 1 — Interpreted vs compiled: why the same computation can be 100× faster — type specialization, benchmarking.
• 2 — Multiple dispatch vs OOP: how Julia picks the method to run, against Python’s object-oriented style — and what it costs when types aren’t concrete.
• 3 — Threads & concurrency: async I/O on one core, then real multithreading — and the parallel sum that returns a wrong answer.
• 4 — Memory & batching: the memory hierarchy, locality, and how batching turns memory-bound into compute-bound.
• 5 — GPU: thousands of threads — provided you feed them.

The red thread: estimating π by Monte-Carlo — carried from a sequential loop, to multithreading, to the GPU.

Julia as the test bench, Python as the point of comparison.

How we work

• These slides frame each session; the demos live in the notebooks.
• Every module ships three ways to run it: a Pluto notebook (live, reactive), a plain-script twin for VS Code, and a Python notebook for Colab — zero install.
• We compare the numbers between the two languages. That’s the whole point.

Note

The Python track is the common one — nobody falls behind. The Julia track is the deep dive (@btime, @code_native, @spawn, the GPU).

Synthesis

Synthesis

1. Compiling and specializing removes the per-operation overhead.
2. Multiple dispatch: generic and fast — provided the types are concrete; it even flips a computation onto the GPU.
3. Parallelizing demands minimizing sharing, otherwise the result is wrong.
4. Feeding the hardware: honour the cache, batch, saturate the GPU.

Performance doesn’t come from a magic language, but from understanding what the machine actually does: the cost of each operation, who shares what, and where the data travels.

Running through the whole course: type discovered at run time = slow; type known at compile time = fast.

Start here

• ▶ 0 — Julia for Python users — the onboarding deck.
• 📓 All notebooks and scripts: course page.