What is a JIT compiler?
A compiler that translates code to machine instructions while the program is running, using information only available at runtime — which is how languages that appear to be interpreted achieve performance close to compiled ones.
The problem it solves. An ahead-of-time compiler must generate code that is correct for every possible input, so it compiles conservatively. An interpreter is flexible but slow, re-examining the same instructions on every pass. A just-in-time compiler gets both: it observes what the program actually does and compiles for that.
How it typically works:
Start interpreting, so execution begins immediately with no compilation delay.
Profile. Count how often functions and loops run and record what types actually flow through them.
Identify hot paths — the small fraction of code where nearly all time is spent.
Compile those to optimised machine code, using the observed information.
Deoptimise if an assumption is violated, falling back to the interpreter and recompiling.
Why runtime information is so powerful. A dynamically typed function might in principle receive anything, but in practice receives only integers. The JIT compiles a version specialised for integers, guarded by a cheap type check — eliminating the dispatch overhead that makes dynamic languages slow. It also inlines aggressively based on which implementation is actually called, which unlocks further optimisation.
Tiered compilation is the norm: a fast compiler produces adequate code quickly, and a slower optimising compiler replaces it for the hottest code.
The trade-offs, which explain real-world behaviour:
Warm-up. Performance is poor at first and improves — which is why benchmarks must discard early iterations, and why short-lived processes and serverless functions never reach peak speed.
Memory and CPU overhead for compilation and profiling.
Unpredictable latency, since compilation and deoptimisation happen at arbitrary moments — a genuine problem for low-latency systems.
Alternatives: ahead-of-time compilation of the same runtimes, for fast startup at lower peak throughput, and profile-guided optimisation, which feeds runtime data back into an ahead-of-time build.