How JIT Compilation Uses Code Caching in Java

What the JIT Looks for When Choosing What to Cache

Image Source

As a Java program runs, the code you wrote isn’t executed line by line as-is. Instead, it gets translated and optimized during runtime by something called the Just In Time compiler, or JIT for short. The JIT is part of the Java Virtual Machine, and one of its biggest jobs is turning Java bytecode into native machine code while the program is running. But once that native code is produced, the JVM doesn’t throw it away.

How the JVM Stores Native Code During Execution

Java code doesn’t go straight from source code into machine instructions. First, the Java compiler turns your source files into bytecode. That bytecode runs inside the JVM, which starts by interpreting each method one instruction at a time. But interpretation alone doesn’t deliver the speed people expect from Java. To fix that, the JVM has a built-in compiler that watches how the program behaves while it runs. It starts looking for code that gets used often, and once it finds it, it compiles that bytecode into machine instructions made for your computer. Instead of recompiling that native code over and over again, it holds onto it using something called the code cache.

This cache is a part of memory dedicated to holding the compiled machine code created at runtime. Once a method has been compiled, any future calls to that method will skip the interpreter. The JVM jumps right into running the compiled version stored in memory. This not only saves time but keeps performance stable once the program gets into a rhythm. The JVM tries to make smart decisions about what to compile and when to keep it, but it all starts with detecting patterns during execution.