I'm trying to learn how to optimize software for very low latency on a single physical machine.
I understand the high-level concepts like Big-O and cache locality, but I'm interested in the deeper connection between software and modern CPU behavior.
What should a software engineer understand about the CPU to become genuinely good at low-latency programming?
For example, how important are cache hierarchy, TLBs, branch prediction, out-of-order execution, instruction latency vs throughput, dependency chains, SIMD, memory ordering, cache coherency, false sharing, and CPU topology?
I'd especially like to hear from people who have practical experience rather than just theoretical knowledge: what concepts or mental models turned out to matter most when optimizing real systems?