Multicore and Chip Multiprocessors

A chip multiprocessor integrates several processor cores on a single die, sharing caches and a memory interface, so that thread-level parallelism rather than higher clock speed drives performance growth.

Definition

A multicore processor, or chip multiprocessor, is a single integrated circuit containing multiple independent processor cores that share some on-chip resources and a memory system, designed to execute multiple threads or programs concurrently.

Scope

This topic covers the organization of multicore chips: core counts and heterogeneity, shared and private cache hierarchies, on-chip interconnects, and how performance scales with cores under Amdahl's law. It treats the hardware structure of multicore systems. It excludes the coherence and consistency mechanisms shared with other shared-memory topics (shared-memory-and-coherence, cache coherence protocols) and the single-core engine (processor microarchitecture).

Core questions

• Why did processors move from single fast cores to multiple cores?
• How are cores, caches, and interconnect organized on a chip?
• How does Amdahl's law bound the speedup from adding cores?
• When are heterogeneous cores preferable to identical ones?

Key concepts

• chip multiprocessor
• thread-level parallelism
• shared vs private caches
• on-chip interconnect
• core count and scaling
• heterogeneous (big.LITTLE) cores
• Amdahl's law
• power and thermal limits

Key theories

Amdahl's law for multicore scaling

The serial fraction of a workload limits the speedup achievable by adding cores; this bound explains why simply increasing core count yields diminishing returns and motivates reducing serial bottlenecks and using heterogeneous designs.

Mechanisms

A multicore chip places multiple cores on one die, typically with private first-level caches, a shared last-level cache, and an on-chip network or ring connecting them to memory controllers. Workloads gain performance by running threads on different cores. Heterogeneous designs combine large high-performance cores with small efficient ones, scheduling work to the most appropriate core to balance speed and energy.

Clinical relevance

Multicore is the dominant processor design today, from phones to servers, because frequency scaling stalled under power limits. Its rise shifted the burden of performance onto parallel software: applications must be multithreaded to benefit, making concurrency a mainstream programming concern and Amdahl's law a practical constraint on real systems.

History

Research chip multiprocessors such as Stanford's Hydra in the 1990s anticipated the multicore era. The breakdown of frequency scaling around 2004–2005 pushed mainstream CPUs to dual- and then many-core designs. Heterogeneous architectures combining performance and efficiency cores later became common, especially in mobile and laptop processors.

Debates

More cores versus stronger cores

Given fixed power and area, designers debate whether to add many simpler cores, fewer powerful cores, or a heterogeneous mix; the best choice depends on how parallel the target workloads are, with Amdahl's law favoring strong cores when serial fractions are significant.

Key figures

• Gene Amdahl
• John L. Hennessy
• David A. Patterson
• Kunle Olukotun

Seminal works

• hennessy2019
• amdahl1967

Frequently asked questions

Why doesn't doubling the number of cores double performance?

By Amdahl's law, any part of a program that must run serially limits the benefit of added cores. Communication, synchronization, and contention for shared resources add further overhead, so real speedup is typically well below the ideal linear scaling.

What are heterogeneous (big.LITTLE) cores?

Heterogeneous designs combine large, high-performance cores with smaller, energy-efficient cores on the same chip. The system schedules demanding work on the big cores and light or background work on the little cores, improving energy efficiency without sacrificing peak performance.