Computer Architecture: The Big Picture

Computer Architecture is the field that focuses on the design and construction of computing machines. Computing is now specialized into many areas, including software, hardware, microcontrollers, supercomputers, cloud computing, edge computing, networking, AI, accelerators, processors, theory, and the list goes on and on. Regardless of the specialization, computer architecture’s goal is to reconcile often-conflicting requirements to produce computers that work well and remain within appropriate budgets.

Human computers

Humanity has been building electronic computers since the 1940s. Still, the idea of using tools for computation goes back about 10,000 years, to clay tokens used for accounting and record-keeping in the ancient Near East. These were useful for administration, especially as temple and palace economies grew and needed to manage contributions, rations, and redistribution of resources. Denise Schmandt-Besserat argues that these tokens actually led to the development of writing. The idea is that tokens were eventually sealed inside hollow clay envelopes (called bullae) and, to record what was inside without breaking them open, people pressed the tokens into the envelope’s wet surface, leaving impressions. Those impressions, the theory goes, gradually became abstract marks made with a stylus, which fed into the development of cuneiform, the early Mesopotamian script, around 5000 years ago. It’s unclear whether this theory is actually correct, but wouldn’t it be interesting if it were? Boring accountants to be the driving force behind the creation of writing and millennia of human progress?

At the dawn of writing, the Sumerians already had a counting board about 5000 years ago: a flat surface divided into columns, each representing a different order of magnitude. One place counters (pebbles or clay tokens) in the columns, added or subtracted by moving them, and “carry over” by clearing a full column and adding a counter to the next one up. The Sumerian version would reflect their base-60 system with a sub-base of 10, so the columns alternate between groupings of 10 and 6 to build up powers of 60, mirroring how their cuneiform numerals worked.

An interesting thing about these early computers is that they make sense to us today: they identified a problem to solve, had a set of constraints (technological, monetary, social, etc.), and then proposed tools and steps (an algorithm) to break the problem down and simplify it. Human computers, who could speed up calculations, were still in use in the 1950s, and a fictional form of which was featured in The Three-Body Problem more recently.

Thinking of what a processor does in terms of communicating humans may make the complexity of a processor seem less daunting. The humans perform a set of predetermined tasks every time a conductor waves a baton from the top of a ladder. Instructions for each human must fit on a small card, so each has only a small set of tasks to carry out between waves. I could go on refining this model, but I suspect you could as well: how far apart should the humans be to still be able to signal to each other? How big should the card be for instructions? How fast should the conductor wave his baton? Is there a separate card used for signaling? Do we arrange humans by height so that they can see further? How and when are the cards with instructions issued? Can humans communicate with each other by voice? If it’s voice, how do we coordinate that they don’t all talk at the same time (so that everyone can hear)? And so on.

You could play this game and potentially reinvent everything in computer architecture from first principles. The details of your solution will depend on your assumptions, but you could tune them to mirror the constraints of a silicon-based processor. As a human, you are already an excellent computer. Use that perspective to explore how interacting with others could amplify your computational abilities without any additional electronics. Imagining this will help you understand the characteristics of algorithms implemented in processors, and you might also find this approach useful for better understanding other humans and organizations.

The contract

Computer architecture links two large and varied communities: those who build software and those who build hardware. Software cannot exist without hardware, and complex hardware relies on software to simplify it. Given the right architecture, one practicing in one area need not understand everything about the other. However, a computer architect needs to understand both well enough to design good abstractions and to make well-justified engineering trade-offs. A Computer Architect’s main job is to predict and reconcile product requirements, technical feasibility, and cost, often years before the product is built.

The contract between software and hardware is expressed in the instruction set architecture (ISA) that has traditionally been difficult to change due to the cost of reoptimizing software for a new ISA. Even in the age of “magical” compilers and AI-based code generation, moving from one instruction set to another remains a costly proposition: not just because low-level libraries have to be reoptimized, but because updates then need to trickle through the ecosystem and be made available for others to use.

We talk about the “attention economy” when social networks aim to capture more and more of their users’ attention, but attention has a cost (and a price) in most other contexts as well. Getting companies’ attention to change what they are already doing, even when the change is to their advantage, is complex and time-consuming. Most companies won’t jump at the opportunity to support a new ISA and incur all the update and verification costs associated with it, and they definitely won’t do it right away unless enough of their customers are demanding it; they have more important and profitable things to do.

This poses a chicken-and-egg problem when establishing a new ISA and asking software vendors to support it: a new architecture is made useful by the software that runs well on it, but software won’t run well unless someone spends the effort to optimize it for an ISA. However, sometimes this Gordian knot can be cut. Apple and its software ecosystem have moved from Motorola 68K processors to PowerPC, then to x86, and most recently to ARM. They pulled this off by providing an emulator in macOS, allowing existing software to continue running on the new architecture on day one, giving time for the new ISA to be broadly adopted before software has to be ported. The transition period for Apple’s software ecosystem has been 5-7 years.

These and other forces caused the industry to converge on only two dominant instruction sets: x86 and ARM. Intel, AMD, ARM, and their ecosystems devote considerable effort to creating and optimizing software for their architectures. This costly and long-term effort removes barriers to adopting new features.

ISA evolution

What makes one instruction set better than another is very much in the eye of the beholder. Early instruction sets were optimized for efficient encoding and ease of implementation, saving memory and design time. Later ones were optimized for assembly-writer productivity, then exposed more instruction-level parallelism (ILP) to the compiler. After that, support for larger address spaces became the driving concern. All of these changes tended to coincide with major changes to the instruction set. At first, this wasn’t a big issue, as there wasn’t much software around. You most likely have not heard of Digital Equipment Corporation because it never managed to transition from minicomputers to a mass-market computer maker and was eventually acquired by Compaq. This was partially due to its unsuccessful transition from the VAX to the Alpha instruction set, which eventually led to the company’s demise. A clean new architecture is attractive to engineers, but the real value of an ISA is in the software ecosystem that relies on it, so it’s best to stay compatible as much as possible.

Throughout the evolution of an architecture, architects have found ways to add significant new features (e.g., virtual memory and 64-bit addressing) to an existing ISA while preserving sufficient backward compatibility. They have also improved processing efficiency through specialized instructions that optimize for specific computation kernels; most recently, security and AI have been key areas of focus. The nice thing about specialized instructions is that they can usually be added without a wholesale change to the instruction set, “under the hood.” Every instruction set has a currently unused encoding space that can be populated with new instructions to optimize for new workloads as needed; in fact, architects plan for possible future changes to minimize disruption.

At the end of the day, expected future costs and related market opportunities are what drive the evolution of architectures. However, deploying that change can take much longer than anyone would expect. In 2026, 8-bit and 16-bit microcontrollers still account for about 30% of microcontroller unit volume, despite ARM and its partners having had competitive products in that space for almost 20 years. It takes a lot to get people to abandon what they are already used to. Having said that, almost all of the remaining 32-bit microcontrollers are ARM-based, so change is definitely possible; it just takes a long time.

Because of the “stickiness” of software (and therefore of instruction set architectures), it is usually preferable to innovate at the microarchitecture level whenever possible. The microarchitecture’s job is to efficiently execute the instructions defined in the ISA in hardware. How much can be implemented in the microarchitecture, and how much should be left to programmers and compilers through the ISA, depends on the available area and power budgets. ISA changes push the cost to the software ecosystem, whereas microarchitectural changes push it to the processor maker. Power, Performance, and Area (PPA) are key objectives for hardware designers, serving as proxies for what’s feasible from a cost perspective.

Cambrian explosion

There was a golden age of microarchitecture design from about the 1980s until the mid-2000s, benefiting from Moore’s law and Dennard scaling. There were many companies designing processors, as chips could contain an ever-increasing number of transistors at attractive price points, and demand for them was growing as electronics were infiltrating consumer and business society. This meant that architects could dream big, experiment, and enable exponential improvements in performance and capabilities, without the customer having to pay more.

The biggest beneficiaries of this era were Intel and Microsoft, who created a flywheel for decades of improvements. Microsoft and the DOS/Windows ecosystem created software that pushed the limits of what was feasible on a processor, and Intel made that software work better with an annual upgrade.

Intel’s accomplishment in this era is in no way trivial. This era saw most other processor makers go out of business or barely survive due to poor execution, eventually giving Intel a monopoly position that it could leverage to achieve incredible profit margins. What Intel did was build a tightly integrated organization that could predictably deliver working processors that made good use of the exponentially increasing transistor budgets. This tight integration, which worked so well for Intel then, is also the reason it has struggled to stay competitive over the last decade or so.

While great from a technological perspective, Moore’s law scaling was a coordination nightmare: teams had to plan for 5+ years when estimating PPA budgets, and even a small delay in a project could mean they would lose their competitive advantage. This is because of the exponentials: a simpler design delivered on time could erode all the advantages of a more complex design that was only a few months late. Any proposed complexity had to be balanced with predictable execution. Most of the benefits came from the available process technology, so miscalculations in transistor parameters and their number (yield) caused major disasters. Sustaining it required extensive coordination among equipment makers, process engineers, packaging engineers, processor designers, verification engineers, architects, and many others. Intel would even have multiple competing design teams, targeting the same release date to manage execution risk.

Slow burn

ARM was a different kind of company from the beginning. For one thing, it never had its own manufacturing lines (fabs), so it had to rely on others to turn its designs into usable silicon. This meant that it had to develop a patient business model that encouraged adoption and created increasingly powerful network effects to “pull through” ARM’s designs. The company got a lucky break when it was used in Nokia’s mobile phones through chips supplied by Texas Instruments starting in the early nineties, and it became the architecture that every mobile phone ended up using. Architectural standardization helped both silicon vendors design compatible chips and software vendors build reusable software components.

Because of the markets it served, ARM was always very interested in designing small, efficient processors. In general, keeping the number of transistors low usually results in more power-efficient and also cheaper designs. This served the needs of the emerging systems-on-chip (SoC) companies very well, who addressed a wide range of application areas from automotive, networking, consumer electronics, and mobile phones. These areas were considered embedded designs, mostly with proprietary software stacks. This meant that ARM flew under the radar, as far as “real” computer companies were concerned; it served a niche, a very cost- and power-sensitive market from the beginning.

Towards the end of Moore’s law, things changed: power suddenly became a first-class design constraint, and ARM emerged with an architecture that offered inherent advantages and an established software ecosystem, making it suitable for real high-end computing. It was Intel that made ARM an “overnight success” when it started playing up the threat ARM posed and talking loudly about it at analyst meetings.

But ARM had inherent advantages over Intel: it had a large number of silicon partners to proliferate across every area of computing, who had ambitions to beat Intel at its own game, and it was cheap. ARM was a company built on patience and network effects. So when Moore’s law ended, there was now an architecture mature enough to compete with Intel’s x86 and give adopters a cost and power advantage.

Evolution

ARM started designing its 64-bit architecture in 2001. The first processor to ship in volume with that architecture was the Apple A7 in the iPhone 5S in 2013. The first hyperscaler to adopt ARM was AWS in 2018, when it introduced AWS Graviton. Twelve years of development until the fruits of that labor started to bear royalties in the mobile space. Seventeen years until the architecture finally ended up in cloud servers in a meaningful way.

These are not unusually long time periods for adopting a new architecture in a new market segment. While Intel had already dominated desktop computing in the 1990s, the server market was new to them. They started with the Pentium Pro in 1995, launched the Xeon server brand in 1998, and, to cinch the deal, introduced a new 64-bit architecture called IA-64 (and the Itanium processor) in 2001. It largely failed because it was late, expensive, and slow to run existing 32-bit code, so Intel’s high-end takeover didn’t happen as planned, and allowed AMD to shine. AMD evolved x86 to a compatible 64-bit address space in the mid-2000’s. Eventually, Intel adopted these extensions, thereby unifying the x86 space and dominating the server market by the end of that decade.

When evolving instruction sets, it’s worth taking an incremental approach whenever possible. This incrementalism is evident in the messy encodings of both the ARM and x86 architectures: they evolved through pragmatic additions and clever compromises, which show up as counterintuitive instructions and encodings. These instruction sets evolved in a backward-compatible way due to microarchitectural innovations enabled by Moore’s law. Increased transistor counts allowed architects to allocate them to features such as register renaming, decoded instruction, trace, and micro-operation caches. What started as a war of competing philosophies for designing instruction sets ended in a truce: CISC and RISC processors came to resemble each other.

Disruption

In the 2010s, the ARM architecture and its processors were becoming competitive with the best that the traditional, merchant silicon manufacturers could build. Merchant silicon is the term for a processor available on the open market that anyone can buy (captive silicon is a term for a company’s own designs). Contrast this with AWS Graviton or Apple’s M and A series processors, which are designed in-house and are only available for use in the company’s own products. The fact that these companies can design their own processors cost-effectively is the benefit they derive from ARM’s business model and the decades of software investment in the ARM ecosystem.

Captive silicon (for large companies) becomes attractive as a consequence of the end of Moore’s law. Companies still want to deliver the benefits of an increased number of transistors to their customers, but these extra transistors don’t come for free any longer; they cost real money: the manufacturing cost goes up, and so does the non-recurring engineering cost (NRE) associated with a design. Vertical integration helps not only by enabling a more optimal solution, but also by avoiding margin stacking across the supply chain. By bringing design and production in-house, companies can eliminate paying merchant silicon providers’ profit margins and more easily spend that money on increasing the transistor count.

NVIDIA is the latest “overnight success” of the processor wars. It was founded in 1993, but made most of its money on graphics processing until recently. Building accelerators is usually not considered to be a good business to be in. Accelerators are hidden behind software APIs, meaning that whatever great acceleration the hardware provides can be easily swapped out for a different vendor’s, creating competition on pricing and profit margins. However, NVIDIA persevered and doubled down on its niche, which, while very hard to program, offers readily available acceleration. They also fostered a company culture that allowed it to jump straight into new markets like crypto and use the experience to make its architecture as broadly applicable to acceleration as possible. Their success in AI is the payoff of that strategy, and they are leveraging their position to become the premier general-merchant silicon provider before AI acceleration commoditizes the market.

My perception is that companies that set out to be disruptive seldom are. Either because they underestimate how long it takes to really do that (their timing is wrong or funding is insufficient), or they underestimate the countermeasures potential disruptees can deploy. In ARM’s case, it managed to fundamentally change a market because it was not perceived as a threat to the merchant silicon providers for almost two decades (it operated in a different market). In NVIDIA’s case, it was the resilience that comes with being “just a peripheral” long enough, along with the belief that an eventual “next big thing” will bring them success. Both of these companies looked like commodities, easily replaceable, until one day they proved difficult to avoid.

Not the end times

The processor industry always looks pretty stable at any given year, only for things to suddenly change overnight. One of the reasons for this has been the long planning cycles across many companies as they develop highly complex and expensive technologies, and the failure of some long-term but necessary innovations. Another has more to do with the business cycles.

All companies have lifecycles, and successes eventually fade. IBM used to be the king of the hill, having coined the term “computer architecture” and invented or deployed most of the key ideas in computers decades before the current cycle began in the 1980s. However, they lost their crown long ago. Commodore was the highest-volume microcomputer manufacturer in the 1980s; it went out of business in 1994. Nokia was the number one mobile phone brand for about two decades, but now they are a “B2B technology innovation leader,” doing not sure what. Apple almost went out of business in 1998 and was rescued by a loan from Microsoft and sales of its ARM shareholdings after ARM IPOed.

Multiple forces drive these cycles, but one in particular has a witty name: “enshittification.” Cory Doctorow used this to describe the pattern by which digital platforms progressively degrade. First, delighting users to lock them in, then abusing those users to benefit business customers, then squeezing those business customers to claw value back for the platform itself, until the service becomes a hollowed-out shell optimized for extraction rather than usefulness.

If you look beyond platform companies, enshittification is a byproduct of prioritizing short-term profit over long-term customer relationships, leading customers to move elsewhere or to stay and become resentful. Customers leave if they have options, causing a relatively quick death. If they are forced to stay because they cannot move elsewhere (due to switching costs, network effects, or a lack of a good alternative), they’ll punish the company in other ways. An example of the latter is Microsoft, which was resented for practices around its Windows monopoly, as it was unable to enter the mobile space in any meaningful way. They only exited purgatory a decade and a half later, in the 2010s, when the collective unconscious decided that while Microsoft mobiles were a no-go, its cloud was okay.

Computer evolution doesn’t stop just because Moore’s law has ended. It’s just that this end means certain assumptions have to change, which affects companies and individuals. As Garth from Wayne’s World said, “We Fear Change.” Despite the Buddha’s best efforts, a narrative that explains how humans and companies tend to behave is that they fear change and do whatever they can to avoid it. When they think they cannot avoid it any longer, they often try to change too much. Neither the fear nor the overreaction is usually the best path forward. But how do we know what the right path might be?

Narratives

Stories are central to the human experience. We can trace how the stories the Sumerians recorded on cuneiform tablets 4000 years ago made their way into the Old Testament millennia later (e.g., The Flood, Creation, Humans made of Clay, Confusion of Languages, and many others). The way most impactful movies are organized today follows a pattern called the “Hero’s Journey,” which was first recorded in the Epic of Gilgamesh 4000 years ago, and popularized by Homer for modern audiences in the Odyssey about 3000 years ago. The contents convey complex ideas that various societies deemed important, making them broadly accessible and providing blueprints for solving problems.

I started the story of computer architecture in medias res to give you a few narratives to help you make sense of the material you will encounter. There will be lots of others during the course. What will make you successful is really understanding ideas at the “big picture” level, not just the technical details. Knowing why a certain solution was chosen (and why others weren’t) is more important than the exact details. In fact, comparing multiple implementations helps you focus on the important details: what are recurring ideas and what are one-offs. The lectures will focus primarily on the narratives and the issues around a set of key ideas. However, you will have to study their implications and implementation details outside of class and bring back questions for further discussions.

Take branch prediction as an example: if you really grok why branch prediction is unavoidable in processor design, and what really matters about them, you will have learned a much more fundamental set of insights than knowing everything about designing a specific predictor. These insights will help you prioritize problems and efficiently tune a predictor within available budgets when you actually have to build one. Of course, you will need to figure out how to build one during the course, but the constraints on its implementation will change when you build one again in the future (misprediction pipeline depths, update policies, cycle budget, throughput requirements, etc.), so you have to understand the design at multiple levels. You will succeed if you have captured the right narratives for yourself so that you can make the right low-level decisions in the future in light of them.

Labs are designed to give you a hands-on introduction to building a processor that will evolve into a group project during the term. The most important part of this project is the times when you are stuck. This is when it becomes apparent that there is a narrative mismatch somewhere in your mind, and you have to figure out where exactly it is. You had one model of the processor and your infrastructure in mind, and it turns out to be incorrect. This is where you need creativity and self-awareness to guide your explorations efficiently. It will take hours at first to find bugs that will seem trivial in retrospect. By the end of the course, you will have practiced creative problem-solving enough and will be much quicker (bugs will still look trivial in retrospect).

Four pieces of specific advice: 1) Always draw and redraw diagrams about hardware, as it helps clarify your own understanding, and communicate your design more effectively. 2) Do not start writing Verilog right away; the apparent similarity with other programming languages is misleading. 3) Debug with your peers, always show the latest picture of your design when you ask a question. The main goal of the project is for you to improve your problem-solving skills and arrive at a good-enough solution. 4) Good teamwork requires over-communication. Work together on your projects face-to-face as much as you can. Don’t expect everyone to contribute equally, but try to maximize the ways and places where each of you can make contributions. Try to see your teammate’s issues from their perspective, not just your own.

Learn many narratives, and when you encounter a situation, analyze the course of action they suggest. Don’t be afraid to be skeptical and to criticize them. The hardest thing to decide is usually which narrative actually applies, and you can only extract that essence through criticism. To choose the best narrative takes curiosity, experience, and judgment. Harvard Business School is famous for using case studies to guide businesses, and the same narrative approach can be applied effectively to understand the right trade-offs in computers.

Competing narratives

Understanding the real engineering constraints and what you can do about them goes way beyond understanding the technology: you have to understand how a piece of technology fits with others and gets used, and what real impact that can have on its users. When Apple moved the Mac to the ARM architecture in 2020 with the M1 processor, the speedup and efficiency gains were astonishing compared to its Intel predecessors. At the surface, this could seem like a story that says Intel missed the plot and that ARM and Apple were much more clever. A better explanation turns out to be less divisive.

A key thing about computers is that what really matters is the memory system (how quickly data can be moved to the processor when needed). What Apple did was borrow the in-package integrated DRAM-and-processor approach from its mobile phones and apply it to its PCs. Seems like a relatively easy thing to do, but it has major implications for the supply chain, how DRAM is bought and priced, and explains why it hasn’t been a mainstream approach in PCs before. Processor makers have traditionally aimed to commoditize DRAM and make it as cheap as possible. They also never had to buy it before to include it with their processors; hedging its price based on demand was the PC maker’s job (who isn’t usually the processor maker). This change was easy for Apple to make because its mobile phone business was already doing it; it was already vertically integrated, but it was difficult for others who weren’t used to doing so. All of this translated into a step-function improvement in performance and efficiency, and it still does, while the rest of the industry is playing catch-up. Being able to execute on this is actually what differentiates Apple.

While lots of people talk about innovation and disruption, it is a herd mentality in the computer industry. Once one player does something that seems good, others quickly copy it, and there are long periods when nothing out of the ordinary happens: here is a new processor; it’s marginally better than the previous one. Still, of course, it’s the world’s best at something or other. Because it’s so expensive to design and manufacture new chips, building on what already works and not changing things is the right approach: “if it ain’t broke, don’t fix it.” This means that technology development is highly path-dependent; even if better solutions exist for a problem, the threshold for putting them into a product is very high: anything new increases perceived risk. This may be a frustrating experience, but is it irrational?!

Something like this might be happening with how Moore’s law is used in marketing today. Companies keep saying they are on track or outpacing it, even though, as Moore framed it, the law has been dead for a while. What these companies mean is that they are using more transistors than before, but Moore was talking about equivalent cost when doubling their number. For some reason, marketing teams think that hiding the truth about this is preferable to just not mentioning it.

This illustrates the inertia of useful narratives: they turn into memes and become common points of reference, even when the specifics change.

August 24, 2026

Copyright (C) 2026 by Krisztian Flautner, All Rights Reserved.

In-person attendance is required for both lectures and labs.

Slides are released after lectures and labs.

Video recordings may be available; avoid for your own good.

Lab assignments need to be turned in before next lab. An unsubmitted lab dilutes your engagement score, and there are no drops.

Extensions are given for emergencies or approved conflicts with at least 24 hours’ notice.

Accommodations: send an SSD VISA letter.

LLMs: use them to research concepts. Do not use them to summarize, and do not use them to generate Verilog and text unless stated otherwise.

Chatham House Rule: share knowledge, do not share the identities of the people involved outside of class.

Collaboration for learning, debugging, and group projects is expected from you.

Handwritten notes are encouraged; the best students in this class are excellent note-takers.

Grade components

Homework, exams and regrades

Homework is submitted via GradeScope or GitHub. Up to 48 hours late carries a 25% penalty. Unsubmitted homework also dilutes your engagement score, and there are no drops.

Exams: one sheet, double-sided “cheat sheet” allowed. It must be handwritten, not computer generated.

Regrades: only for actual grading mistakes, not for extra partial credit. Individual requests may be throttled.

Policies may change.

• 

  Ron Dreslinski

  Professor (EECS 472)
  Quilting

• 

  Krisztian Flautner

  Professor (EECS 470)
  InternConnect @ CSE
  ex-VP of R&D @ ARM
  ex-CEO @ Banzai Cloud
  CEO @ Replicam

• 

  Mason Nelson

  GSI
  PhD student @ UofM
  Interned @ AMD

• 

  Artem Demenchuk

  IA
  Unemployed

• 

  Gabriela Ribeiro Znamensky

  GSI
  2^nd year grad student
  Theatre