Cache Performance and Organization

Introduction

There are two pieces to today's lesson. The first and more extensive is cache performance; I don't expect to lecture any further on this topic, so I want you to understand it well. The second is a very general intro to how caches are actually designed, with an emphasis on how they exploit spatial locality.

Remember that we are now looking at the memory hierarchy, so our frame of reference is no longer instructions to be executed. Rather, it's a stream of memory addresses coming from the processor. In LEGv8, these addresses can be PCs for the instruction fetch, or addresses for loads and stores in the data memory. It's the memory hierarchy's job to go to each address and retrieve or update the data that lives there. As we learned last time, the memory hierarchy means that different accesses can take different amounts of time, which makes analyzing performance more difficult than we thought. And that's where today's material starts.

I've assigned you a bunch of videos from Udacity. They're quite dry, but they have the advantage of being short and, at least in my opinion, clear. Remember that your book also covers these topics if you want a second opinion, and of course I'm available by email.

Here are the goals for today's lesson. If you've achieved these, you're ready to attempt the Moodle quiz, which is due at 2 PM on Monday, November 20.

• Cache performance
  1. Define the terms hit, miss, hit time, miss time, miss penalty, hit rate, miss rate, AMAT
  2. Relate the terms above to each other, and identify reasonable values for a typical cache.
  3. Explain the possible sequences of events for handling memory references in a multi-level cache hierarchy.
  4. Calculate AMAT for different cache structures and parameters.
• Cache design
  1. Define the term cache line (which is synonymous with cache block or cache entry), and explain how it relates to spatial locality.
  2. Explain how processor designers determine an appropriate block size for a cache.
  3. Explain, very broadly, how a cache determines if an access is a hit or a miss. This is where we'll pick up next week.

Cache Performance

Watch these videos, preferably in the order given, until you feel like you can meet the objectives above.

1. Basics of lookup, concept of hits and misses
2. Cache size in real processors and a review of basic concepts
3. Basic AMAT calculation
4. AMAT with multi-level caches
5. A nice realistic example [Part 1] [Part 2]

Once you've finished those, these mini-quizzes are great for building intuition. They're the kind of short-answer problem I might put on an exam.

• Problem and solution
• Problem and solution
• Problem and solution

Cache Design

This material is going to be more abstract for now. Next week, we'll dive into the specifics of cache mappings. If you don't understand these introductory videos on at least a conceptual level, next week is going to be rough.

1. Introduction to cache mappings; definition of cache line
2. Choosing an appropriate line size: problem and solution