Main memory and performance

Main memory satisfies the demands of caches and serves as the I/O interface. And it is the destination of inputs well as the source for output. Performance measures of main memory emphasize both latency and bandwidth. (Memory bandwidth is the number of bytes read or written per unit time), The main memory affects the cache miss penalty, is the primary concern of the cache. While main memory bandwidth is the primary concern of I/O and multiprocessors.                          Although caches are interested in low latency memory, it is generally easier to improve memory bandwidth with new organizations than it is to reduce latency.With the popularity of second-level caches and their larger block sizes, main memory bandwidth becomes important to caches as well. In fact, cache designers increase block size to take advantage of the high memory bandwidth.                      First Technique for Higher Bandwidth: Wider Main Memory                            First-level caches are often organized with a physical width of one word because most CPU accesses are that size. Doubling or quadrupling the width of the cache  and the memory will  double or quadruple the memory bandwidth. There is cost in the wider connection between the CPU and memory, typically called a memory bus. CPUs will still access the cache a word at a time, so there now needs to be a multiplexor between the cache and the CPU—and that multiplexor may be on the critical timing path. Second-level caches can help since the multiplexing can be between first- and second-level caches, not on the critical path. Disadvantages:                          Since main memory is traditionally expandable by the customer, a drawback to wide memory is that the minimum increment is doubled or quadrupled when the width is doubled or quadrupled. In addition, memories with error correction have difficulties with writes to a portion of the protected block, such as a byte. The rest of the data must be read so that the new error correction code can be calculated and stored when the data are written. Second Technique for Higher Bandwidth: Simple Interleaved Memory                         Increasing width is one way to improve bandwidth, but another is to take advantage of the potential parallelism of having many chips in a memory system. Memory chips can be organized in banks to read or write multiple words at a time rather than a single word. In general, the purpose of interleaved memory is to try to take advantage of the potential memory bandwidth of all the chips in the system .In contrast, most memory systems activate only the chips containing the needed words.                        This mapping is referred to as the interleaving factor; interleaved memory normally means banks of memory that are word interleaved. This interleaving optimizes sequential memory accesses. A cache read miss is an ideal match to word-interleaved memory, as the words in a block are read sequentially.                          How many banks should be included? One metric, used in vector computers, is as follows:          Number of banks = Number of clock cycles to access word in bank Disadvantages: 1.      Shrinking number of chips is the main disadvantage of interleaved memory banks. 2.      Second disadvantage of memory banks is again the difficulty of main memory expansion. Either the memory system must support multiple generations of memory chips, or the memory controller changes the interleaving based on the size of physical memory, or both. Third Technique for Higher Bandwidth: Independent Memory Banks                       The original motivation for memory banks was higher memory bandwidth by interleaving sequential accesses. This hardware is not much more difficult since the banks can share address lines with a memory controller, enabling each bank to use the data portion of the memory bus.                        A generalization of interleaving is to allow multiple independent accesses, where multiple memory controllers allow banks to operate independently. Each bank needs separate address lines and possibly a separate data bus                         Independent of memory technology, higher bandwidth is available using memory banks, by making memory and its bus wider, or doing both.