Memory Hierarchy Design and Characteristics

What is Memory Hierarchy?

When a Computer System is designed, a large number of different devices are used in it, like processors, different memory devices, motherboards, etc., and these parts are very expensive, which means we are not free to use them as many numbers of times as we want, meaning we are bound to use each particular part only a specific number of times in the design, to balance the overall cost.

Now, talking about the memory devices in a system, when a computer system is designed, a wide variety of memory devices are used, the most commonly used memory device that we all are familiar with is RAM, the Random Access Memory. But it is not the only memory that is used in our systems, we have registers, cache memory, flash memory, magnetic disks, magnetic tapes, etc. different kinds of memories as well.

So what is memory hierarchy?

The memory hierarchy is the memory organization of a particular system to balance its overall cost and performance.
As a system has several layers of memory devices, all having different performance rates and usage, they vary greatly in size and access time as compared to one another.

The memory Hierarchy provides a meaningful arrangement of these various memory types to maximize the performance of a system.

Let's understand each of them one by one

Memory Hierarchy Design

The above diagram shows a clear understanding of the memory hierarchy design, and you might wonder, why the shape is a pyramid?, it is to reflect the size (in bits) and access time of each of these memory layers, as the most important difference that distinguishes these layers is the size in which they are available inside a system and the access time of theirs concerning the CPU (Central Processing Unit).

Let's understand each of these levels one by one

Registers

These are very small memory devices, generally holding around bytes of data nearly 32-bit to 64-bit, and are of various types, for example - general-purpose registers, address registers, etc. The registers are directly associated with the CPU which means they are the fastest memory available in a system, which also means they are the costliest of all, Due to this they are used in very small sizes and possess the fastest access time.

Cache Memory

Cache memories are also called SRAM (Static Random Access Memory), they are measured in kilobytes or megabytes, and are used to store the segments of programs that are frequently accessed by the processor. Registers and Cache memories are embedded on the CPU itself, and hence are the fastest and are collectively referred to as internal memory.

Main Memory (RAM)

This is the most commonly used memory, also called DRAM (Dynamic Random Access Memory), the main memory directly communicates with the CPU and auxiliary memory devices through input and output. These are less expensive as compared to cache memory and are measured in Gigabytes.

All the above three memories ie, Registers, Cache Memory, and Main memory are volatile, meaning all the data inside them will be erased if the power is cut off.

Flash Memory

Memories like Pendrives and Solid State Drives come in this category and are generally measured in Gigabytes.

Magnetic Disks

Most commonly available memories, are slower as compared to flash memories, Memory storage like hard drives come in this category, and are measured in Gigabytes or Terabytes.

Flash Memory and Magnetic Disks are collectively referred to as Secondary Storage.

Magnetic Tapes

Magnetic tapes are the largest in size provide the slowest access time, and are generally measured in Petabytes, they are mainly used as backup storage and are not directly connected to the CPU.

Magnetic tapes are referred to as tertiary storage.

Cost

Now if we talk about the cost, then you may think, that in Registers we are storing only some bytes of data, and in Magnetic Tapes we are storing petabytes of data (nearly 10¹⁵ bytes of data) and hence Registers should cost less and the Magnetic tapes should cost more, but it is the opposite which is true, Registers are costliest among all of them, and Magnetic tapes, are the cheapest if we talk about the price to memory (in bit) ratio.

The reason for this cost variation is their access time, meaning if we try to access data present in registers, it is going to be the fastest, After comes the cache memory, then the main memory, then flash drives, then magnetic disks, and finally, the slowest accessing time will be in magnetic tapes.

Access Time

Let's have a look at the below table, to have a glimpse of access times for each of the memory levels.

Frequency

If we talk about the frequency ie, which memory is used most frequently by the CPU, then they are registers, as they are directly embedded onto the CPU, and we know that even to do the smallest of the tasks, the CPU accesses the registers, so they are the most used memory in any system. And the least used memory device is Magnetic tapes, as they are generally used for keeping the backups, and we know that on a regular basis, backups are not accessed, so they are the least used memory in a computer system.

Characteristics of Memory Hierarchy Design

Performance

Long ago, the designing of computer systems was not implemented using memory hierarchy, due to which the speed difference among different memory components and CPU registers keeps getting enhanced gradually and which resulted in lower performance of systems, so to improve that, memory hierarchy designs were introduced, which resulted in better performance of the computer systems.

Access Time

Access time is the time interval between the availability of the data and the subsequent read/write requests.

Access time increases if we travel from top to bottom in the memory hierarchy design.

Capacity

As we travel from top to bottom, the capacity increases, it simply means the volume of data the memory can store.

Cost Per Bit

The ratio of cost per bit increases as we go from bottom to top, meaning the registers are the costliest memory and secondary and tertiary storage are the cheapest.

Types of Memory Hierarchy in Operating System

The Memory Hierarchy is divided into two types.

Internal Memory

Internal memory is also referred to as primary memory, in a computer system, internal memory is memory that stores the data that needs to be accessed frequently and quickly, this type of memory is directly reachable by the system and other processes through input/output modules. It consists of

• Registers
• Cache Memories
• RAM
• ROM

External Memory

Also known as Secondary Memory, and are used to store huge amounts of data, They are non-volatile memories, meaning they will be storing the data even if the power gets off. They consist of

• Magnetic Disks
• Optical Disks
• Magnetic Tapes

Advantages of Memory Hierarchy

There are several advantages of using a memory hierarchy in computer systems, including:

• Faster Access: By having multiple levels of the memory hierarchy, computer systems can provide faster access to frequently accessed data. The fastest memory such as cache memory is used to store the most frequently used data, which can be accessed much faster than other types of memory.

• Cost-Effective: The use of a memory hierarchy enables the computer system to be cost-effective, as the cost of implementing high-speed memory such as cache memory is more expensive compared to other types of memory like main memory or secondary storage. With the use of a memory hierarchy, the most expensive memory can be used where it is needed the most, without needing to use it everywhere.

• Efficient use of resources: Memory hierarchy allows efficient use of resources by storing frequently accessed data in fast memory and less frequently accessed data in slower memory. This ensures that the computer system does not waste resources by using high-speed memory for data that is rarely accessed.

• Increases the processing speed: The use of a memory hierarchy allows computer systems to perform operations much faster. By using the fastest memory for frequently used data, the CPU can access the data quickly and complete the operations faster, increasing the processing speed of the system.

• Increased capacity: Memory hierarchy increases the overall capacity of the computer system by allowing the system to store large amounts of data in the secondary storage while keeping the most frequently accessed data in the high-speed memory. This allows the system to handle large amounts of data without slowing down its performance.

Overall, using a memory hierarchy in computer systems helps improve the system's overall performance and efficiency, making it faster, cost-effective, and efficient in its use of resources.

Disadvantages of Memory Hierarchy

The memory hierarchy design provides a solution to optimize the overall performance of a system while balancing the cost. However, it also has some disadvantages:

• Complexity: It adds complexity to the system as there are multiple layers of memory, and the system needs to manage them efficiently.

• Access time: Although the memory hierarchy design aims to reduce the access time to the data, it still adds some latency in the system. The data needs to move from one layer to another, which increases the access time.

• Cost: While memory hierarchy design aims to balance the overall cost, it still adds to the cost of the system. The higher the number of layers, the more costly the system becomes.

• Limited capacity: Each layer of the memory hierarchy has limited capacity, which means that the system cannot store an unlimited amount of data. This limitation can sometimes lead to performance issues.

• Management overhead: The system needs to manage the data movement between the different layers of memory, which adds overhead to the management process.

Despite these disadvantages, memory hierarchy design is still widely used as it provides an efficient way to manage memory and optimize system performance.