Over the past decades, the classical computer architecture has been based on the famous Von Neumann architecture, where the CPU is separate from system memory (RAM).
While this approach worked quite well with traditional computing problems, the data-intensive applications of our era, such as artificial intelligence, machine learning, big data analysis, and real-time graph processing, push the classical design to its maximum performance limit.
Each time the processor requires the execution of any computation, it needs to move huge amounts of data across a physical bus to and from the memory chip. This movement results in significant performance issues known as memory walls.
The article ‘Understanding Processing-in-Memory‘ explains a groundbreaking approach to computing, where this issue is solved by combining the computing logic with the memory module.
Processing-in-Memory, which can be abbreviated as PIM, is a new approach to designing computer hardware systems, where processing units are placed either directly inside or next to the memory module (DRAM or HBM).
Instead of fetching data through a bus just to perform trivial mathematical operations, the PIM approach allows one to conduct all the data-intensive computations in-place.
It dramatically cuts down on the physical distance that needs to be traveled by the data packets. As a result, the workload is performed with extreme efficiency due to reduced transit time from storage registers to processing cores.
The development of an efficient hardware system that can perform calculations within memory requires certain design principles:
Combining computation with storage is revolutionizing the approach of today’s data centers towards complex computation pipelines:
Classic computer systems waste a lot of time and resources waiting for data movement through system buses rather than actual computations. The use of Processing-In-Memory technology overcomes this physical barrier altogether. Computation on-site eliminates delays, thus allowing artificial intelligence models and big data stores to work at maximum speed.
Advanced models for generative intelligence involve computations with trillions of parameters all at once. The typical server architecture often suffers from bottlenecks in deep learning inference and training. PIM architectures help accelerate neural networks by performing parallel matrix multiplications right in memory chips.
Implementation of memory-centric computer architecture immediately provides tangible gains in high-performance computing environments:
As the growth of artificial intelligence and big data analytics reaches its exponential limit, there needs to be a change in hardware technology. Using Processing-in-Memory allows designers to break the memory wall and get rid of slow data transfer issues.
Ans: Hardware design that incorporates computation functions directly inside memory chips.
Ans: Transfer consumes too much time and energy.
Ans: It helps speed up parallel matrix computations right inside memory arrays.