Reddit Review #1: GPU Hype
I realized there has been a trend where content creators upload Reddit post review videos on TikTok. In the video, they often take screenshots of some interesting, funny, or controversial Reddit posts, summarize the content, and then react or comment on these posts. This is similar to how YouTubers react or review products, events, and various topics. Then, an idea popped into my mind: Create a Medium blogpost to react to or review Reddit posts. A few days ago, I came across this Reddit post about next-generation GPU.
I remember seven years ago I overheard a conversation among a group of computer geeks in a student org meeting, sharing how they built their gaming PC with GPU. They talked about how GPU accelerates everything, graphics rendering, etc, which I had no idea what they were. Until my junior year of college, I started to learn about concepts of parallel programming and had opportunities to work on research projects that involved heavy numerical processing. Gratefully, we have a campus supercomputing cluster, which provides a high-performance computing (HPC) processing power that substantially boosts the speed of our numerical computation jobs. We utilized Slurm to submit and schedule those jobs to the cluster, and Slurm assigns tasks to available compute nodes in the HPC system.
Fast-forward to now, GPU processing and architectures have become the key subjects of interest, driving increased investment across various industries. Working in the semiconductor industry, my news feed is swamped with updates on major chip manufacturers such as AMD, NVIDIA, Intel, and TSMC. Although GPUs (Graphics Processing Units) have been in the computing world for quite some time, the GPU hype erupts due to the rise of large-scale AI & ML, which requires significant computational power, leading to increased interest in GPU Programming.
Speaking of GPU programming, this is a good book (Hands-On GPU Programming with Python and CUDA by Dr Brian Tuomanen) to begin with. I am going through this book by chapter and documenting the key points in this forked repository. It took me a while to set up the local GPU programming environment as you have to ensure that you installed the suitable NVIDIA CUDA Compiler Driver NVCC that is compatible with the PyCUDA version and added the right PATH on Windows. Then, in the rest of the chapters, you will learn how to use PyCUDA, Scikit-CUDA, etc.
Throughout the book, I realized running codes on a GPU is not always faster than running on a CPU. It depends on what kind of tasks you are going to perform, plus there is an overhead when transferring data between CPU and GPU and launching kernels on GPU. Before thinking about rewriting the code to run on GPU, we should profile and benchmark the code implementations, to determine whether a CPU or GPU will perform better for your specific use case. In the first chapter of the book, the author shared a famous quote from Donald Knuth, “Premature optimization is the root of all evil.” What he tried to emphasize is to always start developing a basic, functional version of code on the CPU, and then only optimize for performance when there is a need or significant bottleneck. Start Simple, Don’t Dive Deep.
While wrapping up this post, YouTube somehow recommended videos relevant to COMPUTEX 2024. Little did I know that COMPUTEX 2024, held in Taipei, Taiwan, is one of the world’s largest technology trade shows. This year, they showcased AMD’s Ryzen 9000 Series (the world’s fastest consumer PC Processor), NVIDIA’s RAPIDS cuDF’s GPU acceleration to Pandas on Google Colab, Intel’s Lunar Lake AI NPU Chip, and a list of exciting technology advancements. Despite not understanding some of the technical details, I am in awe of their efforts to innovate and their ambition to lead us (consumers) into a more powerful technological era. Through the YouTube videos and the Reddit post, I could sense the fierce competition in the chip industry, with companies demonstrating their R&D capabilities to push the boundaries of performance and efficiency.
Looking forward to how these competitive developments in the chip processor industry will evolve over the next few decades
