GPU acceleration: Squeezing every last drop of compute

–

Through our Moonshot service, we can leverage our considerable expertise in CUDA programming to support organisations looking to optimise their GPU usage for a range of target applications and tasks. To get in touch with the Moonshot team email: moonshot@vividq.com.

–

GPUs are purpose-built for parallelism, making them exceptionally well-suited for computational-intensive tasks in fields such as machine learning, medical imaging, robotics, edge computing, autonomous vehicles, data analysis, simulations (such as computational fluid dynamics, aerodynamics and finite element analysis), and real-time graphics (including holography). 

In all of these applications performance bottlenecks can be a limiting factor, and there is a constant need to make more efficient utilisation of computational resources. As a result, it is essential that people building cutting-edge applications can squeeze as much out of GPU performance through acceleration. 

It was critical for us to fully harness the power of GPUs in our pioneering work on computer-generated holography (CGH), which is a computationally intensive task.In holography, GPU acceleration is needed for the following reasons:

• Complex data processing: In order to accurately represent 3D scenes, holography involves the manipulation of large datasets.
• Parallelizable: While computational requirements are expensive they are also highly parallelizable on GPUs. In addition, this allows for specialised algorithms to be used, which in turn give us opportunities to speed things up even further.
• Real-time rendering: For latency and motion sickness reasons, real-time holographic displays demand rapid refresh rates, and therefore the processing power in order to generate dynamic holograms for various applications like gaming or automotive Heads-Up Displays.
• Resolution and quality: High-resolution holographic displays demand significant computational power for fine details and achieving competitive field of view and eye-box sizes.
• Interactivity: GPU acceleration is crucial for interactive holographic applications, such as holographic gaming.

‍

Harnessing the full capability of GPUs presents the same significant challenges, whether you are using them for holography or any of the tasks we highlighted earlier. These include:

• Parallel computing: Leveraging the full potential of GPUs requires expertise in parallel computing in order to exploit the available compute resources. 
• Memory management: Efficient memory management is essential to avoid bottlenecks, making GPU programming more complex than traditional CPU programming.
• Optimising algorithms: Adapting algorithms for GPU processing can be tricky, as not all are naturally suited for parallel execution. Ultimately, algorithms need to be specialised in order to perform optimally.
• Hardware variability: Different GPUs have distinct architectures, which require tailored optimization for each, adding complexity to the process. In order to have scalable solutions (rather than only running optimally on a specific GPU), the various architectures must be taken into account.
• Debugging and profiling: Debugging GPU code and profiling performance is significantly more complex that CPU based concurrent code. This requires specific tools, knowledge, and experience to yield the best results.

‍

In our quest to push the boundaries of CGH we’ve been able to establish world-leading capabilities in CUDA programming. This has resulted in a number of achievements:

1. Significant speed gains: Our expertise allowed us to analyse, profile and optimise our CUDA application, leading to a substantial increase in processing speed. For example, we were able to speed up a key kernel in our most recent algorithm by a factor of 40.
2. Efficient memory usage: We optimised memory management, reducing GPU global memory accesses in memory bound parallel algorithms, resulting in significant speedups of the application.
3. Integration into graphics pipelines: Our expertise allowed us to integrate into existing graphics pipelines (e.g. game engines, render APIs) and profile as well as optimise the integration and interaction.
4. Wider GPU compatibility: We adapted our code to be compatible with a range of Nvidia GPUs, making our solution more versatile.
5. Real-time holography: Through expert CUDA programming, we pioneered real-time rendered, interactive and smooth holograms, which had until that point been impossible.

Without GPU acceleration, computer-generated holography would still be stuck in the lab, rather than making in-roads into consumer-facing devices. Being able to harness this amazing computing resource and to bend it to your will requires a deep understanding of the issues and expertise in both CUDA and OpenCL. 

We recognise that this capability has such wide ranging applications, so through our Moonshot service, we work with companies looking to make better use of GPUs for their applications on a fee-for-service basis. You can read more about Moonshot here or you can get in touch via moonshot@vividq.com.

‍