If you are doing deep learning AI research and/or development with GPUs, big chance you will be using graphics card from NVIDIA to perform the deep learning tasks. A vantage point with GPU computing is related with the fact that the graphics card occupies the PCI / PCIe slot. From the frugality point of view, it may be a brilliant idea to scavenge unused graphics cards from the fading PC world and line them up on another unused desktop motherboard to create a somewhat powerful compute node for AI tasks. Maybe not.

With the increasing popularity of container-based deployment, a system architect may consider creating several containers with each running different AI tasks. This means that that the underlying GPU resources should then be shared among the containers. NVIDIA provides a utility called NVIDIA Docker or nvidia-docker2 that enables the containerization of a GPU-accelerated machine. As the name suggests, the utility targets Docker container type.

This utility, however, cannot be immediately usable for all NVIDIA graphics card models. Only graphics card having GPUs with architecture newer than Fermi can benefit from this feature. This post summarizes the list of NVIDIA desktop GPU models that serve as a better fit for building a deep learning AI system, especially through containerization support of AI tasks. The list is sourced from the Wikipedia entry of NVIDIA GPUs.

What about non-desktop GPUs such as those on workstation or laptop? Can we also use them for deep learning? NVIDIA provides the list for all products with CUDA support on this page: https://developer.nvidia.com/cuda-gpus.

CUDA-enabled vs Deep-Learning Ready GPUs

It is important to note that not all CUDA-enabled GPUs can perform deep learning tasks. NVIDIA introduced a terminology called CUDA compute capability that refers to the general specifications and available features of a CUDA-enabled GPU. The GPUs built with Fermi architecture has a maximum compute capability of 2.1, while the Kepler architecture has a minimum compute capability of 3.0.

The deep learning frameworks that rely on CUDA for GPU computing operate by invoking CUDA-specific GPU-accelerated deep learning methods to speed up the computation. These methods are provided in the cuDNN library. The library itself is compatible with CUDA-enabled GPUs with compute capability at least 3.0. It is suffice to say that only GPUs with Kepler architecture or newer are capable of accomplishing the deep learning tasks, or in other words, deep-learning ready.