Overview of creating an instance with attached GPUs

Compute Engine provides graphics processing units (GPUs) that you can add to your virtual machines (VMs). You can use these GPUs to accelerate specific workloads on your VMs such as machine learning and data processing.

This document provides an overview of the steps required for creating a VM with attached GPUs.

For more information about GPUs on Compute Engine, see About GPUs.

Select the GPU model

For a list of GPU models that are available, see GPU platforms. Also make a note of the machine type that is supported for the selected GPU model.

For each model, it might also be helpful to review the following:

Limitations

In addition to the restrictions for all VMs with GPUs, each machine series with attached GPUs has the following limitations:

A4 VMs

  • You can only request capacity by using the supported provisioning options for an A4 machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A4 machine type.
  • You can only use an A4 machine type in certain regions and zones.
  • You can't use Persistent Disk (regional or zonal) on a VM that use A4 machine type.
  • The A4 machine type is only available on the Emerald Rapids CPU platform.
  • You can't change the machine type of an A4 machine type VM. If you need to change the machine type, you must create a new VM.
  • You can't change the machine type of a VM to an A4 machine type. If you need a VM that uses an A4 machine type, you must create a new VM.
  • A4 machine types don't support sole-tenancy.
  • You can't run Windows operating systems on an A4 machine type.

A3 Ultra VMs

  • You can only request capacity by using the supported provisioning options for an A3 Ultra machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A3 Ultra machine type.
  • You can only use an A3 Ultra machine type in certain regions and zones.
  • You can't use Persistent Disk (regional or zonal) on a VM that use A3 Ultra machine type.
  • The A3 Ultra machine type is only available on the Emerald Rapids CPU platform.
  • You can't change the machine type of an A3 Ultra machine type VM. If you need to change the machine type, you must create a new VM.
  • You can't change the machine type of a VM to an A3 Ultra machine type. If you need a VM that uses an A3 Ultra machine type, you must create a new VM.
  • A3 Ultra machine types don't support sole-tenancy.
  • You can't run Windows operating systems on an A3 Ultra machine type.

A3 Mega VMs

  • You can only request capacity by using the supported provisioning options for an A3 Mega machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A3 Mega machine type.
  • You can only use an A3 Mega machine type in certain regions and zones.
  • You can't use regional Persistent Disk on a VM that use A3 Mega machine type.
  • The A3 Mega machine type is only available on the Sapphire Rapids CPU platform.
  • You can't change the machine type of an A3 Mega machine type VM. If you need to change the machine type, you must create a new VM.
  • You can't change the machine type of a VM to an A3 Mega machine type. If you need a VM that uses an A3 Mega machine type, you must create a new VM.
  • A3 Mega machine types don't support sole-tenancy.
  • You can't run Windows operating systems on an A3 Mega machine type.

A3 High VMs

  • You can only request capacity by using the supported provisioning options for an A3 High machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A3 High machine type.
  • You can only use an A3 High machine type in certain regions and zones.
  • You can't use regional Persistent Disk on a VM that uses A3 High machine type.
  • The A3 High machine type is only available on the Sapphire Rapids CPU platform.
  • You can't change the machine type of an A3 High machine type VM. If you need to change the machine type, you must create a new VM.
  • You can't change the machine type of a VM to an A3 High machine type. If you need a VM that uses an A3 High machine type, you must create a new VM.
  • A3 High machine types don't support sole-tenancy.
  • You can't run Windows operating systems on an A3 High machine type.
  • For a3-highgpu-1g, a3-highgpu-2g, anda3-highgpu-4g machine types, the following limitations apply:
    • For these machine types, you must either use Spot VMs or a feature that uses the Dynamic Workload Scheduler (DWS) such as resize requests in a MIG. For detailed instructions on either of these options, review the following:
    • You can't use Hyperdisk Balanced with these machine types.
    • You can't create reservations.
    If you try to create a VM by using standard provisioning or try to create a reservation for these machine types, you'll receive an error message.

A3 Edge VMs

  • You can only request capacity by using the supported provisioning options for an A3 Edge machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A3 Edge machine type.
  • You can only use an A3 Edge machine type in certain regions and zones.
  • You can't use regional Persistent Disk on a VM that use A3 Edge machine type.
  • The A3 Edge machine type is only available on the Sapphire Rapids CPU platform.
  • You can't change the machine type of an A3 Edge machine type VM. If you need to change the machine type, you must create a new VM.
  • You can't change the machine type of a VM to an A3 Edge machine type. If you need a VM that uses an A3 Edge machine type, you must create a new VM.
  • A3 Edge machine types don't support sole-tenancy.
  • You can't run Windows operating systems on an A3 Edge machine type.

A2 Standard VMs

  • You can only request capacity by using the supported provisioning options for an A2 Standard machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A2 Standard machine type.
  • You can only use an A2 Standard machine type in certain regions and zones.
  • The A2 Standard machine type is only available on the Cascade Lake platform.
  • If your VM uses an A2 Standard machine type, you can only switch from one A2 Standard machine type to another A2 Standard machine type. You can't change to any other machine type. For more information, see Modify accelerator-optmized VMs.
  • You can't use the Windows operating system with A2 Standard machine types. When using Windows operating systems, choose a different A2 Standard machine type.
  • You can't do a quick format of the attached Local SSDs on Windows VMs that use A2 Standard machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.
  • A2 Standard machine types don't support sole-tenancy.

A2 Ultra VMs

  • You can only request capacity by using the supported provisioning options for an A2 Ultra machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A2 Ultra machine type.
  • You can only use an A2 Ultra machine type in certain regions and zones.
  • The A2 Ultra machine type is only available on the Cascade Lake platform.
  • If your VM uses an A2 Ultra machine type, you can't change the machine type. If you need to use a different A2 Ultra machine type, or any other machine type, you must create a new VM.
  • You can't change any other machine type to an A2 Ultra machine type. If you need a VM that uses an A2 Ultra machine type, you must create a new VM.
  • You can't do a quick format of the attached Local SSDs on Windows VMs that use A2 Ultra machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.

G2 VMs

  • You can only request capacity by using the supported provisioning options for an G2 machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an G2 machine type.
  • You can only use an G2 machine type in certain regions and zones.
  • The G2 machine type is only available on the Cascade Lake platform.
  • Standard Persistent Disk (pd-standard) is not supported on VMs that use G2 standard machine types. For supported disk types, see Supported disk types for G2.
  • You can't create Multi-Instance GPUs on G2 machine types.
  • If you need to change the machine type of a G2 VM, review Modify accelerator-optmized VMs.
  • You can't use Deep Learning VM Images as boot disks for your VMs that use G2 machine types.
  • The current default driver for Container-Optimized OS doesn't support L4 GPUs running on G2 machine types. Container-Optimized OS also only support a select set of drivers. If you want to use Container-Optimized OS on G2 machine types, review the following notes:
    • Use a Container-Optimized OS version that supports the minimum recommended NVIDIA driver version 525.60.13 or later. For more information, review the Container-Optimized OS release notes.
    • When you install the driver, specify the latest available version that works for the L4 GPUs. For example, sudo cos-extensions install gpu -- -version=525.60.13.
  • You must use the Google Cloud CLI or REST to create G2 VMs for the following scenarios:
    • You want to specify custom memory values.
    • You want to customize the number of visible CPU cores.

N1+GPU VMs

To learn about the limitations for N1 instances with GPUs, see features for the N1 machine series and GPUs for the N1 machine series.

Choose an operating system

If you are using GPUs for machine learning, you can use a Deep Learning VM Images for your VM. Each Deep Learning VM Images has a GPU driver pre-installed and include packages, such as TensorFlow and PyTorch. You can also use a Deep Learning VM Images for general GPU workloads. For information about the images available and the packages installed on the images, see Choosing an image. You can also use any public image or custom image, but some images might require a unique driver or install process that is not covered in this document.

You must identify which drivers are appropriate for your OS image. For steps to install drivers, see installing GPU drivers.

Check GPU quota

To protect Compute Engine systems and users, new projects have a global GPU quota, which limits the total number of GPUs you can create in any supported zone. To review GPU quota, see GPU quota.

If you need additional GPU quota, request a quota increase. When you request GPU quota, you must request quota for the GPU types that you want to create in each region and an additional global quota for the total number of GPUs of all types in all zones.

If your project has an established billing history, it will receive quota automatically after you submit the request.

GPU VMs and preemptible allocation quotas

Resources for VMs that use the default standard provisioning model (standard VMs) typically cannot use preemptible allocation quotas, which are intended for temporary workloads and usually more available. If your project does not have preemptible quota, and you have never requested preemptible quota, all VMs in that project consume standard allocation quotas.

However, once you request preemptible allocation quota, then the standard VMs that meet all the following criteria can consume only preemptible allocation quota.

By consuming preemptible allocation quota for such workloads, you gain both the benefits of uninterrupted run time from standard VMs and improved obtainability from preemptible allocation quota.

Regardless of the quota used, standard VMs don't qualify for Spot VMs pricing and are not subject to preemption.

For more information, see Preemptible quotas.

Create a VM that has attached GPUs

To create a VM that has attached GPUs, complete the following steps:

  1. Create the VM. The method used to create a VM depends on the GPU model selected.

  2. For the VM to use the GPU, you need to install the GPU driver on your VM. If you enabled an NVIDIA RTX virtual workstation (formerly known as NVIDIA GRID), install a driver for virtual workstation.

What's next?