Kubernetes 1.37 DRA extended resources go GA, letting GPU pods keep legacy requests

Kubernetes v1.37 graduated DRA extended resources to general availability, and a hands-on guide by Oleksandr Kuryzhev on dev.to walks through what that changes for GPU fleets: nodes can be shifted from the legacy device plugin model to a Dynamic Resource Allocation (DRA) driver without touching a single workload manifest.

How the bridge works

Under the classic model, a device plugin on each node tells the kubelet how many units of a resource it has, and the scheduler sees only an opaque integer. It cannot distinguish an H100 from a T4, and it has no view of topology or partitions. DRA replaces that count with structured data: drivers publish ResourceSlice objects describing real devices and their attributes, and the scheduler allocates specific devices through ResourceClaim objects, matched by CEL selectors in a DeviceClass or claim.

The extended resources feature connects the two worlds. According to the guide, a DeviceClass can name an extended resource such as nvidia.com/gpu; when a pod requests that name in its limits, the scheduler creates a claim on the pod's behalf, allocates a device from the matching class, and records the outcome in the pod's status. Pod authors see no difference, while the platform team gets DRA's allocation model underneath. The author recommends confirming field names against your exact patch version with kubectl explain deviceclass.spec.

Where teams go wrong

The guide identifies four common failure modes:

• Running both stacks on the same GPUs. Installing a DRA driver while the device plugin DaemonSet is still running means both advertise the same physical devices, and the system can over-promise. Pods may fail at container start or fight over a device. The documented rule is that a device is managed by one mechanism at a time.
• Rewriting every manifest to ResourceClaims on day one. Conversion is only needed for workloads that require selection or sharing; the bridge exists precisely so most pods can stay unchanged.
• Treating it as a drop-in for any cluster. Mixed-version node pools, older kubelets, or drivers without the extended resource mapping produce pods stuck in Pending. Check the driver's release notes, not just the Kubernetes version.
• Forgetting the accounting side. Quotas, cost dashboards and alerts often read node allocatable nvidia.com/gpu, which may stop being the source of truth once a pool moves to DRA.

The recommended rollout

The advised approach is to migrate one labeled node pool at a time: cordon and drain the pool, exclude it from the device plugin DaemonSet via node affinity, install the vendor DRA driver, create the DeviceClass, then uncordon the nodes. NVIDIA's driver is distributed as the nvidia-dra-driver-gpu Helm chart.

The DeviceClass pairs a CEL selector matching the driver, gpu.nvidia.com in NVIDIA's case, with an extendedResourceName field that maps the legacy resource name onto DRA-backed allocation. Verification then takes three checks: kubectl get resourceslices should list slices per GPU node, kubectl get resourceclaims should show an allocated claim in the pod's namespace, and the pod's YAML should record the claim in its status. A pod stuck in Pending usually points to a missing extendedResourceName or a CEL expression that matches nothing.

Advanced patterns and limits

Once a pool runs on DRA, styles can be mixed. Most pods keep the legacy request, while the few needing a specific GPU model, a MIG profile or a shared device use an explicit ResourceClaimTemplate with a CEL selector on device attributes. The guide suggests inspecting the attributes a driver actually publishes with kubectl get resourceslices -o yaml before writing selectors. Multiple DeviceClasses can also express GPU tiers, moving policy from node labels and taints into declarative objects governed through namespaces and RBAC. The guide additionally notes that v1.37 ships Workload/PodGroup gang scheduling in beta, which lets distributed training pods schedule as a set, though its feature gates and interaction with DRA should be verified first.

The bridge has limits: it covers only the simple "N devices of a kind" case, and combining a legacy request with rich selection on the same device needs testing. For a small fleet of identical whole-device GPUs with no sharing needs, the device plugin remains a reasonable choice. On performance, the guide argues costs shift rather than grow: the scheduler runs a DynamicResources plugin that evaluates CEL selectors, and each pod generates a claim object, so large fleets with short-lived pods should watch API server object counts and etcd size, and compare metrics such as scheduler_pending_pods before and after migrating a pool.

Why it matters

ML platform teams often hold hundreds of training manifests requesting GPUs by count, while the device plugin model cannot express the per-model selection, MIG slices and shared devices those teams increasingly need. DRA extended resources offer a staged migration path: move the infrastructure node pool by node pool, avoid a risky fleet-wide manifest rewrite, then adopt rich allocation selectively where it pays off. That removes what has been the main practical obstacle to DRA adoption on GPU fleets, and makes features like topology-aware and partitioned GPU scheduling reachable without a big-bang migration.