Manage HugePages
Kubernetes v1.18
stable
- The version name is vX where X is an integer.
- Stable versions of features will appear in released software for many subsequent versions.
Kubernetes supports the allocation and consumption of pre-allocated huge pages by applications in a Pod as a GA feature. This page describes how users can consume huge pages and the current limitations.
Before you begin
- Kubernetes nodes must pre-allocate huge pages in order for the node to report its huge page capacity. A node can pre-allocate huge pages for multiple sizes.
The nodes will automatically discover and report all huge page resources as schedulable resources.
API
Huge pages can be consumed via container level resource requirements using the
resource name hugepages-<size>
, where <size>
is the most compact binary
notation using integer values supported on a particular node. For example, if a
node supports 2048KiB and 1048576KiB page sizes, it will expose a schedulable
resources hugepages-2Mi
and hugepages-1Gi
. Unlike CPU or memory, huge pages
do not support overcommit. Note that when requesting hugepage resources, either
memory or CPU resources must be requested as well.
A pod may consume multiple huge page sizes in a single pod spec. In this case it
must use medium: HugePages-<hugepagesize>
notation for all volume mounts.
apiVersion: v1
kind: Pod
metadata:
name: huge-pages-example
spec:
containers:
- name: example
image: fedora:latest
command:
- sleep
- inf
volumeMounts:
- mountPath: /hugepages-2Mi
name: hugepage-2mi
- mountPath: /hugepages-1Gi
name: hugepage-1gi
resources:
limits:
hugepages-2Mi: 100Mi
hugepages-1Gi: 2Gi
memory: 100Mi
requests:
memory: 100Mi
volumes:
- name: hugepage-2mi
emptyDir:
medium: HugePages-2Mi
- name: hugepage-1gi
emptyDir:
medium: HugePages-1Gi
A pod may use medium: HugePages
only if it requests huge pages of one size.
apiVersion: v1
kind: Pod
metadata:
name: huge-pages-example
spec:
containers:
- name: example
image: fedora:latest
command:
- sleep
- inf
volumeMounts:
- mountPath: /hugepages
name: hugepage
resources:
limits:
hugepages-2Mi: 100Mi
memory: 100Mi
requests:
memory: 100Mi
volumes:
- name: hugepage
emptyDir:
medium: HugePages
- Huge page requests must equal the limits. This is the default if limits are specified, but requests are not.
- Huge pages are isolated at a container scope, so each container has own limit on their cgroup sandbox as requested in a container spec.
- EmptyDir volumes backed by huge pages may not consume more huge page memory than the pod request.
- Applications that consume huge pages via
shmget()
withSHM_HUGETLB
must run with a supplemental group that matchesproc/sys/vm/hugetlb_shm_group
. - Huge page usage in a namespace is controllable via ResourceQuota similar
to other compute resources like
cpu
ormemory
using thehugepages-<size>
token. - Support of multiple sizes huge pages is feature gated. It can be
enabled with the
HugePageStorageMediumSize
feature gate on the kubeletAn agent that runs on each node in the cluster. It makes sure that containers are running in a pod. and kube-apiserverControl plane component that serves the Kubernetes API. (--feature-gates=HugePageStorageMediumSize=true
).
Future
- NUMA locality guarantees as a feature of quality of service.
- LimitRange support.
Feedback
Was this page helpful?
Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on Stack Overflow. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.