--- abstract: Demonstrate a quick and dirty deployment with kubeadm. author: Xander Harris blogpost: true category: Quick and Dirty date: 2025-07-01 tags: kubeadm, q&d title: Quick and Dirty control plane with kubeadm --- For this demonstration we will be using {term}`kubeadm` to deploy and configure a control plane node which is the Lenovo ThinkCentre M910q that was acquired for this purpose. ## Q&D Control Plane Requirements The remainder of this guide assumes the presence of an {term}`ArchLinux` system on a locally-accessible network that accepts {term}`ssh` connections from a non-root user with {term}`sudo` access to the system in question, which should also have a functional installation of {term}`yay`. ## Prepare for {term}`kubeadm` In the interests of keeping things in relatively standard locations, all of the configuration files used by {term}`kubeadm` will be kept in {file}`/etc/kubeadm`, which we can create now. ```{code-block} shell :caption: Create the {term}`kubeadm` configuration directory ssh kcp01 yay -S containerd kubeadm kubelet sudo su - mkdir /etc/kubeadm cd /etc/kubeadm modprobe br_netfilter sysctl net.ipv4.ip_forward=1 ``` ### List of commands The remainder of this guide assumes that the commands listed above have been successfully executed, so far you should have: 1. ```{code-block} shell :caption: created an {term}`ssh` connection ssh kcp01 ``` 2. ```{code-block} shell :caption: installed {term}`kubelet` and {term}`kubeadm` with yay yay -S kubelet kubeadm ``` 3. ```{code-block} shell :caption: used {term}`sudo` to become the root user sudo su - ``` 4. ```{code-block} shell :caption: enabled the {term}`kubelet` service systemctl enable --now kubelet ``` ```{tip} It is safe to ignore any errors that the command above may throw, they will be resolved by {term}`kubeadm` later. ``` 5. ```{code-block} shell :caption: created the {term}`kubeadm` configuration directory mkdir -pv /etc/kubeadm ``` 6. ```{code-block} shell :caption: updated your working directory cd /etc/kubeadm ``` 7. ```{code-block} shell :caption: enable the br_netfilter kernel module modprobe br_netfilter ``` 8. ```{code-block} shell :caption: enable ip forwarding sysctl net.ipv4.ip_forward=1 ``` ### Create an initial configuration While it is possible to write a {term}`kubeadm` initial configuration file from scratch, you are much better off letting {term}`kubeadm` do that for you. ```{code-block} shell :caption: create default {term}`kubeadm` init configuration kubeadm config print init-defaults \ --component-configs KubeProxyConfiguration,KubeletConfiguration > init.yaml ``` This should leave you with something similar to the output below: ```{code-block} shell [root@kcp01:/etc/kubeadm]{0} # ls -l total 4 -rw-r--r-- 1 root root 1139 Jun 29 18:53 init.yaml ``` ```{code-block} yaml --- caption: default init.yml linenos: --- apiVersion: kubeadm.k8s.io/v1beta4 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 1.2.3.4 bindPort: 6443 nodeRegistration: criSocket: unix:///var/run/containerd/containerd.sock imagePullPolicy: IfNotPresent imagePullSerial: true name: node taints: null timeouts: controlPlaneComponentHealthCheck: 4m0s discovery: 5m0s etcdAPICall: 2m0s kubeletHealthCheck: 4m0s kubernetesAPICall: 1m0s tlsBootstrap: 5m0s upgradeManifests: 5m0s --- apiServer: {} apiVersion: kubeadm.k8s.io/v1beta4 caCertificateValidityPeriod: 87600h0m0s certificateValidityPeriod: 8760h0m0s certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controllerManager: {} dns: {} encryptionAlgorithm: RSA-2048 etcd: local: dataDir: /var/lib/etcd imageRepository: registry.k8s.io kind: ClusterConfiguration kubernetesVersion: 1.33.0 networking: dnsDomain: cluster.local serviceSubnet: 10.96.0.0/12 proxy: {} scheduler: {} --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 bindAddress: 0.0.0.0 bindAddressHardFail: false clientConnection: acceptContentTypes: "" burst: 0 contentType: "" kubeconfig: /var/lib/kube-proxy/kubeconfig.conf qps: 0 clusterCIDR: "" configSyncPeriod: 0s conntrack: maxPerCore: null min: null tcpBeLiberal: false tcpCloseWaitTimeout: null tcpEstablishedTimeout: null udpStreamTimeout: 0s udpTimeout: 0s detectLocal: bridgeInterface: "" interfaceNamePrefix: "" detectLocalMode: "" enableProfiling: false healthzBindAddress: "" hostnameOverride: "" iptables: localhostNodePorts: null masqueradeAll: false masqueradeBit: null minSyncPeriod: 0s syncPeriod: 0s ipvs: excludeCIDRs: null minSyncPeriod: 0s scheduler: "" strictARP: false syncPeriod: 0s tcpFinTimeout: 0s tcpTimeout: 0s udpTimeout: 0s kind: KubeProxyConfiguration logging: flushFrequency: 0 options: json: infoBufferSize: "0" text: infoBufferSize: "0" verbosity: 0 metricsBindAddress: "" mode: "" nftables: masqueradeAll: false masqueradeBit: null minSyncPeriod: 0s syncPeriod: 0s nodePortAddresses: null oomScoreAdj: null portRange: "" showHiddenMetricsForVersion: "" winkernel: enableDSR: false forwardHealthCheckVip: false networkName: "" rootHnsEndpointName: "" sourceVip: "" --- apiVersion: kubelet.config.k8s.io/v1beta1 authentication: anonymous: enabled: false webhook: cacheTTL: 0s enabled: true x509: clientCAFile: /etc/kubernetes/pki/ca.crt authorization: mode: Webhook webhook: cacheAuthorizedTTL: 0s cacheUnauthorizedTTL: 0s cgroupDriver: systemd clusterDNS: - 10.96.0.10 clusterDomain: cluster.local containerRuntimeEndpoint: "" cpuManagerReconcilePeriod: 0s crashLoopBackOff: {} evictionPressureTransitionPeriod: 0s fileCheckFrequency: 0s healthzBindAddress: 127.0.0.1 healthzPort: 10248 httpCheckFrequency: 0s imageMaximumGCAge: 0s imageMinimumGCAge: 0s kind: KubeletConfiguration logging: flushFrequency: 0 options: json: infoBufferSize: "0" text: infoBufferSize: "0" verbosity: 0 memorySwap: {} nodeStatusReportFrequency: 0s nodeStatusUpdateFrequency: 0s rotateCertificates: true runtimeRequestTimeout: 0s shutdownGracePeriod: 0s shutdownGracePeriodCriticalPods: 0s staticPodPath: /etc/kubernetes/manifests streamingConnectionIdleTimeout: 0s syncFrequency: 0s volumeStatsAggPeriod: 0s ``` Most of the default values are acceptable, but a number of them will need to be updated. ## Configure and initialize the control plane The lines that must be updated have been highlighted below, their values are described in the following table. |number|attribute|value|description| |---|---|---|--| |5|token|`$(kubeadm token create)`{l=sh}|a basic authentication token for bootstrapping| |12|advertiseAddress|`172.17.0.100`|an ipv4 address for the control plane node to listen on| |18|name|`kcp01`|an arbitrary dns-compatible name that describes the control plane| |34|clusterName|`the-hard-way`|an arbitrary dns-compatible name that describes the cluster| |47|podSubnet|`10.244.0.0/16`|the default cidr block for the {term}`flannel` networking plugin| |101|metricsBindAddress|`0.0.0.0:9090`|the address of a metrics receiver to send metrics to| |157|swapBehavior|`LimitedSwap`|enable experimental swap support on the kubelet| ### Configured {term}`kubeadm` {file}`init.yaml` ```{code-block} yaml :caption: updated init.yaml :linenos: :emphasize-lines: 5,12,18,34,47,101,156-157 apiVersion: kubeadm.k8s.io/v1beta4 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: 0fa83j.zskjs5g0h1lqz8mi ttl: 24h0m0s usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 172.17.0.100 bindPort: 6443 nodeRegistration: criSocket: unix:///var/run/containerd/containerd.sock imagePullPolicy: IfNotPresent imagePullSerial: true name: kcp01 taints: null timeouts: controlPlaneComponentHealthCheck: 4m0s discovery: 5m0s etcdAPICall: 2m0s kubeletHealthCheck: 4m0s kubernetesAPICall: 1m0s tlsBootstrap: 5m0s upgradeManifests: 5m0s --- apiServer: {} apiVersion: kubeadm.k8s.io/v1beta4 caCertificateValidityPeriod: 87600h0m0s certificateValidityPeriod: 8760h0m0s certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controllerManager: {} dns: {} encryptionAlgorithm: RSA-2048 etcd: local: dataDir: /var/lib/etcd imageRepository: registry.k8s.io kind: ClusterConfiguration kubernetesVersion: 1.33.0 networking: dnsDomain: cluster.local serviceSubnet: 10.96.0.0/12 podSubnet: 10.244.0.0/16 proxy: {} scheduler: {} --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 bindAddress: 0.0.0.0 bindAddressHardFail: false clientConnection: acceptContentTypes: "" burst: 0 contentType: "" kubeconfig: /var/lib/kube-proxy/kubeconfig.conf qps: 0 clusterCIDR: "" configSyncPeriod: 0s conntrack: maxPerCore: null min: null tcpBeLiberal: false tcpCloseWaitTimeout: null tcpEstablishedTimeout: null udpStreamTimeout: 0s udpTimeout: 0s detectLocal: bridgeInterface: "" interfaceNamePrefix: "" detectLocalMode: "" enableProfiling: false healthzBindAddress: "" hostnameOverride: "" iptables: localhostNodePorts: null masqueradeAll: false masqueradeBit: null minSyncPeriod: 0s syncPeriod: 0s ipvs: excludeCIDRs: null minSyncPeriod: 0s scheduler: "" strictARP: false syncPeriod: 0s tcpFinTimeout: 0s tcpTimeout: 0s udpTimeout: 0s kind: KubeProxyConfiguration logging: flushFrequency: 0 options: json: infoBufferSize: "0" text: infoBufferSize: "0" verbosity: 0 metricsBindAddress: "0.0.0.0:9090" mode: "" nftables: masqueradeAll: false masqueradeBit: null minSyncPeriod: 0s syncPeriod: 0s nodePortAddresses: null oomScoreAdj: null portRange: "" showHiddenMetricsForVersion: "" winkernel: enableDSR: false forwardHealthCheckVip: false networkName: "" rootHnsEndpointName: "" sourceVip: "" --- apiVersion: kubelet.config.k8s.io/v1beta1 authentication: anonymous: enabled: false webhook: cacheTTL: 0s enabled: true x509: clientCAFile: /etc/kubernetes/pki/ca.crt authorization: mode: Webhook webhook: cacheAuthorizedTTL: 0s cacheUnauthorizedTTL: 0s cgroupDriver: systemd clusterDNS: - 10.96.0.10 clusterDomain: cluster.local containerRuntimeEndpoint: "" cpuManagerReconcilePeriod: 0s crashLoopBackOff: {} evictionPressureTransitionPeriod: 0s fileCheckFrequency: 0s healthzBindAddress: 127.0.0.1 healthzPort: 10248 httpCheckFrequency: 0s imageMaximumGCAge: 0s imageMinimumGCAge: 0s kind: KubeletConfiguration logging: flushFrequency: 0 options: json: infoBufferSize: "0" text: infoBufferSize: "0" verbosity: 0 memorySwap: swapBehavior: LimitedSwap nodeStatusReportFrequency: 0s nodeStatusUpdateFrequency: 0s rotateCertificates: true runtimeRequestTimeout: 0s shutdownGracePeriod: 0s shutdownGracePeriodCriticalPods: 0s staticPodPath: /etc/kubernetes/manifests streamingConnectionIdleTimeout: 0s syncFrequency: 0s volumeStatsAggPeriod: 0s ``` The changes to lines `156-157` can be skipped if you disable any active swap[^k8s-swap] file systems on the control plane. ### Initialize the control plane Now that we've got an initial configuration that resembles the intended control plane configuration, we can go ahead and spin it up. ```{code-block} shell kubeadm init --config init.yaml ``` If things go well, your shell should produce something like the following output. ```{code-block} shell :caption: control plan init results [init] Using Kubernetes version: v1.33.0 [preflight] Running pre-flight checks [WARNING Swap]: swap is supported for cgroup v2 only. \ The kubelet must be properly configured to use swap. \ Please refer to https://shorturl.at/BKSgm \ or disable swap on the node [WARNING Service-Kubelet]: kubelet service is not enabled, \ please run 'systemctl enable kubelet.service' [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of \ your internet connection [preflight] You can also perform this action beforehand using \ 'kubeadm config images pull' [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names \ [kubernetes kubernetes.default kubernetes.default.svc \ kubernetes.default.svc.cluster.local kcp01] and IPs \ [10.96.0.1 172.16.0.100] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [localhost kcp01] and IPs [172.16.0.100 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [localhost kcp01] and IPs [172.16.0.100 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "super-admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests" [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 3.001790154s [control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s [control-plane-check] Checking kube-apiserver at https://172.16.0.100:6443/livez [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez [control-plane-check] kube-controller-manager is healthy after 1.506311953s [control-plane-check] kube-scheduler is healthy after 2.300444114s [control-plane-check] kube-apiserver is healthy after 4.00284659s [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node kcp01 as control-plane by adding the labels: [ node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers ] [mark-control-plane] Marking the node kcp01 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] [bootstrap-token] Using token: r51n8g.qj8psxn8wszh915u [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one \ of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes \ by running the following on each as root: kubeadm join 172.16.0.100:6443 --token r51n8g.qj8psxn8wszh915u \ --discovery-token-ca-cert-hash sha256:012345678 ``` ### Check the initialized control plane We can verify that this worked with the `kubectl` command, after we configure the new cluster's authentication. ```{code-block} shell :caption: add authentication mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config kubectl get nodes ``` The last command should produce output similar to this. ```{code-block} shell :caption: list of active nodes NAME STATUS ROLES AGE VERSION kcp01 NotReady control-plane 19m v1.33.2 ``` ```{note} The node status is not ready. This is because we still need to installed the {term}`flannel` network layer, fortunately this is simple. ``` ### Install {term}`flannel` networking layer ```{code-block} shell :caption: apply flannel network layer kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml ``` ```{code-block} shell :caption: flannel application result namespace/kube-flannel created serviceaccount/flannel created clusterrole.rbac.authorization.k8s.io/flannel created clusterrolebinding.rbac.authorization.k8s.io/flannel created configmap/kube-flannel-cfg created daemonset.apps/kube-flannel-ds created ``` Now, after a few moments you should be able to obtain a different result from the node list request. ### Verify the control-plane is ready ```{code-block} shell :caption: get nodes again kubectl get nodes ``` ```{code-block} shell :caption: and we have a working control plane NAME STATUS ROLES AGE VERSION kcp01 Ready control-plane 21m v1.33.2 ``` ```{sectionauthor} Xander Harris ``` [^k8s-swap]: The {term}`Kubernetes` reference manual has [more detail](https://kubernetes.io/docs/concepts/architecture/nodes/#swap-memory) about the necessity of these lines.