In the world of Kubernetes networking, Calico stands as a robust solution for network policy enforcement and connectivity. While the recommended path to deploying Calico involves the elegant simplicity of the Tigera operator, understanding the underlying mechanisms of a manual installation can unlock deeper insights into its architecture and operational nuances. This article pulls back the curtain, revealing the intricate steps involved in a manifest-based Calico deployment and showcasing why the operator approach is often preferred for its automation and ease of management.
Why Explore Manual Calico Installation?
For many, the Tigera operator handles Calico deployment with just a few resource definitions, streamlining the entire process. However, this tutorial serves as an educational journey—a “red pill” experience, if you will—to comprehend the “magic” that happens behind the scenes. By manually configuring components like Calico, Typha, Goldmane, and Whisker, administrators gain invaluable knowledge about certificate management, environment variable tuning, and Kubernetes networking intricacies. This understanding empowers better troubleshooting and a deeper appreciation for automated systems.
Prerequisites for Your Manual Calico Journey
To replicate this advanced setup in your own environment, you’ll need:
* Docker
* K3d (for rapidly spinning up k3s clusters)
* Calico manifest files (version 3.31.0 is used here)
* OpenSSL (essential for generating and signing certificates)
* An active internet connection
Setting Up Your Kubernetes Cluster with k3d
Our journey begins with a fast k3s cluster powered by k3d and docker. This allows us to create a multi-node cluster configured specifically for a manifest-based Calico install, by disabling default networking components like Flannel and Traefik, and network policy enforcement:
The cluster creation command disables default network policy, Flannel, and Traefik to ensure a clean slate for Calico’s manifest-based deployment.
Installing Calico and Typha Manually
With your cluster ready, the next step is to deploy Calico and Typha directly using their respective manifests. Typha acts as a crucial middleware, optimizing communication between calico-node instances and the Kubernetes API server by caching responses.
You apply the Calico-Typha manifest, which sets up the core networking components.
It’s vital to confirm that calico-node pods are running successfully. Unlike an operator, which automatically manages these checks, manual deployments require explicit verification.
A manifest-based installation demands that administrators meticulously configure every aspect of Calico. This flexibility comes with challenges, such as correctly handling YAML formatting and ensuring compatibility with the specific Calico version. This underscores the operator’s value in simplifying these complexities.
Mastering Certificate Generation for Secure Communication
Security is paramount. In a manual Calico installation, securing communication between components is the administrator’s responsibility. This involves creating a Certificate Authority (CA), issuing, signing, and managing the rotation of all necessary certificates—a process fully automated by the Tigera operator.
1. Establishing Your Certificate Authority (CA)
To secure Calico components, a CA is required. We’ll create a self-signed internal CA to generate and sign certificates specific to our needs.
Commands are used to generate the CA’s private key and then a self-signed certificate, establishing our root of trust.
2. Generating Certificates for Calico Typha (Server)
Calico Typha, serving cached API responses to calico-node clients, requires its own server certificate. This certificate is crucial for establishing secure TLS connections.
You generate a certificate request (.csr) for the Typha server, specifying its Common Name (CN), and then sign it with your newly created CA.
3. Generating Certificates for Calico-Node (Typha Client)
The calico-node pods act as clients to the Typha server. Therefore, they need client certificates to authenticate themselves when communicating with Typha.
Similar to Typha, you generate a certificate request for the calico-node client and sign it with your CA.
4. Generating Certificates for Goldmane and Whisker
Goldmane and Whisker, components handling sensitive network flow logs, require certificates with specific roles and Subject Alternative Names (SANs). SANs are necessary for these certificates to be used by other workloads (like calico-node) to emit flow logs.
You create certificate requests for both Goldmane and Whisker, including necessary SANs for proper service identification within Kubernetes, and then sign them with your CA, associating specific extensions for their roles.
Importing Certificates into Your Cluster
Once all certificates are generated and signed, they must be imported into Kubernetes as ConfigMaps (for CA bundles) and Secrets (for key pairs). This is a best practice that facilitates easier updates and management.
Commands are executed to create ConfigMaps for the CA bundle and TLS secrets for Typha server, Typha client, Goldmane, and Whisker, making them accessible to your cluster’s workloads.
Tuning Typha to Utilize Certificates
With certificates imported, Typha needs to be reconfigured to use them. This involves modifying the calico-typha deployment to mount the certificate secrets as volumes and set specific environment variables that point to the certificate files and specify client Common Names for verification.
You patch the calico-typha deployment to add volumes for the CA bundle and Typha server key pair, and then set environment variables like TYPHA_CAFILE, TYPHA_SERVERCERTFILE, TYPHA_SERVERKEYFILE, and TYPHA_CLIENTCN to enable TLS.
Configuring Calico-Node for Secure Communication
Similarly, the calico-node daemonset must be updated. As a client, it needs access to its client certificate and the CA bundle to securely communicate with Typha. It also specifies FELIX_TYPHACN to match the Typha server’s certificate for an additional layer of verification.
You patch the calico-node daemonset, adding volumes for the CA bundle and Typha client key pair. Environment variables such as FELIX_TYPHACAFILE, FELIX_TYPHACERTFILE, FELIX_TYPHAKEYFILE, and FELIX_TYPHACN are configured for secure client-side TLS.
After these changes, verify that both calico-node and calico-typha deployments have successfully rolled out their updates.
Deploying Goldmane: The Flow Log Ingestion Engine
Goldmane is a critical component for ingesting network flow logs. Its deployment involves several steps:
1. Create a ServiceAccount: Provides an identity for Goldmane within the cluster.
2. Create a Service: Exposes Goldmane’s ingestion port (7443) for emitters like calico-node.
3. Create a ConfigMap: Provides Goldmane with its initial configuration file.
Tuning Goldmane’s Deployment for Certificates
Goldmane’s deployment must also be tuned to use the previously generated certificates. This includes setting environment variables for the server certificate path, key path, and CA certificate path, as well as mounting the corresponding secrets and config maps as volumes.
Goldmane’s deployment is configured with specific environment variables pointing to its server certificate, key, and the CA bundle, along with volume mounts to make these files accessible within the pod.
Finally, Goldmane is deployed using a manifest that incorporates all these configurations.
Deploying Whisker UI: Visualizing Network Flows
Whisker provides a user interface for viewing network flow logs and policies collected by Goldmane. Its deployment mirrors Goldmane’s initial steps:
1. Create a ServiceAccount: Assigns an identity to Whisker.
2. Create a Service: Exposes the Whisker UI port (8081). A ClusterIP service is recommended for security, as flow logs can contain sensitive data.
Tuning Whisker’s Deployment for Certificates and Goldmane Connectivity
Whisker requires configuration to use its certificates and to locate the Goldmane service. Environment variables are set for its TLS certificate and key paths, and critically, GOLDMANE_HOST is defined to point to the Goldmane service endpoint.
Whisker’s deployment is updated with environment variables for its TLS certificate and key paths, and the GOLDMANE_HOST is set to enable communication with Goldmane. Relevant secrets and config maps are mounted as volumes.
Whisker is then deployed using its manifest.
Verify the successful deployment of both Goldmane and Whisker before proceeding.
Accessing Whisker
To access the Whisker UI (if using a ClusterIP service), you’ll typically use kubectl port-forward.
You initiate a port-forward command to access the Whisker UI via localhost:8081.
At this point, you might see the Whisker UI but no flow logs. This indicates one final configuration step is needed.
Instructing Felix to Generate Flow Logs for Goldmane
Felix, the “brain” of Calico, is responsible for generating flow logs. We need to tell calico-node (where Felix runs) to enable flow log generation and send them to our deployed Goldmane server. This is done by setting two environment variables in the calico-node daemonset: FELIX_FLOWLOGSGOLDMANESERVER and FELIX_FLOWLOGSFLUSHINTERVAL.
You patch the calico-node daemonset to enable flow log generation by setting FELIX_FLOWLOGSGOLDMANESERVER to Goldmane’s service endpoint and FELIX_FLOWLOGSFLUSHINTERVAL for the desired flush interval.
After this update, calico-node pods will restart. However, flows might still not appear immediately in the Whisker UI due to a common Kubernetes networking puzzle.
The Infamous DNS Troubleshooting Step: “It’s Always DNS!”
If you observe warnings in your calico-node logs about failing to connect to the flow server due to “name resolver error: produced zero addresses,” it’s a classic DNS issue. This often occurs because calico-node pods, especially when running with hostNetwork: true, might send DNS queries to the host’s DNS server, which doesn’t know about Kubernetes internal service records.
The solution is to change the dnsPolicy of the calico-node daemonset to ClusterFirstWithHostNet. This ensures that DNS queries are routed through the cluster DNS first, allowing calico-node to correctly resolve goldmane.kube-system.svc.
You patch the calico-node daemonset to set dnsPolicy: ClusterFirstWithHostNet to resolve internal Kubernetes service names correctly.
With this final adjustment, you should soon observe network flow logs populating your Whisker UI!
Conclusion: The Power of Understanding, The Simplicity of Automation
Congratulations! You have successfully deployed Calico with its Typha, Goldmane, and Whisker components in a meticulously manual, manifest-based installation. This deep dive has illuminated the complex interplay of certificates, environment variables, and Kubernetes resources required for a functional Calico setup.
While this exercise is invaluable for gaining a profound understanding of Calico’s internals, it unequivocally demonstrates the immense value of the Tigera operator. The operator automates virtually every step detailed here—from certificate generation and rotation to component configuration and updates—allowing you to focus on application development and network policy design rather than the intricate mechanics of infrastructure deployment. Embracing the operator frees you from managing these low-level details, embodying true operational excellence in Kubernetes networking.