Configuring a Hybrid Kubernetes Infrastructure Using Ubuntu 24.04.3 LTS Guide

Technology Stack

1 Node On Prem IMac - Running Ubuntu 24.04.3 LTS
2 Worker Nodes on Oracle Cloud Infrastructure Always Free VMs Running Ubuntu 24.04.3 LTS.
3.Windows 11 Laptop - Docker - To Build, Push Containers,Test Only Office,Conncet to n8n Portal
4.Fortiwifi60 - SSL VPN, So both On Prem Imac and OCI Vm are in same Tunnel, Security Policies

Project Overview

This project involved the successful architecture and deployment of a Hybrid Cloud Kubernetes Cluster. The infrastructure bridges an on-premises Control Plane (Master Node) with a remote Oracle Cloud Infrastructure (OCI) Worker Node. The primary objective was to host a containerized version of the Syphe IT Website, a containerized n8n web application to build AI agents Workflows and host oNlyoffice container application for word processing.

Infrastructure Specifications

Hardware & OS

  • Master Node: Local environment running Ubuntu 24.04.3 LTS.
  • Worker Node: OCI Compute Instance (VM.Standard.E4.Flex) running Ubuntu 24.04.3 LTS.

Security Gateway

Device: FortiWifi 60E Firewall.

Hybrid Networking

VPN Tunnel: Established a Site-to-Site VPN tunnel via the FortiWifi 60E to bridge the internal local network with the OCI Virtual Cloud Network (VCN).

Cloud Routing: Configured OCI Security Lists and Route Tables to allow private communication between nodes while exposing the application via a Public IP.

Kubernetes Setup

Implementation Phases

Cluster Initialization
  • Deployed kubeadm, kubelet, and kubectl on all nodes.
  • Initialized the Master Node and established the Control Plane.
  • Joined the OCI Worker Node to the local Master via the secure VPN link.
  • Deployed Calico CNI to manage the overlay network across the hybrid environment.

Deep Dive: How Images are Pulled & Run

To understand how Kubernetes pulls and runs an image, we look at the Kubelet. As the "primary node agent" on the OCI Worker Node, it acts as the bridge between the iMac's instructions and the physical cloud hardware.

1 The Instruction

The Control Plane (iMac) sends a "PodSpec" to the Kubelet. This includes the image path: ord.ocir.io/axtrsezpp4kw/syphewebsite:latest.

2 CRI Handshake

The Kubelet requests the Container Runtime (containerd) via the CRI to ensure the image is available locally.

3 Authentication

The Runtime uses the ImagePullSecret to authenticate with Oracle OCIR and downloads image layers.

4 Creating Containers

The Runtime creates a "Copy-on-Write" layer for storage, assigns networking via CNI, and starts the Nginx process.

5 Monitoring

The Kubelet performs continuous Health Checks. If the process crashes, the runtime restarts a fresh container immediately.

Chain of Command Summary:

iMac (Master): Sends the Contract (YAML) Oracle Node (Kubelet): Reads the contract Runtime (containerd): Pulls from OCIR Linux Kernel: Runs the isolated Pod process.

iMac Master ---SSL VPN --- OCI Security List ---- Ubuntu VM Port 10250.

Home