Kgosi Letebele

Systems Architect & Site Reliability Engineer

I design, build, and secure self-hosted, distributed infrastructure. Instead of relying purely on managed cloud ecosystems, I specialize in frugal innovation—architecting zero-trust networks, sovereign communication nodes, and lightweight AIOps observability frameworks on physical bare-metal hardware.

🟢 Node: Storage & Mail Node Online📍 Region: Johannesburg, ZA

...

Interactive Cluster Topology

Hover over active nodes to inspect logical routing layers and service maps.

System idling. Hover over a node to establish telemetry tunnel.

...

Engineering Case Studies

Sanitized architectural blueprints of projects designed, deployed, and tested on my private cluster.

Zero-Trust SSO & Distributed AIOps Engine

Core Infra

SMEs lack the budgets for premium monitoring tools (e.g., Datadog) and rely on legacy local hardware. This project addresses the gap by deploying lightweight, automated Python agents to scrape logs/vitals on an on-premise Rocky Linux node, shipping sanitized payloads over a WireGuard tunnel to a localized Llama 3.1 model on a secondary compute node for autonomous anomaly diagnosis.

The Hardest Bug:Evaluating 1,000+ token logs triggered 60s client timeouts in the Python urllib transport layer due to mobile CPU latency. Resolved by decoupling the I/O, shrinking log sample windows, and raising the keep-alive timeout threshold to 300s.

Technology Stack:Rocky Linux, Docker, LLDAP, Authelia SSO, WireGuard, Ollama (Llama 3.1), Nginx.

Kgosiview — Telemetry Harvester & Currency Backtesting Engine

Database & Storage

Designed to execute high-density database stress queries and ingest real-time currency statistics. Features a highly secured database architecture implementing the Principle of Least Privilege for autonomous agents.

Migration Retrospective (Lessons Learned):All historical telemetry data was corrupted during a hard physical node migration due to improper storage sync states. This critical failure directly birthed the architecture of Chapter 9 in my operations runbook—implementing GPG-encrypted automated backup cron jobs with standard off-site replication pipelines.

Technology Stack:MariaDB, mysqlslap, LVM storage pools, Bash automation, GPG encryption.

Sovereign Enterprise Mail & Decoupled Storage Cluster

Collaborative Apps

Bypassed residential ISP port blocks and CGNAT limitations to deploy a fully private email and collaborative cloud ecosystem. Deployed Nextcloud for files on a low-resource Vault node, while offloading resource-heavy document processing (ONLYOFFICE) to a secondary high-RAM node over private Tailscale subdomains.

The Hardest Bug:A configuration directory typo in the Dovecot user-attributes database query generated double slashes in folder paths, causing critical write failures under strict SELinux labels. Resolved by aligning database attributes and enforcing targeted SELinux write policies.

Technology Stack:Postfix, Dovecot, Mailgun relay, Nextcloud, ONLYOFFICE, Caddy, SELinux.

Best-Spot! & Gauteng Route-Mapping Systems

Observability & GIS

Two complementary data systems: "Best-Spot!" bypasses fake RSSI signal bars by tracking actual network latency using an Isolation Forest anomaly detection algorithm, while the Gauteng Map tracks spatial travel metrics to align student internship locations with cost-efficient logistics routes.

System Design:Integrated Streamlit UI elements with localized caching to prevent page redraw clipping, displaying interactive Plotly maps and inline SVG sparklines computed from raw network data frames.

Technology Stack:Streamlit, Pandas, Scikit-Learn (ML), PostGIS, TomTom API, Plotly.