Backend / Distributed Systems

workflow

workflow is a multi-tenant AI workflow orchestration backend built around uploaded documents. It handles ingestion, chunking, retrieval, workflow execution, async job retries, dead-letter handling, and audit events while keeping tenants isolated.

  • Go
  • PostgreSQL
  • Redis
  • MinIO
  • OpenTelemetry
10,045.73

runs/sec on the large benchmark path described in the README.

Multi-tenant

Tenant-scoped authentication and storage boundaries are part of the system shape.

Async-safe

Retries, workers, and dead-letter handling make failures observable instead of silent.

Why this is the standout project

This project feels closest to the kind of engineering work I want to keep growing into. It deals with the infrastructure concerns that show up in real platforms: tenant isolation, stateful workflows, async recovery, auditability, and the boundary between data systems and AI steps.

It also goes beyond a simple API demo. The system includes document processing, run history, async execution, and benchmarks, which makes it a stronger example of backend ownership.

System Diagram

workflow system diagram

Run Lifecycle

1

Ingest documents

Tenants upload documents that are chunked for later retrieval.

2

Retrieve context

A workflow step pulls the relevant chunks needed to answer a query or continue a run.

3

Execute flow logic

The workflow can branch, call a tool, pause for approval, or finish a run.

4

Recover failures

Async work is retried, bad jobs are isolated, and audit events keep the run visible.