AI Roundtable: Why I Built a Deliberation Tool Instead of Another Chatbot

Overview

Lately, I’ve stopped asking AI for “the best answer.”

I’ve been experimenting with getting it to argue with itself.

The shift happened after I spent a week asking different models for architectural recommendations on a new feature. Every answer sounded incredibly clean and confident. But when I actually built the code, I ran into security gaps and cost constraints the models had completely glossed over.

That’s why I built AI Roundtable: a small experiment where three different LLMs (GPT, Claude, and Gemini) debate the same question from different perspectives. I built this application following the structured workflow I outlined in my AI-assisted development workflow article: defining the technical spec first, breaking the work into small, testable increments, and using cross-model validation during development.

The goal isn’t to let AI make the decision. It’s to make the disagreement visible before I commit.

Designing for Disagreement

Instead of prompting a single model to output a list of pros and cons, the app sets up a structured debate. Each model is assigned a specific role that acts as a filter for its arguments.

By framing the models as advocates for opposing priorities, the interface surfaces the same tensions that show up in real engineering conversations. I use three core roles to model these tensions:

• The Optimist: Focused on momentum, speed of delivery, and learning through progress.
• The Mentor: Focused on practical tradeoffs, sequencing, and responsible delivery.
• The Bureaucrat: Focused on rules, documentation, governance, and risk control.

When you pass a technical question to this trio, you do not get a consensus. You get a clear view of what the clean answer leaves out.

A Practical Example: RAG vs. Static Context

To see how this works, consider a common architectural choice:

Should a public AI chatbot use RAG, static prompt context, or a traditional search interface?

This is the exact design choice I faced when building the chatbot for this site. In my autonomous LLM guardrails article, I decided to avoid the complexity of RAG and bundle static context directly in the prompt. But having different models debate the options highlights the exact tension points behind that choice:

1. The Optimist argues for RAG, pointing out that it allows the chatbot’s knowledge to scale dynamically without hitting context limits, enabling a roadmap of rich features.
2. The Mentor looks for the practical path: what is good enough to launch safely right now, and what should wait for a later phase. They might argue that a simple, static prompt is faster to ship and cheaper to maintain for a small knowledge base.
3. The Bureaucrat flags the compliance risks of dynamic retrieval, the threat of prompt injection through external documents, and the cost tracking challenges of variable context sizes.

The value here is not that any single model is correct. The value is that you can see exactly what each approach forces you to sacrifice.

From Concept to Code

Building a platform to support this kind of real-time debate meant solving a few specific engineering challenges. It required coordinating sequential, turn-by-turn streaming from three different LLM providers, injecting a shared conversation history without breaking character, and programmatically stitching multi-speaker text-to-speech audio in-memory.

Because this is a public application with open model endpoints, I also applied the security and cost guidelines I outlined in my autonomous LLM guardrails article: implementing server-side rate limits, token budgets, and bot protection.

For a detailed breakdown of the technical stack, the code behind the in-memory WAV audio synthesis, and the turn-by-turn Server-Sent Events (SSE) setup, read the full AI Roundtable project page.

What I Took Away

A single AI answer is useful for simple questions. But for important technical decisions, I want to see the disagreement before I decide. The debate does not tell me what to do. It shows me what I might be ignoring.

That is the real value of AI Roundtable. Not three models arguing for entertainment, but a structured way to surface the tradeoffs that a polished answer can hide.