Confgraph: Have I reinvented the wheel while building an AI secretary for video calls

I periodically catch myself thinking: “what if someone has already done this?” — only after I’ve written another piece of code. With confgraph, my AI secretary for video conferences, I decided not to guess but to honestly find out: how much of a reinvention this is, and whether it’s worth continuing.

What is the project

Confgraph is a backend bot built on LiveKit that joins a video call as a subscribe-only participant (without its own microphone, often hidden in the participant list), listens to each speaker separately and:

• recognizes speech locally, using faster-whisper on CPU — without sending audio anywhere;
• classifies phrases by topics and builds a discussion graph from them — not a flat list of tags, but nodes (topics) with relations between them, using an LLM (OpenRouter);
• saves the transcript, embeddings and the graph in SQLite — raw audio does not go into the database;
• answers questions about the call content through a RAG endpoint;
• after the call, compiles a summary: by topics, off-topic items, decisions, deadlines, tasks for the next meeting.

On top — Meet, a fork of livekit-examples/meet with a “Secretary” panel: list of topics, a discussion tree with auto-updates, summary button. All of this runs in its own docker-compose (LiveKit + confgraph + meet), without external SaaS dependencies, except for one LLM call via API. You can try it live at meet.regionruza.ru.

A small note for those who like configuration details — of what can actually be configured:

ACTIVE_SPEAKER_MODE=replace   # phrase boundaries = active speaker + hangover 2s
CLASSIFY_TOPIC_CONFIDENCE=0.9 # threshold below which a short "ok" doesn't switch the topic
LIVEKIT_HIDDEN=true           # bot does not appear in the participant list
CHAT_LOG_ENABLED=false        # don't spam the meeting's general chat

Check for reinventing the wheel

Before writing another line, I surveyed the market — here’s what I found.

Commercial SaaS

Otter.ai, Fireflies, Fathom, tl;dv, Grain, Read.ai — this whole category works on the same principle: a guest-bot joins someone else’s call (Zoom, Google Meet, Teams) as a separate participant, audio is sent to the cloud, and in return you get a transcript and a summary. Prices range from a free tier with limitations (Fathom, for example, limits the number of AI summaries per month) to $30–40 per seat per month on higher plans. Some (Fireflies, MeetGeek) have “topic tracking” — but that’s tracking recurring topics as analytics, a flat list, not a graph with relations between topics.

Self-hosted and open-source alternatives

This part is more interesting because these are direct competitors in the “free and no subscription” spirit:

• Meetily — open source, MIT license, local Whisper/Parakeet, summaries via Ollama or any API of choice. In practice this is the closest free analogue to confgraph in terms of “STT + local summaries”. But it’s a desktop application for a single person on their own machine — not a service with an API for a room with multiple participants, not integrated into any video call as a product.
• Vexa — open, self-hostable API for bot participants joining Google Meet/Teams/Zoom, with real-time transcripts and webhooks. Closest to confgraph in architecture (self-hosted, API-first), but it’s a toolkit for developers aimed at joining other people’s calls, not a meeting platform of its own. There’s no topic graph or per-room RAG there.
• LiveKit Agents — the framework on which confgraph is built, provides STT/TTS/VAD primitives for voice agents, but it’s a building material, not a ready-made secretary. Using LiveKit here is not reinventing the wheel but using a ready-made brick for its intended purpose.

So it’s not entirely reinventing the wheel

If we narrow it down to the mere fact “local transcription + LLM summary without subscription” — that’s already solved, Meetily proves that, and I’m not the first.

But I didn’t find three things in that combination anywhere:

1. The secretary is built into its own platform for calls, not entering as a guest into someone else’s. You don’t need anyone’s permission to let the bot in, you don’t need to trust someone else’s audio infrastructure — it’s part of my own Meet.
2. A graph of topics with relations, not a flat list. Plus sticky logic: a short “yes/ok” does not switch the conversation to another topic without high classifier confidence. None of the found tools have that.
3. RAG queries tied to a specific room via its own API — not a global search across the account history, but specifically the context of one conversation.

Conclusion

Partly yes — the basic node “listen and transcribe” I didn’t reinvent; it existed before me. But the assembled whole — my own platform + topic graph + per-room RAG — was nowhere to be found in full for my use case. And the main practical advantage for me isn’t the uniqueness of the architecture, but that it’s mine: no subscription for a seat in a one-person team, full control over what happens to call data, and the ability to change anything I don’t like at any time, rather than writing to support.