DEV Community

Martin Schenk
Martin Schenk

Posted on

Building an AI Chat Terminal That Routes Private Data to Local AI

#ai #privacy #opensource #terminal

I was tired of accidentally pasting API keys into ChatGPT, so I built a terminal that intelligently routes conversations.

The Problem

Every time you chat with ChatGPT, Claude, or any cloud AI:

  • Everything goes to their servers - passwords, API keys, emails, everything
  • It's all logged - no way to delete it
  • You can't verify what they do with your data

I kept cringing every time I accidentally pasted something sensitive.

The Solution

AI Chat Terminal - keyword detection that runs BEFORE anything touches the network:

You: "save my password SecretPass123" → Routes to LOCAL Qwen AI ✅ → Encrypted SQLite (AES-256) ✅ → NEVER sent to cloud ✅ You: "What's the capital of Spain?" → Routes to OpenAI ✅ You: "best food there?" → OpenAI understands context (Spain) ✅
Enter fullscreen mode Exit fullscreen mode

How It Works

1. Keyword Detection (<1ms)

Pattern matching for save, show, delete and 30+ synonyms:

if matches_keywords(input, ['save', 'note', 'record']): route_to_local() else: route_to_cloud()
Enter fullscreen mode Exit fullscreen mode

Works in English, German, Spanish:

  • "save my email" → Local
  • "guarda mi contraseña" → Local
  • "speichere meine Telefonnummer" → Local

2. Local AI Processing

Uses Qwen 2.5 Coder (7B) via Ollama:

User: "save my email test@example.com" → Qwen generates SQL: INSERT INTO mydata (content, meta) VALUES ('test@example.com', 'email') → Encrypted with SQLCipher (AES-256)
Enter fullscreen mode Exit fullscreen mode

3. Cloud for Everything Else

OpenAI GPT-4o with full context:

You: "capital of France?" AI: "Paris" You: "best food there?" AI: "Croissants..." (knows "there" = Paris)
Enter fullscreen mode Exit fullscreen mode

Tech Stack

  • Python 3.9+
  • Ollama - Local AI runtime
  • Qwen 2.5 Coder (7B) - SQL generation
  • SQLCipher - AES-256 encryption
  • OpenAI API - Cloud queries
  • ~5GB disk space for model

Installation (macOS)

One-line install:

curl -fsSL https://raw.githubusercontent.com/martinschenk/ai-chat-terminal/main/install.sh | zsh
Enter fullscreen mode Exit fullscreen mode

This automatically:

  1. Installs Ollama if needed
  2. Pulls Qwen 2.5 Coder model
  3. Sets up encrypted database
  4. Adds shell integration

Takes ~5 minutes (mostly downloading the 4.5GB model).

Usage Examples

Save Private Data 

$ chat 👤 You ▶ save my email test@example.com 🤖 AI 🗄️ Stored 🔒 👤 You ▶ save password SecretPass123 🤖 AI 🗄️ Stored 🔒 👤 You ▶ guarda mi dirección Calle Mayor 1 🤖 AI 🗄️ Guardado 🔒
Enter fullscreen mode Exit fullscreen mode

Retrieve Anywhere 

👤 You ▶ show my email 🤖 AI 🗄️🔍 test@example.com 👤 You ▶ list all 🤖 AI 🗄️🔍 Found 3 items: 1. test@example.com (email) 2. SecretPass123 (password) 3. Calle Mayor 1 (dirección)
Enter fullscreen mode Exit fullscreen mode

General Questions (Cloud) 

👤 You ▶ capital of Spain? 🤖 AI Madrid. 👤 You ▶ best food? 🤖 AI Tapas. (remembers Spain!)
Enter fullscreen mode Exit fullscreen mode

Privacy Architecture 

┌─────────────────────────────────────────────┐ │ User Input │ └────────────┬────────────────────────────────┘ ↓ ┌─────────────────────┐ │ Keyword Detection │ ← Fast (<1ms) │ save/show/delete │ └─────────┬───────────┘ ↓ ┌──────────────┐ │ Detected? │ └──┬────────┬──┘ │ │ YES │ │ NO ↓ ↓ ┌──────────────┐ ┌──────────────┐ │ Qwen 2.5 │ │ OpenAI │ │ Coder (7B) │ │ GPT-4o │ │ SQL Direct │ │ (Cloud) │ └─────┬────────┘ └──────┬───────┘ ↓ ↓ ┌─────────────┐ ┌──────────────┐ │ Encrypted │ │ Response │ │ SQLite DB │ │ with Context │ │ (AES-256) │ │ │ └─────────────┘ └──────────────┘
Enter fullscreen mode Exit fullscreen mode

Key Features

🔒 Privacy First

  • Keyword detection before any network call
  • Local Qwen AI for sensitive data
  • AES-256 encrypted SQLite
  • Zero cloud exposure for private data

🧠 Smart Context

  • OpenAI maintains conversation history
  • Understands references ("there", "it", "that")
  • Natural follow-up questions

🌍 Multilingual

  • English, German, Spanish
  • 30+ action verbs per language
  • Flexible phrasing

⚡ Fast

  • Keyword matching: <1ms
  • Local operations: <1s
  • OpenAI queries: 5-7s
  • Always shows 🗄️ icon for local DB operations

System Requirements

  • macOS 12.0+ (Monterey or later)
  • Zsh shell
  • Python 3.9+
  • ~5GB disk space (for Qwen model)
  • 8GB RAM minimum (16GB recommended)
  • OpenAI API key

Why Qwen 2.5 Coder?

I tested several local models and Qwen 2.5 Coder was surprisingly good at:

  • Generating SQL from natural language
  • Understanding multilingual input
  • Pattern recognition for data extraction
  • Running efficiently on 7B parameters

Future Improvements

Looking for contributors! Especially for:

  • More languages (French, Italian, Portuguese)
  • Windows/Linux ports (currently macOS only)
  • Better error handling
  • Testing on different macOS versions

Open Source

Conclusion

This scratches an itch I had - wanting the power of GPT-4 without accidentally logging sensitive data. The keyword-based routing is simple but effective.

Would love feedback on the privacy model - I'm sure there are edge cases I haven't considered!

Links

Top comments (0)