🛡️ GOLEM: An Open-Source AI Red Teaming Platform with Adaptive AI-vs-AI Engine
Hey everyone 👋,
For the past several months, I've been deep in the trenches building GOLEM – a project born from my own frustrations with LLM security testing. I wanted to share the core concept with this community and get your valuable thoughts before we launch.
😤 The Problem I Faced
Red teaming LLMs today often means:
| ❌ Pain Point |
💡 Reality |
| 🐍 Fragile Python environments |
Dependency hell & version conflicts |
| 📜 Thousands of lines of terminal output |
Impossible to analyze results visually |
| 📋 Static payload lists |
Attacks get patched within days |
| 🔧 Complex setup (Docker, venv, pip) |
High barrier to entry for quick testing |
I knew there had to be a better way.
✨ The Vision
I set out to build a tool that is:
| 🎯 Goal |
✅ Solution |
| Portable |
Single standalone binary, no dependencies |
| Visual-Rich |
Beautiful web dashboard with real-time feedback |
| Adaptive |
AI that learns and evolves attacks on the fly |
| Private |
100% local execution, your data never leaves your machine |
🧠 The Core Idea
GOLEM is an AI red teaming platform written in Go 🐹. It compiles to a single standalone binary (no Docker 🐳, no Python 🐍, no runtime headaches) and serves a local web dashboard for real-time analysis. Default attack stack contains 200+ attack prompts, and this amount can be increased using mutation engine.
📸 See the GOLEM Dashboard in action:
👉 https://imgur.com/a/2rCHU90
⚔️ The Neural Attack Engine (NAE)
But this is the part I'm most excited about 🔥
In addition to just replaying a fixed list of prompts, GOLEM pits three LLMs against each other in a closed adversarial loop:
| Role |
Function |
| 🥷 The Attacker |
Generates prompts to probe and exploit the target model |
| 🎯 The Target |
The model being audited (your LLM) |
| ⚖️ The Judge |
Evaluates whether the attack succeeded or failed |
🔄 How It Learns
Round 1: Attacker asks directly → Target refuses ❌
↓ Judge feedback: "Direct approach failed"
Round 2: Attacker tries translation bypass → Partial leak ⚠️
↓ Judge feedback: "Translation shows promise"
Round 3: Attacker frames as compliance audit → SUCCESS 🚨
↓ Full system prompt extracted!
When an attack fails, the Attacker doesn't give up. It receives feedback from the Judge, learns from the refusal, and mutates its strategy for the next round. It's an evolutionary approach to finding vulnerabilities, not a static one.
📸 See the NAE in action (GIF):
👉 https://imgur.com/a/xJvPmU8
🎥 Full Demo Video:
👉 https://www.youtube.com/watch?v=RiFo_5iB0OQ
🚀 Key Features
| Feature |
Description |
| 📦 Standalone Binary |
One file. Windows, macOS, Linux. No dependencies chaos. |
| 🔒 Local-First & Private |
Dashboard runs on localhost. Your keys, prompts, and logs never leave your machine. |
| ⚔️ 200+ Attack Vectors |
Prompt injections, jailbreaks, evasion techniques – all mapped to OWASP LLM Top 10. |
| 🎨 Visual Dashboard |
Dark-mode web UI with real-time attack feed via Server-Sent Events. |
| 🧠 Neural Attack Engine |
AI-vs-AI adaptive loop that evolves attacks based on target feedback. |
| 📊 GSS Scoring |
GOLEM Security Score (0-100) to quantify your model's resilience. |
| ⚡ Pyramid Mode |
3-phase scanning optimized for slow reasoning models. |
| 📑 Export Reports |
Generate executive-ready HTML or raw JSON reports with one click. |
🎯 Who Is This For?
| Audience |
Use Case |
| 👨💻 AI Developers |
Stress-test your models before deployment to catch vulnerabilities early. |
| 🔐 Security Researchers |
Portable, dependency-free tool for professional LLM auditing. |
| 🏠 Local LLM Enthusiasts |
Understand the vulnerabilities of the models you run locally (Ollama, LM Studio, etc.). |
| 🏢 Red Teams |
Add LLM security testing to your existing penetration testing workflow. |
📅 Release Timeline
| Date |
Event |
| 🗓️ October 24, 2024 |
🔓 Early Access opens (7 days before public release) |
| 🗓️ October 31, 2024 |
🚀 Public v0.1a Release |
🔔 Want Early Access?
Sign up 7 days before the public release to get:
- ✅ Early access to the beta binary
- ✅ Priority support via Email
- ✅ Input on feature development
- ✅ Adding in early testers list in repository
📩 Join the Early Access List:
👉 DM to [osint.technologies@gmail.com](mailto:osint.technologies@gmail.com)
🛠️ Current Status
| Component |
Status |
| Core Engine |
✅ Functional |
| Web Dashboard |
✅ Functional |
| Neural Attack Engine |
🟡 Beta (active development) |
| Attack Library (200+) |
🟡 Growing |
| Documentation |
🟡 In progress |
The project is currently in alpha. The core engine and dashboard are functional, but there are still bugs to squash and features to polish. I'm building this in the open because I believe security tools should be transparent and accessible.
🔗 Links
💬 I'd Love Your Feedback
Since this is still in development, I'm particularly interested in your thoughts on:
- 🤖 The AI-vs-AI approach: Do you think an adaptive attacker is more valuable than a large static list of prompts? Why?
- ⚖️ Feature Priorities: What's more important to you: a wider variety of pre-built attacks, or a smarter adaptive engine that learns?
- 🎯 Local LLM Testing: What specific vulnerabilities are you most concerned about when running local models? (Prompt injection? Data leakage? Jailbreaks?)
- 🔧 Integration: Would you want GOLEM to integrate with CI/CD pipelines for automated security testing?
If you have questions about the architecture, how the judge/attacker loop works, or the tech stack (Go + embedded web server), feel free to ask! 👇
⚠️ Disclaimer
GOLEM is intended for authorized security testing and research only.
Always ensure you have explicit written permission before testing any model, API, or system. Unauthorized testing may violate terms of service and applicable laws.
Thanks for reading! 🙏 Excited to hear your thoughts and build this together with the community. 🚀