The AI space has a bubble problem. Too much money chases too little substance. Products wrapped in marketing layers, selling potential instead of value. And figuring out what's real costs time — the one thing nobody has enough of.
I've been using AI itself to solve this. Not in a meta, "AI evaluating AI" way. More like: give the model a GitHub repo, get back an honest read on whether the thing is built on solid ground or thin air.
The bubble is real
Let's call it what it is. The AI hype cycle follows a familiar pattern: massive investment, breathless headlines, products that sound transformative until you look at the code. The signal-to-noise ratio is brutal right now.
Two examples landed in my inbox this week. Both looked promising on the surface. Both taught me something different when I dug in.
Case 1: massCode — the real deal
massCode is a free, open-source developer workspace. Snippets, notes, HTTP client, math notebook, dev tools — all in one local-first app.
Here's what stood out when I ran it through my filter:
- It solves a real problem. Developers juggle snippets in five different places. massCode consolidates them with actual editing power — CodeMirror, Prettier, live HTML preview, JSON inspector — not just storage.
- The architecture has depth. Snippets with fragments (multiple files per snippet), folder + tag organization, Markdown notes with Mermaid and mind maps, a full HTTP client that imports from Postman, OpenAPI, and Bruno. This is not a weekend project wrapped in a landing page.
- Local-first, not cloud-dependent. Plain Markdown files on disk. Sync via Git, iCloud, Dropbox — your choice. A design decision that says the creator understands their users.
- Real integrations. VS Code extension. Raycast extension. Import from SnippetsLab, GitHub Gists, Obsidian. These take actual work to build and maintain.
- AGPL v3 license. Genuinely open source, not "open core" with a paywall around the useful parts.
Is it perfect? No. The UI is Electron-based, which means memory overhead. The snippet search could be faster. But these are real tradeoffs from real engineering decisions, not shortcuts dressed up as features.
Verdict: genuine value. A tool built by someone who uses it daily, solving problems they've hit themselves.
Case 2: YouTube Automation Agent — the wrapper
youtube-automation-agent promises fully automated YouTube channel management. Research trending topics, write scripts, generate thumbnails, optimize SEO, upload videos — 24/7, no coding required.
Sounds compelling. Here's what the filter caught:
- It's a scheduler with API calls. The "AI agents" are Node.js modules that call the OpenAI or Gemini API, then call the YouTube Data API. The architecture diagram shows six agents, but each one is essentially: prompt → API response → format → next step. That's orchestration, not intelligence.
- The "automation" requires manual auth. OAuth flow, credential setup, environment variables. The README says "no coding required" but then walks you through creating a Google Cloud project, enabling APIs, downloading JSON credentials, and running an interactive setup wizard. That's coding-adjacent at minimum.
- Revenue claims without evidence. "50K subscribers in 3 months," "$5K/month ad revenue," "1M+ views per month on autopilot" — these are in the README with zero attribution. Classic growth hack for GitHub stars.
- The free tier math doesn't work. YouTube Data API gives you 10,000 quota units per day. A single video upload costs ~1,600 units. Add thumbnails, playlists, analytics calls — you hit the ceiling fast. The README doesn't mention this.
- Video generation is absent. The agent writes scripts and generates thumbnail images. It doesn't produce actual video. For channels that need video — most of YouTube — you still need a separate production pipeline. The "automation" stops at metadata.
Verdict: overwrapped. Not useless — the scheduling and SEO optimization could save time if you already have a workflow. But the marketing promises full automation that the code can't deliver.
How I use AI for this evaluation
The process is straightforward:
- Send the repo URL to the model. I include the README, the file structure, and key source files.
- Ask specific questions. Not "is this good?" but "what does the dependency tree look like?", "how many of these agents are thin wrappers vs. substantial logic?", "what would break if the upstream API changed pricing?"
- Cross-reference with repo stats. Stars, contributors, commit frequency, issue response time — these are public and usually tell you more than the landing page.
- Apply the smell test. If removing AI from the product leaves nothing, the product is the AI wrapper. If removing AI still leaves a functional tool, the AI is additive.
This takes about five minutes per repo. Before I used this approach, I'd spend 30-45 minutes cloning, reading, and testing. The model doesn't replace my judgment — it filters the queue so I spend my time on repos worth examining closely.
What survives the bubble
Products that outlast hype cycles share a pattern:
- They solve a problem the creator has. massCode's Anton Reshetov built it because existing snippet managers didn't cut it. That's evident in the feature depth.
- AI is a component, not the product. The best tools use AI to accelerate a workflow that's already valuable without it.
- The marketing matches the code. If the README promises "fully automated 24/7" and the code requires OAuth setup, cron jobs, and manual topic selection, the marketing is the product.
- They have escape hatches. Local storage, open formats, export options. If the product disappears tomorrow, can you still access your data?
The AI bubble will pop — not because AI is useless, but because most of the money and attention went to products that are thin wrappers around an API. The ones built on real problems, by people who use their own tools daily, will stick around.
Using AI to filter AI products isn't ironic. It's practical. The models are good at pattern matching, and "overpromised, underbuilt" is a pattern they recognize easily. The trick is asking the right questions — and knowing that the model's answer is a starting point, not a conclusion.