Back to Free Stuff

Free Guide — April 2026

Build a Content
Intelligence System

Track what your Instagram competitors are posting, which Reels are performing, what hooks are winning, and where the content gaps are. Built for Instagram. Works for TikTok, YouTube, and any social platform. Powered by Claude Code.

Section 1

What This System Actually Does

Every morning, you open your terminal and type one thing: "competitor pulse."

Claude pulls fresh data on every Instagram competitor you're tracking. It tells you who posted Reels this week, which ones got the most views, what hooks they opened with, and what changed since yesterday. No scrolling through competitor profiles. No spreadsheets. No "let me check their page real quick." It's already done.

This is not a dashboard you update manually. It's a content intelligence pipeline. Data flows in from Instagram automatically, gets enriched by AI (transcription, hook classification, engagement scoring), and surfaces the patterns you'd never spot by scrolling.

The system is built for Instagram, but the exact same architecture works for TikTok, YouTube Shorts, or any platform where you're competing for attention with content.

Here's what the system tracks:

1

Posting activity

Which competitors posted Reels this week? How many? Who went silent?

2

Top-performing Reels

The 10 highest-viewed Reels across all competitors in the last 7 days.

3

Hook patterns

Which opening hooks correlate with higher views? Questions vs. lists vs. stories vs. controversial takes.

4

Caption CTAs

What "comment KEYWORD" triggers are competitors using in captions? Which ones drive the most comments?

5

Outlier detection

Reels performing 2x or more above a competitor's average. These are signals the algorithm is boosting a topic or format.

6

Posting time analysis

When are competitors posting Reels? Which hours correlate with the highest view counts?

Section 2

The 5-Layer Architecture

The system I built tracks Instagram competitors, but the architecture behind it follows a 5-layer pattern that works for any content platform. Here's how the layers stack.

1

Collect

Scrape public content from competitor Instagram profiles on a schedule. Reels, carousels, captions, view counts, likes, comments. A scraping tool pulls it automatically so you never have to visit their profiles yourself.

2

Store

Raw data lands in a database. Two tables: one for WHO you're tracking (competitor accounts), one for WHAT they post (every Reel, carousel, and post with its metrics). This structured storage is what makes everything else possible.

3

Enrich

AI processes the raw content into structured intelligence. This is where the magic happens. AI transcribes every Reel so you can read exactly what competitors said. Then it classifies each hook into a category (list, question, story, controversy, stat, tutorial) so you can see which formats actually perform. Raw content becomes categorized, scored, and searchable.

4

Track

Weekly metric refresh catches changes over time. How did that Reel's views grow after the first 24 hours? Which competitors are gaining momentum? Which posts went viral late? Tracking turns a snapshot into a trend.

5

Analyze

Smart queries surface content patterns, outliers, and actionable signals. Instead of scrolling competitor profiles, you ask Claude: "What hooks are working this week?" "Who had a breakout Reel?" "When should I post?" The system answers with data.

Same Architecture, Any Platform or Industry

This guide walks through the Instagram version, but the 5 layers stay the same no matter what you're tracking. Swap the data source and the same system works for TikTok, YouTube, or even industries outside social media entirely:

Content Creator

Track competitor reels, posts, and videos

Collect

Scrape reels, captions, view counts

Store

Creators table + videos table

Enrich

Transcribe spoken hooks, classify opening patterns

Track

View velocity, engagement rate over time

Analyze

Winning hooks, best posting times, outlier content

Restaurant Owner

Track competitor reviews, menus, and ratings

Collect

Scrape Google/Yelp reviews, menu pages

Store

Restaurants table + reviews table

Enrich

Sentiment analysis, menu categorization

Track

Rating changes, new menu items, review volume

Analyze

What dishes get praised, complaint patterns, pricing gaps

SaaS Founder

Track competitor features, launches, and reviews

Collect

Scrape changelogs, Product Hunt, G2 reviews

Store

Competitors table + features table

Enrich

Feature categorization, sentiment scoring

Track

New launches, pricing changes, review trends

Analyze

Feature gaps, positioning opportunities, user complaints

Real Estate Agent

Track competitor listings, pricing, and market moves

Collect

Scrape new listings, price changes, days on market

Store

Agents table + listings table

Enrich

Property classification, price trend extraction

Track

Price drops, inventory velocity, new entrants

Analyze

Pricing patterns, underserved areas, listing quality

E-commerce Brand

Track competitor products, pricing, and customer sentiment

Collect

Scrape product pages, pricing, reviews

Store

Brands table + products table

Enrich

Price tracking, review sentiment, feature extraction

Track

Price fluctuations, new SKUs, review volume shifts

Analyze

Pricing positioning, product gaps, customer pain points

Section 3

What You Need

Four tools. Most of them have free tiers. Total cost for most setups: under $5/month.

1

Claude Code

This is the brain. Claude Code orchestrates everything: it writes the scraping scripts, sets up your database, builds the enrichment pipeline, and creates the skills you'll use to query it all. You describe what you want. Claude builds it.

2

A scraping tool (like Apify)

This is what actually goes out and pulls public data from competitor Instagram profiles. Apify has a marketplace with pre-built scrapers for Instagram, TikTok, YouTube, and dozens of other platforms. You don't write scrapers from scratch. You pick one and point it at the accounts you want to track.

Cost: about $0.50 to $2 per run, depending on how many competitors you're tracking.

3

A database (like Supabase or Airtable)

Somewhere to store all the data the scraper collects. Supabase is a free, hosted database that works great for this. Airtable works too if you prefer something more visual. The point is: the data needs a home where Claude can query it later.

Cost: free tier handles this easily.

4

An AI enrichment layer (like OpenAI)

This is what turns raw data into intelligence. For video content, it transcribes the audio so you can read what competitors actually said. Then it classifies hooks, analyzes sentiment, or extracts key details. Raw data goes in. Structured insights come out.

Cost: fractions of a cent per item. Negligible at this scale.

You don't need to know how to code. Claude Code handles the technical implementation. You tell it what competitors you want to track, what data matters to you, and what questions you want answered. Claude writes the scripts, sets up the database, and builds the pipeline. Your job is to describe the system. Claude's job is to build it.

Section 4

Your Database (The Foundation)

Every content intelligence system has the same database shape: two tables. One for who you're tracking. One for what they post. The columns change depending on your platform, but the structure is always the same.

Table 1: Creators. This stores every competitor account you're watching. Their username, follower count, platform, and when they were last scraped. Think of it as your competitor roster.

Table 2: Their content. This is where the actual data lives. Every Reel, carousel, or post gets a row. Each row captures the caption, view count, likes, comments, engagement rate, when it was posted, and any enrichment data the AI adds later (like transcripts, hook classifications, and CTA keywords).

The relationship between the two tables is simple: every item in Table 2 connects back to a competitor in Table 1. That's what lets you ask questions like "Who had the best week?" or "Which competitor's content is outperforming their average?"

What the Tables Look Like Per Industry

Content Creator

Competitors table

  • Username / handle
  • Platform
  • Follower count
  • Last scraped date

Content table

  • Post URL
  • Caption / first line
  • View count, likes, comments
  • Engagement rate
  • Transcript (AI-added)
  • Hook category (AI-classified)
  • Posted date

Restaurant Owner

Restaurants table

  • Restaurant name
  • Location / neighborhood
  • Cuisine type
  • Overall rating

Reviews table

  • Review source (Google, Yelp)
  • Star rating
  • Review text
  • Sentiment score (AI-added)
  • Mentioned dishes (AI-extracted)
  • Review date

E-commerce Brand

Brands table

  • Brand name
  • Website URL
  • Product category
  • Price range

Products table

  • Product name
  • Current price
  • Previous price
  • Rating, review count
  • Key features (AI-extracted)
  • Last checked date

You tell Claude what competitors you want to track and what data points matter to your industry. Claude creates the tables for you. No SQL required.

Section 5

The Pipeline (How Data Flows)

Once your database is set up, you need a pipeline that keeps it fresh. Four steps run in sequence, either daily or weekly depending on your needs.

1

Collect

The scraper goes out and pulls the latest Reels, carousels, and posts from every competitor Instagram account you're tracking. Captions, view counts, likes, comments, video URLs. All of it.

You choose how aggressive to be. A daily run grabs the 5 most recent posts per competitor. A weekly refresh grabs 20+. An initial seed run pulls 100 posts to build your historical baseline.

Other platforms: For TikTok, you'd swap to a TikTok scraper actor. For YouTube, a YouTube scraper. The pipeline doesn't change. Just the data source.

2

Store

Raw data gets parsed and loaded into your database. The pipeline handles deduplication automatically. If a Reel was already scraped yesterday, it gets updated (new view counts, new comments) instead of duplicated.

During this step, the pipeline also extracts key fields from each post: the first line of the caption (that's the hook people see first), any CTA keywords (like "comment CONTENT below"), and hashtags. These extracted fields are what power the analysis later.

3

Enrich

This is the AI layer. It takes the raw content and adds intelligence on top of it.

For Reels, AI transcribes the audio so you can read exactly what a competitor said in their video without watching it. Then it classifies the hook into a category: was it a list? A question? A controversial take? A story? A stat? A tutorial? This classification is what lets you later ask "Which hook type is performing best this month?"

The enrichment step adapts depending on what you're tracking:

  • Instagram / TikTok: Transcribe Reel audio, classify opening hooks into 10 categories
  • Restaurants: Run sentiment analysis on reviews, extract mentioned dishes
  • SaaS: Categorize features, score user sentiment on G2 reviews
  • Real estate: Classify property types, extract price trends from listing history
  • E-commerce: Track price changes, analyze review sentiment per product
4

Track

A weekly refresh re-checks recent Reels and updates their metrics. This is how you measure velocity. A Reel that had 5,000 views on Monday might have 50,000 by Friday. Without tracking, you'd never see the growth curve.

The system stores the previous view count alongside the current one. That delta is what powers outlier detection. When a Reel grows 2x faster than that competitor's average, the system flags it. That's a signal worth paying attention to.

The whole pipeline chains together. You can run all four steps with a single command. Claude Code sets this up for you. Once it's built, you run one command and the entire pipeline executes: scrape, store, enrich, track. Fresh intelligence, every morning.

Section 6

The Intelligence Layer

You have the data. Now you need to ask the right questions. The system answers six types of questions that turn raw data into decisions.

1

Activity Summary

"Who's active and who went quiet?"

Shows every competitor ranked by performance this week. How many Reels they posted, their average views, their peak performer. When a competitor goes silent, that's a signal too. Maybe they're regrouping. Maybe they're burned out. Either way, you know.

2

Top Content

"What's winning right now?"

The 10 highest-viewed Reels from the last 7 days across all competitors. You see the hook that won, the views it got, and the engagement it drove. This is your cheat sheet for what content the algorithm is pushing right now.

3

Patterns

"What formats and approaches are working?"

This groups Reels by opening hook style and shows which patterns correlate with higher views. Are question hooks outperforming list hooks? Are controversial takes getting more reach than tutorials? You stop guessing what format to film and start seeing the data.

4

Engagement Triggers

"What's driving action?"

Tracks the specific "comment KEYWORD" triggers competitors are using in their captions and how well each one performs. "Comment SKILLS" might drive 3x more engagement than "comment GUIDE." You see which caption CTAs are working across the entire competitive landscape, not just one account.

5

Outlier Detection

"What just broke out?"

Finds Reels performing 2x or more above that competitor's 30-day average. These aren't just good posts. They're anomalies. Something about the topic, the format, or the timing hit differently. These are the signals that tell you what the Instagram algorithm is rewarding right now.

6

Timing Analysis

"When should I publish?"

Maps out when competitors are posting Reels and which time slots correlate with the highest view counts. Instead of guessing the "best time to post," you see the actual data. You might find that 5 PM Eastern consistently outperforms everything else. Or that weekends are a dead zone in your niche. Data beats intuition.

For TikTok, the same six questions work identically. Swap the Instagram scraper for a TikTok scraper, and you're tracking competitor TikToks instead of Reels. Same database, same enrichment, same analysis. The architecture doesn't change.

Section 7

Making It Conversational

Here's where it gets really interesting. Claude Code has a feature called skills. A skill is a set of instructions that tells Claude how to do a specific job. Once a skill is set up, you trigger it with a simple phrase instead of remembering commands or queries.

For this system, you build two skills that sit on top of your pipeline:

Skill 1

Competitor Intel

This skill does two things. Say "scrape competitors" and it runs the full pipeline: pulls the latest Reels from every competitor, stores them, transcribes the audio, classifies hooks, and updates metrics. Say "what's working on Instagram?" and it queries your database and gives you a brief with top Reels, hook patterns, and trends.

It also maintains a simple config file listing which competitors to track. Adding a new one is as easy as telling Claude "add @newcompetitor to my tracking list."

Skill 2

Competitor Pulse

The quick-hit daily brief. Type "competitor pulse" and you get a full Instagram intelligence report: who posted Reels this week, what's getting the most views, any outlier content, hook patterns across all competitors, which caption CTAs are driving engagement, and the best posting times. All in one formatted output.

This is the skill you run every morning. One phrase. Full competitive picture.

The beauty of skills is that they turn a technical pipeline into a conversation. You don't need to remember database queries or script names. You just talk to Claude the way you'd talk to a research assistant.

Example prompts you'd use:

  • "Competitor pulse" → full weekly intelligence brief
  • "What hooks are working this week?" → pattern analysis with performance data
  • "Scrape competitors" → runs the full data collection pipeline
  • "Who had a breakout post?" → outlier detection with context
  • "When should I post?" → timing analysis based on competitor data

Work with Me

Need AI to actually work for your business?

I help businesses cut through the AI hype and build the workflows, automations, and systems that actually move the needle. Direct, hands-on, no fluff.

Work with me