<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[serpbase]]></title><description><![CDATA[serpbase]]></description><link>https://aidenlinserpbase.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sat, 21 Nov 2026 14:41:35 GMT</lastBuildDate><atom:link href="https://aidenlinserpbase.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Brand monitoring in 15 lines: only the pages Google found in the last 24 hours]]></title><description><![CDATA[Most brand-monitoring tools charge $50+ a month for something you can replace with a tiny cron job. If all you want is a daily list of new pages mentioning…]]></description><link>https://aidenlinserpbase.hashnode.dev/brand-monitoring-in-15-lines-only-the-pages-google-found-in-the-last-24-hours</link><guid isPermaLink="true">https://aidenlinserpbase.hashnode.dev/brand-monitoring-in-15-lines-only-the-pages-google-found-in-the-last-24-hours</guid><dc:creator><![CDATA[Aiden Lin]]></dc:creator><pubDate>Fri, 20 Nov 2026 03:19:54 GMT</pubDate><content:encoded><![CDATA[<p>Most brand-monitoring tools charge $50+ a month for something you can replace with a tiny cron job. If all you want is a daily list of new pages mentioning your product鈥攚ithout wading through your own site or two-year-old reviews鈥攜ou only need a google search api past 24 hours query.</p>
<p>Google's <code>tbs=qdr:d</code> parameter restricts results to the last 24 hours. Pair that with a quoted brand name and <code>-site:yourdomain.com</code>, and the whole script fits in 15 lines of Python.</p>
<h2>The 15-line script</h2>
<pre><code class="language-python">import os, requests

def brand_mentions(brand: str, own_domain: str):
    resp = requests.post(
        "https://api.serpbase.dev/google/search",
        headers={"X-API-Key": os.environ["SERPBASE_API_KEY"]},
        json={
            "q": f'"{brand}" -site:{own_domain}',
            "gl": "us", "hl": "en", "tbs": "qdr:d",
        },
        timeout=30,
    ).json()
    return [(o.get("date"), o["title"], o["link"]) for o in resp.get("organic", [])]

for date, title, link in brand_mentions("SerpBase", "serpbase.dev"):
    print(f"[{date or 'recent'}] {title}\n  {link}")
</code></pre>
<p>That is the entire core. Every call returns structured JSON (<code>organic[]</code> with <code>title</code>, <code>link</code>, <code>snippet</code>, and <code>date</code>), so there is no HTML parser to break when Google tweaks its markup.</p>
<h2>Why each piece matters</h2>
<ul>
<li><code>"SerpBase"</code> <strong>(quotes)</strong>: forces exact phrase matching so Google does not broaden your product name into loosely related words.</li>
<li><code>-site:serpbase.dev</code>: drops your own docs, changelog, and blog posts so you only see external mentions.</li>
<li><code>"tbs": "qdr:d"</code>: limits Google's index lookup to pages dated within the past 24 hours. If you run the script weekly instead of daily, change it to <code>"qdr:w"</code>. If you want newest-first ordering on top of that, use <code>"qdr:d,sbd:1"</code>.</li>
</ul>
<h2>Making it quiet enough for a daily cron</h2>
<p>Running <code>qdr:d</code> once a day has one quirk: Google decides a page's date from the page itself, so an older post whose "Updated" timestamp changed can show up again tomorrow. Adding a plain text file of seen URLs stops repeat alerts without needing a database:</p>
<pre><code class="language-python">from pathlib import Path

seen_file = Path("seen_links.txt")
seen = set(seen_file.read_text().splitlines()) if seen_file.exists() else set()

new_hits = [row for row in brand_mentions("SerpBase", "serpbase.dev") if row[2] not in seen]
if new_hits:
    with seen_file.open("a") as f:
        for date, title, link in new_hits:
            print(f"[{date or 'recent'}] {title} -&gt; {link}")
            f.write(link + "\n")
</code></pre>
<p>Drop it into cron (<code>0 9 * * *</code>) and pipe stdout to Slack, Discord, or email. At one request per brand query per day, that is roughly 30 requests a month.</p>
<p>For the full list of <code>tbs</code> date values鈥攊ncluding custom date ranges like <code>cdr:1,cd_min:1/1/2026,cd_max:3/31/2026</code> and the edge cases around Google's date handling鈥攕ee <a href="https://serpbase.dev/blog/google-search-api-date-filter-get-results-from-the-past-day-week-or-a-custom-ran">Google Search API Date Filter: Get Results From the Past Day, Week or a Custom Range</a>. Full disclosure: I work on SerpBase, and this is a trimmed-down version of the monitor we run ourselves.</p>
]]></content:encoded></item><item><title><![CDATA[Track AI Overview and People Also Ask on your keywords every day (Python)]]></title><description><![CDATA[A daily Python snapshot of AI Overview and People Also Ask presence, stored as JSONL and diffed to show what changed.]]></description><link>https://aidenlinserpbase.hashnode.dev/track-ai-overview-and-people-also-ask-on-your-keywords-every-day-python</link><guid isPermaLink="true">https://aidenlinserpbase.hashnode.dev/track-ai-overview-and-people-also-ask-on-your-keywords-every-day-python</guid><category><![CDATA[Python]]></category><category><![CDATA[SEO]]></category><category><![CDATA[api]]></category><category><![CDATA[Google]]></category><category><![CDATA[SERP]]></category><dc:creator><![CDATA[Aiden Lin]]></dc:creator><pubDate>Tue, 17 Nov 2026 04:04:13 GMT</pubDate><content:encoded><![CDATA[<p>A rank tracker can tell you a page is third. It cannot tell you that third place is now below an AI Overview, a People Also Ask block, and a video carousel. The number stayed the same. The page did not.</p>
<p>That is the gap I wanted to close with a small daily job: for a fixed set of keywords, save which SERP features are present. Not a screenshot. A JSON record I can diff.</p>
<h2>What to record</h2>
<p>One record per query, market, and day. Keep it boring:</p>
<ul>
<li><code>query</code>, <code>gl</code>, <code>hl</code>, <code>collected_at</code></li>
<li>booleans for <code>ai_overview</code>, <code>people_also_ask</code>, <code>featured_snippet</code>, <code>top_stories</code>, <code>knowledge_graph</code>, <code>local_results</code></li>
<li>the top 10 domains in <code>organic</code></li>
<li>the domains cited inside the AI Overview, and the domains answering PAA</li>
</ul>
<p>Presence is the first layer. Ownership is the second. If an AI Overview appears and cites your page, that is visibility. If it appears and cites three competitors, you are still below it and not in it.</p>
<h2>The snapshot script</h2>
<p>You need Python 3.8+, <code>requests</code>, and a SerpBase key in an environment variable.</p>
<pre><code class="language-bash">pip install requests
export SERPBASE_API_KEY="your_key"
</code></pre>
<pre><code class="language-python">import json
import os
import time
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlparse

import requests

API = "https://api.serpbase.dev/google/search"
KEY = os.environ["SERPBASE_API_KEY"]
OUT = Path("serp_snapshots.jsonl")

FEATURES = [
    "ai_overview",
    "people_also_ask",
    "featured_snippet",
    "top_stories",
    "knowledge_graph",
    "local_results",
]


def host(url: str) -&gt; str:
    return urlparse(url).netloc.replace("www.", "")


def snapshot(query: str, gl: str = "us", hl: str = "en") -&gt; dict:
    resp = requests.post(
        API,
        headers={"X-API-Key": KEY, "Content-Type": "application/json"},
        json={"q": query, "gl": gl, "hl": hl, "page": 1},
        timeout=20,
    )
    resp.raise_for_status()
    data = resp.json()

    # HTTP 200 is not the success signal. status == 0 is.
    if data.get("status") != 0:
        raise RuntimeError(
            f"status={data.get('status')} request_id={data.get('request_id')}"
        )

    organic = data.get("organic", [])[:10]
    aio = data.get("ai_overview") or {}
    paa = data.get("people_also_ask") or []

    return {
        "query": query,
        "gl": gl,
        "hl": hl,
        "collected_at": datetime.now(timezone.utc).isoformat(),
        "features": {name: bool(data.get(name)) for name in FEATURES},
        "top_domains": [host(r.get("link", "")) for r in organic if r.get("link")],
        "aio_sources": [
            host(s.get("link", "")) for s in aio.get("sources", []) if s.get("link")
        ],
        "paa_domains": [host(i.get("link", "")) for i in paa if i.get("link")],
    }


def save(snap: dict) -&gt; None:
    with OUT.open("a", encoding="utf-8") as f:
        f.write(json.dumps(snap, ensure_ascii=False) + "\n")


if __name__ == "__main__":
    queries = [
        "best crm software for agencies",
        "serp api pricing",
    ]
    for query in queries:
        record = snapshot(query)
        save(record)
        print(query, record["features"])
        time.sleep(1)
</code></pre>
<p>Each feature field is present only when Google shows that module. <code>ai_overview</code> is an object, <code>people_also_ask</code> is an array, <code>featured_snippet</code> is an object. <code>bool()</code> on the missing key gives <code>False</code>, which is what we want for a presence flag.</p>
<h2>Diff yesterday and today</h2>
<p>One JSON line per snapshot makes the diff simple.</p>
<pre><code class="language-python">def load() -&gt; list[dict]:
    if not OUT.exists():
        return []
    lines = OUT.read_text(encoding="utf-8").splitlines()
    return [json.loads(line) for line in lines if line.strip()]


def diff(prev: dict, curr: dict) -&gt; dict:
    gained = [
        name
        for name, value in curr["features"].items()
        if value and not prev["features"].get(name)
    ]
    lost = [
        name
        for name, value in prev["features"].items()
        if value and not curr["features"].get(name)
    ]
    return {"gained": gained, "lost": lost}


rows = load()
for query in {row["query"] for row in rows}:
    series = [row for row in rows if row["query"] == query]
    if len(series) &gt;= 2:
        print(query, diff(series[-2], series[-1]))
</code></pre>
<p>Output that matters looks like this:</p>
<pre><code class="language-text">best crm software for agencies {'gained': ['ai_overview', 'people_also_ask'], 'lost': []}
</code></pre>
<p>That is a usable alert. A PAA block appeared on a commercial query. The rank did not move. The page above your result changed.</p>
<h2>Who owns the feature</h2>
<p>The presence flag tells you the page changed. The sources tell you whether you are in it.</p>
<p>The snapshot already stores <code>aio_sources</code> and <code>paa_domains</code>. Once a day, check whether your domain is in either list:</p>
<pre><code class="language-python">YOUR_DOMAIN = "serpbase.dev"

for row in rows:
    if row["features"]["ai_overview"]:
        print(row["query"], "cited:", YOUR_DOMAIN in row["aio_sources"])
</code></pre>
<p>If the AI Overview cites you, you are part of the answer. If it cites three competitors, you are below the answer. Both are different problems from "we rank third".</p>
<h2>Occupancy across a keyword set</h2>
<p>One keyword is an anecdote. A keyword set is a plan.</p>
<pre><code class="language-python">def occupancy(rows: list[dict], feature: str) -&gt; float:
    if not rows:
        return 0.0
    count = sum(1 for row in rows if row["features"].get(feature))
    return round(100 * count / len(rows), 1)


print("ai_overview", occupancy(rows, "ai_overview"))
print("people_also_ask", occupancy(rows, "people_also_ask"))
</code></pre>
<p>If AI Overviews appear on most of your informational keywords, the layout is the story. If PAA is everywhere but your domain is never in <code>paa_domains</code>, you have a content format problem, not a rank problem.</p>
<h2>Run it daily</h2>
<pre><code class="language-bash">crontab -e
0 8 * * * cd /path/to/project &amp;&amp; /usr/bin/python3 snapshot.py &gt;&gt; serp.log 2&gt;&amp;1
</code></pre>
<p>Keep <code>gl</code>, <code>hl</code>, and the hour fixed. A feature snapshot taken in a different market or at a different time of day is a different measurement.</p>
<p>One Google web search is 1 credit. New accounts get 500 free credits without a card. At the Pro Monthly tier the rate is $0.25 per 1,000 searches; a daily run of 20 keywords is about 600 requests a month.</p>
<p>The longer version of this method, including feature ownership and the database schema, is here: <a href="https://serpbase.dev/blog/serp-feature-monitoring-track-the-search-page-not-just-rankings">SERP feature monitoring: track the search page, not just rankings</a>. SerpBase is our API; the code above is the same call I use when I need the page as JSON.</p>
<p>Start with 20 keywords in one market and run it for a week. The first useful alert will not be a rank change. It will be the day an AI Overview appears on a keyword you thought was stable, and you can see exactly what the page looked like before.</p>
]]></content:encoded></item><item><title><![CDATA[Google AI Mode API in Python: answer text and citations as JSON]]></title><description><![CDATA[Call Google AI Mode from Python, parse the answer text, Markdown and cited URLs, and save one JSON record per question.]]></description><link>https://aidenlinserpbase.hashnode.dev/google-ai-mode-api-in-python-answer-text-and-citations-as-json</link><guid isPermaLink="true">https://aidenlinserpbase.hashnode.dev/google-ai-mode-api-in-python-answer-text-and-citations-as-json</guid><category><![CDATA[api]]></category><category><![CDATA[Google]]></category><category><![CDATA[Python]]></category><category><![CDATA[SEO]]></category><category><![CDATA[AI]]></category><dc:creator><![CDATA[Aiden Lin]]></dc:creator><pubDate>Mon, 16 Nov 2026 09:56:36 GMT</pubDate><content:encoded><![CDATA[<p>A client asked me to "pull the AI answer" for forty questions and put it in a spreadsheet. The hard part was not the request. It was that the answer, the Markdown version, and the pages Google cites are three different things, and I wanted all three in one record.</p>
<p>AI Mode is not AI Overview. An AI Overview is a module on a normal results page. AI Mode answers the question in a conversation-style block and attaches the pages it used. SerpBase exposes it as <code>POST /google/ai_mode</code>, and the response has a clean <code>ai_mode</code> object: <code>text</code>, <code>markdown</code>, and <code>sources</code>.</p>
<p>Here is the whole call:</p>
<pre><code class="language-python">import json
import os

import requests

API = "https://api.serpbase.dev/google/ai_mode"


def ai_mode(question: str, gl: str = "us", hl: str = "en") -&gt; dict:
    resp = requests.post(
        API,
        headers={
            "X-API-Key": os.environ["SERPBASE_API_KEY"],
            "Content-Type": "application/json",
        },
        json={"q": question, "gl": gl, "hl": hl},
        timeout=60,
    )
    resp.raise_for_status()
    data = resp.json()

    # HTTP 200 is not the success signal. status == 0 is.
    if data.get("status") != 0:
        raise RuntimeError(
            f"status={data.get('status')} request_id={data.get('request_id')}"
        )

    block = data.get("ai_mode") or {}
    return {
        "question": question,
        "answer": block.get("text", ""),
        "markdown": block.get("markdown", ""),
        "sources": [
            {
                "title": s.get("page_title") or s.get("title"),
                "url": s.get("link"),
                "snippet": s.get("snippet"),
            }
            for s in block.get("sources", [])
        ],
        "request_id": data.get("request_id"),
        "credits_charged": data.get("credits_charged"),
    }


if __name__ == "__main__":
    result = ai_mode("how does a heat pump work")
    print(json.dumps(result, ensure_ascii=False, indent=2))
</code></pre>
<p>The response fields I rely on:</p>
<ul>
<li><code>ai_mode.text</code> — the answer as plain text.</li>
<li><code>ai_mode.markdown</code> — the same answer with headings, lists and tables kept as Markdown.</li>
<li><code>ai_mode.sources[]</code> — each cited page with <code>title</code>, <code>page_title</code>, <code>link</code>, <code>snippet</code>. <code>link</code> is the real destination, not a <code>google.com/goto</code> wrapper.</li>
<li><code>status</code>, <code>request_id</code>, <code>credits_charged</code> — from the envelope, next to the payload.</li>
</ul>
<p>A trimmed response looks like this (fields only, text shortened):</p>
<pre><code class="language-json">{
  "status": 0,
  "request_id": "r-...",
  "credits_charged": 1,
  "product_id": "google_ai_mode",
  "query": "how does a heat pump work",
  "ai_mode": {
    "text": "A heat pump moves heat instead of generating it...",
    "markdown": "## How it works\n\nA heat pump moves heat...",
    "sources": [
      {
        "title": "Heat pump",
        "page_title": "Heat pump - Wikipedia",
        "link": "https://en.wikipedia.org/wiki/Heat_pump",
        "snippet": "A heat pump is a device that transfers heat energy..."
      }
    ]
  }
}
</code></pre>
<h2>Checking whether Google cited you</h2>
<p>The reason I wanted <code>sources</code> as structured data was to answer a simple question: did the AI answer use our page? Once the call returns a dict, that is a one-liner:</p>
<pre><code class="language-python">from urllib.parse import urlparse


def cited(result: dict, domain: str) -&gt; list[dict]:
    return [
        s
        for s in result["sources"]
        if urlparse(s["url"]).netloc.endswith(domain)
    ]


hits = cited(result, "serpbase.dev")
print(len(hits))
</code></pre>
<p>Run it for a fixed list of questions once a day, store the JSON, and you have a citation monitor. The same loop also gives you the answer text, so you can diff what changed.</p>
<h2>Two details that matter</h2>
<p><code>format</code> changes the response type. The default <code>json</code> gives the envelope above. With <code>"format": "markdown"</code> the endpoint returns Markdown directly (<code>text/markdown</code>, no envelope), which is the right shape for feeding an LLM. Keep the JSON mode when you need the citations and the request ID.</p>
<p>The endpoint is priced per request. At the time of writing (2026-11-16) it is 1 credit under a launch promo; the docs say it becomes 3 credits on 2026-11-26 16:00 UTC. Failed requests return a non-zero <code>status</code> and are not charged.</p>
<p>The request contract and field names are in the <a href="https://serpbase.dev/docs/google_ai_mode">Google AI Mode API docs</a>. I work on SerpBase, so that is our endpoint; the code above is the same call I use when I need the answer and its sources in one record.</p>
<p>Pick twenty questions your buyers actually ask, run them through the function, and check the <code>sources</code> list before you write anything else. The answer is the easy part. The citations tell you whether you are in the conversation at all.</p>
]]></content:encoded></item><item><title><![CDATA[Get a SERP API key and run your first Google search in Python]]></title><description><![CDATA[A friend pinged me last week holding a key he had copied from a "get your API key" page, with one question: where does this actually go? The docs he had…]]></description><link>https://aidenlinserpbase.hashnode.dev/get-a-serp-api-key-and-run-your-first-google-search-in-python</link><guid isPermaLink="true">https://aidenlinserpbase.hashnode.dev/get-a-serp-api-key-and-run-your-first-google-search-in-python</guid><dc:creator><![CDATA[Aiden Lin]]></dc:creator><pubDate>Fri, 06 Nov 2026 04:09:08 GMT</pubDate><content:encoded><![CDATA[<p>A friend pinged me last week holding a key he had copied from a "get your API key" page, with one question: where does this actually go? The docs he had read ended at the key. So here is the whole loop in one place — create a SerpBase key, confirm it with one curl, then run the same search from Python.</p>
<h2>1. Create the account and the key</h2>
<p>Sign up at <a href="https://serpbase.dev/register">serpbase.dev/register</a>. New accounts get 500 free credits and no card is required, so you can finish this tutorial without paying.</p>
<p>Open <strong>API keys</strong> in the dashboard and create one. You see it once, so copy it into a local env var instead of your source file:</p>
<pre><code class="language-bash">export SERPBASE_API_KEY="your_key_here"
</code></pre>
<p>One rule that saves trouble later: the key goes in the <code>X-API-Key</code> header only. Not in the URL, not in front-end JavaScript. Anyone who reads it can spend your credits.</p>
<h2>2. Send one request with curl</h2>
<p>Before writing Python, prove the SERP API key works with a single request. The Google Web Search endpoint takes a JSON body over POST:</p>
<pre><code class="language-bash">curl -X POST 'https://api.serpbase.dev/google/search' \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $SERPBASE_API_KEY" \
  --max-time 60 \
  -d '{"q":"coffee shops","gl":"us","hl":"en","device":"mobile"}'
</code></pre>
<p>You should get JSON back that starts like this (shortened):</p>
<pre><code class="language-json">{
  "status": 0,
  "credits_charged": 1,
  "request_id": "r-4b27fe8a36d9ac303a5da7dd",
  "search_type": "search"
}
</code></pre>
<p>Read <code>status</code> before anything else. <code>0</code> means the search was delivered; any other value carries an error message, and a failed search costs nothing. <code>credits_charged</code> is what the call actually cost — one Google search is 1 credit — and <code>request_id</code> is the string support will ask for if something looks wrong.</p>
<h2>3. The same search in Python</h2>
<p>Install <code>requests</code> if you have not already, then keep the request shape identical:</p>
<pre><code class="language-python">import os
import requests

API_KEY = os.environ["SERPBASE_API_KEY"]

resp = requests.post(
    "https://api.serpbase.dev/google/search",
    headers={
        "Content-Type": "application/json",
        "X-API-Key": API_KEY,
    },
    json={
        "q": "coffee shops",
        "gl": "us",
        "hl": "en",
        "device": "mobile",
    },
    timeout=60,
)

data = resp.json()

if data.get("status") != 0:
    raise SystemExit(f"search failed: {data}")

print(f"cost: {data['credits_charged']} credit(s)")
print(f"request: {data['request_id']}")

for item in data.get("organic", [])[:5]:
    print(item["rank"], item["title"], item["link"])
</code></pre>
<p>Run it and you get the cost, the request id, and the first five organic results with their rank, title, and URL.</p>
<h2>4. Where the useful fields are</h2>
<p>The response keeps Google's blocks in separate keys instead of flattening them: <code>organic</code> for the blue links, <code>people_also_ask</code> for the question box, <code>related_searches</code> for the bottom suggestions, and <code>local_results</code> when a local pack shows up. If you are feeding this into an LLM, the same call can return Markdown instead of JSON.</p>
<p>Credits are prepaid and the standard packs do not expire on a calendar date, so a larger pack at a lower rate per search does not force you to spend it before a deadline.</p>
<hr />
<p>That is the whole loop. Once the Python snippet runs, swap <code>q</code> for your own term and you have a working SERP API key in a script.</p>
<p>*Canonical version of this tutorial: <a href="https://serpbase.dev/docs">serpbase.dev/docs</a>.</p>
]]></content:encoded></item></channel></rss>