Snag Watcher

Guides

Sites and robots.txt

These are the sites we can read, grouped by type. To watch one, put the entry from its where.sites column in your watcher's where.sites.

If a site's robots.txt disallows the pages with the data, the watcher needs fetch.accept_site_terms: true, which means you take responsibility for fetching it under that site's terms (see robots.txt). If a site walls our server off or isn't connected yet, creating or previewing the watcher fails with site_unavailable.

Dates in attributes.from and attributes.to are YYYY-MM-DD, and both days are included. Sites that take a time zone read attributes.time_zone, an IANA name like America/New_York. These and the other settings below (like party size, ids and selector) tell us what to fetch, and they don't filter the results the way size or color do.

Stores

Sitewhere.sitesmatchGives
Any online storenalgene.com, keychron.com, tandemceramics.com/shopurl, line, categoryThe platform (Shopify, WooCommerce, Squarespace, BigCommerce or Magento) is found on the first check and read as in its row below. Other stores' product pages are read from schema.org data. Checked at most hourly, like Shopify, since the platform isn't known before the first check
Any Shopify storea /products/ link, as a url matchurl, line, categoryprice, stock, quantity, every option
Any WooCommerce storenalgene.com/product-category/bottlesurl, line, categoryprice (exact), stock, low-stock count, every variation option
Any Squarespace storetandemceramics.com/shop, <name>.squarespace.comurl, line, categoryprice (exact), stock and count per variant, every option. A listing reads 8 products in full per run
Any BigCommerce storestore.com/shop-all (a category page), <name>.mybigcommerce.comurl, line, categoryurl: price and stock per option combination (up to 24). Listing: price and Add to Cart per product. A line watch needs a category page, since robots.txt disallows BigCommerce's search
Any Magento / Adobe Commerce storewww.vitamix.com, or a store view like bulk.com/ukurl, line, categoryprice (exact) and stock per configurable variant, every option, sale and regular price
Any page with schema.org product datasweetwater.comurlprice, stock
Nikenike.comurl, skuprice, stock per size, low stock, release date
Amazonamazon.comurl, sku, exactprice, stock
Running Warehouse, Zappos, Brooks, Saucony, On, adidas, New Balancethe store's hosturl, some sku or exactprice, stock per size

If a store isn't listed, preview one of its product URLs. Most stores run on one of these platforms or publish schema.org data, which we can read without setup.

Marketplaces

Sitewhere.sitesmatchGives
eBayebay.com (or ebay.co.uk, ebay.de, ebay.ca, ebay.com.au), a search link ebay.com/sch/i.html?_nkw=leica+m6&_udhi=1500&LH_Auction=1, or ebay.com/itm/<id>url, line, category, listingListings, newest first (auctions ending soonest first), with exact price (the current bid for auctions), listed date, condition, shipping, seller feedback, image, and ends_at and bids for auctions. No seller names. A plain search is mostly Buy It Now, so set "auction": true in target.attributes for auctions only, or false for Buy It Now only. A search link's price range, format, condition and category go to eBay. Fails with site_unavailable until eBay is connected. No page scraping, since robots.txt disallows /sch/ and item pages answered 403
Discogsdiscogs.com/release/249504 or discogs.com/seller/<name>url, listing, line, categoryRelease: the lowest asking price (USD, exact) and copies for sale. Seller: their newest listings, which need a terms review before use because robots.txt disallows /users/.
Reverbreverb.com, a search link reverb.com/marketplace?query=boss+ce-2&price_max=200&condition=used, or reverb.com/item/<id>url, line, category, listingMusic gear with make, model, exact price, shop, listed date, condition, finish and year. Searches come in Reverb's order, not newest first (robots.txt disallows sorting), so keep them narrow
Etsyetsy.com, a search link etsy.com/search?q=mechanical+keyboard&min=50&max=200, or etsy.com/listing/<id>url, line, category, listingActive listings, newest first, with exact price, quantity, listed date, shop and favorites. Fails with site_unavailable until Etsy is connected
Redditreddit.com/r/Pen_Swap, or reddit.com/r/pens+fountainpensurl, line, categoryNewest posts with title, link, author, posted time, the price from the title or text, and whether it's sold. New posts show up within about 15 minutes. A new subreddit's first check takes about 5 minutes
CraigslistA search link, www.craigslist.org/search/area/sfbay?cat=apa&max_price=3800&min_bedrooms=2, or the older sfbay.craigslist.org/search/apa?...listing, lineOne listing per first-page result, with title, link, exact price and neighborhood, plus beds and baths when stated. Set fetch.accept_site_terms: Craigslist's terms forbid automated searching, and the flag means you take that on. A new listing is items added. One that drops off the first page is removed, which doesn't mean it was taken down. Checked at most hourly, and the server fetches at most 100 Craigslist pages a day

This watcher looks for new Craigslist apartments under $3,800:

JSON
{
  "target": { "query": "www.craigslist.org/search/area/sfbay?cat=apa&max_price=3800&min_bedrooms=2", "match": "listing" },
  "conditions": { "all": [{ "field": "items", "op": "added" }] },
  "where": { "sites": ["www.craigslist.org/search/area/sfbay?cat=apa&max_price=3800&min_bedrooms=2"] },
  "schedule": { "every": "1h", "until": "+30d" },
  "fetch": { "accept_site_terms": true }
}

Auctions

Sitewhere.sitesmatchGives
HiBidhibid.com/lots?q=rolex, or hibid.com/lot/<id>url, line, category, listingThe first page of lots (25), with high bid (or the minimum bid when nobody has bid), bid count, close time, auction house and image. Flagged list_truncated when there are more. Live catalog, webcast and account pages aren't read
Spectrum Winespectrumwine.com, with a queryurl, line, category, listingThe current auction, through the search API robots.txt allows, with current bid (USD), vintage, size, score, estimate and lot link. At most 100 lots per check
BidsquareA catalog or lot link, hakes.com/auctions/... or bidsquare.com/auction-house/...url, line, category, listingOne listing per lot on the page, with the current bid when labeled (otherwise the starting bid, only when nobody has bid), bid count, estimate and image. The item link is the lot's page. /search isn't read. Other pages are read as a store
Hart Davis Harthdhauctions.com, or a link to its .xlsx lot listurl, line, category, listingEvery row of the sale's spreadsheet: wine, producer, bottle size, quantity, low and high estimates. No current bid, and the bid site isn't read
WineBidwinebid.comurl, line, category, listingCurrent lots from the public sitemap, with title, vintage and lot link. No bids, because lot pages are behind a firewall we don't open. Set fetch.accept_site_terms. WineBid asks for a licensing email, which is up to you to send

We don't read Auction Mobility (Hart Davis Hart's bid site), because it loads lots in the browser and has no verified public list.

Cars

Sitewhere.sitesmatchGives
Dealer.com inventoryA dealer's inventory page (hendrickcars.com/new-inventory/index.htm) or one vehicle's page (/new/BMW/2026-BMW-2-Series-abc.htm)url, listingOne product per car on the page, with exact price, VIN, stock number, year, color, new or used, and dealer name. The next page isn't followed (flagged list_truncated), so watch a filtered inventory URL or one vehicle. Other pages are read as a store. No key needed
Xtimeconsumer.xtime.comurl, listingNot supported: the scheduler disallows every path and its homepage is a 404. Refused with site_unavailable
myKaarmaapp.mykaarma.comurl, listingNot supported: the app disallows every path and only has a login page. Refused with site_unavailable

Apartments

Sitewhere.sitesmatchGives
Apartment listings (AppFolio)sdppm.appfolio.com, cbpm.appfolio.com/listings, or a manager's own page with AppFolio's embed (sdppm.com/available.html)url, listingOne listing per unit for rent, with exact monthly rent, beds, baths and sqft as numbers, move-in day as date (a unit available now gets the check's day, Pacific time), and address. attributes.property_group limits it to one AppFolio property group, and an unknown group reads as empty, so check the name. A unit with no rent gets price_missing. Craigslist is under Marketplaces
Apartment unit maps (SightMap)A community page with a sightmap.com unit map (oakrow.com/floorplans), or sightmap.com/embed/<id>url, listingOne listing per available unit, with exact rent (plus total_monthly when fees are added), available_date, beds, baths, sqft, unit number, floor plan, building, floor, lease term and leasing system (yardi_rentcafe, realpage_v2, ...)
RentCaferentcafe.com, securecafe.comurl, listingNot supported: Cloudflare blocks us, robots.txt disallows the unit pages, and the API needs a Yardi key. Watch the community's own floor plans page instead if it has a SightMap unit map. Refused with site_unavailable
Entrata<property>.prospectportal.comurl, listingNot supported: Cloudflare blocks every request, even robots.txt. Refused with site_unavailable

If a page's path is about units for rent (like /floorplans, /available.html or /apartments/...), we read it as an apartment page, or with a learned recipe if it shows neither AppFolio nor SightMap.

Booking and appointments

Booking sites give one slot per open time. To match one, use { "field": "availability", "op": "=", "value": "open_slot" }.

Sitewhere.sitesmatchGives
Cal.comcal.com/peer/15, cal.com/team/cal/intro, acme.cal.com/jane/30minlisting, urlOne slot per open time from from to to (default 14 days) in time_zone (default UTC), with start and offset, duration, host, price, and seats left on seated events. attributes.duration picks a length on events with several
Calendlycalendly.com/<user>/<event>listing, urlOne slot per open start time (default 14 days, up to 93), with seats left for group events, length, location and host. attributes.timezone reads the days in another time zone. Set fetch.accept_site_terms
Acuity Schedulingapp.acuityscheduling.com/schedule.php?owner=26704703, <name>.as.meurl, line, category, listingOne item per appointment type, with name, exact price, duration_minutes, category and service_id. Open times need a listing match. Set fetch.accept_site_terms, for service menus too
Square Appointmentsbook.squareup.com/appointments/<id>/location/<id>/servicesurl, line, category, listingOne item per service, with fixed price, duration_minutes, category, staff and place. Open times need a listing match (service or service_id, staff or staff_id, from, to, up to 62 days) and the browser. Set fetch.accept_site_terms, for service menus too
Jane App clinicsversamc.janeapp.com, versamc.janeapp.com/locations/versa-movement-collective/book, .../book#/staff_member/2/treatment/1, <clinic>.janeapp.co.uklisting, urlOne slot per open time for one visit type at one location, from to to (default about 14 days, at most 28), with start and the clinic's offset, length, practitioner, visit type, location, and price when shown. Set the visit type with treatment_id or treatment, the practitioner with staff_member_id or practitioner (omit for all), and location when there are several. Without a visit type, the check fails and lists the clinic's visit types
SimplePractice (therapy, counseling, small clinics)example-practice.clientsecure.melisting, line, categoryOne slot per open time for the next 7 days (at most 14), with exact start and end and their offset, clinician, service, length, office or video, and listed rate. Narrow it with clinician, service, video and time_zone. Without time_zone, days are picked in New York time (time_zone_default). Only public booking times are read, never client data. Call-to-book services are skipped
Vetstoria vet bookingus.vetstoria.com/booking/<clinic>listing, lineNot read: open times only show after starting a booking, and the terms forbid bots
NexHealthapp.nexhealth.com/appts/<practice>listing, urlOnce reviewed, one slot per open time per provider at one location (location_id, optional provider_id, slot_length). Set fetch.accept_site_terms
Zocdoczocdoc.com/doctor/<name>, zocdoc.com/dentist/<practice>, zocdoc.com/neurologists/phoenix-220419pmlisting, urlOne slot per open time from the availability grids on a public profile or a specialty and city page, with provider, specialty, place and exact start with offset. from and to filter by the appointment's local day. Fails with site_unavailable until Zocdoc is connected. Each request counts against a monthly Bright Data budget we share with Reddit (Server keys). Search, booking and specialty index pages aren't fetched, and only times already in the page are read
Epic MyChart open scheduling<health system>/mychart/SignupAndSchedule/EmbeddedSchedule?id=..&dept=..&vt=..listing, urlNot supported yet: the pages need a browser session, and we haven't read Epic's and the health system's terms. Refused with site_not_reviewed
Freshafresha.com/a/perfect-color-salon-waukegan-1206-golf-road-xkd1xi4gurl, line, categoryOne item per service variant, with exact price (a "from" price goes in price_from with price_type: starts_at), duration_minutes, category, restriction (like "Female only") and place. attributes.service picks one service. Group appointments the page doesn't list are left out, and the check flags partial_data. No open times: Fresha only shows them in a booking cart on the salon's account (and may ask for a reCAPTCHA), so they're off until someone decides carts are acceptable
Booksybooksy.com/en-us/388730_barber-shop-nyc_barber-shop_30067_new-york-cityurl, line, category, listingOne item per service variant, with exact price ("$65.00+" goes in price_from), duration_minutes, category and staff. attributes.staff (a name) or staff_id keeps one person's services. When only schema.org offers can be read (partial_data), the number goes in price_from with price_type: unknown and price is empty. Open times need a listing match with service and fetch.accept_site_terms. The region API can still answer with a captcha (blocked)
Vagarovagaro.com/<business>url, line, category, listingServices with name, price and duration_minutes. Open times need a listing match with service or service_id, up to 7 days. Set fetch.accept_site_terms. time_zone adds an offset to the times
Boulevardjoinblvd.com/b/<business>/widget, blvd.app/@<business>url, line, category, listingNot supported yet: the widget creates a cart on the salon's account as soon as it opens, before showing services or times. Needs a review first
Trusted Traveler interviewsttp.cbp.dhs.gov/5140 (a location id)listingGlobal Entry, NEXUS and SENTRI interview slots with open spots, in the center's local time
Recreation.gov permitsrecreation.gov/permits/233262listing, urlOne slot per entry point per day, with permits left and walk-up flag. attributes.division picks one trailhead. Set fetch.accept_site_terms
Recreation.gov campsitesrecreation.govlistingOpen campsite nights. Set fetch.accept_site_terms

Restaurants

Set attributes.party_size (2 by default) and either date or from/to in the restaurant's local time, at most 14 days per check. time_from and time_to (HH:MM) narrow the hours. Each slot has party_size, seating, local_date, local_time and time_zone.

Sitewhere.sitesmatchGives
Resyresy.com/cities/ny/venues/lilialisting, urlEvery open time for your party, with seating (Dining Room, Bar, Patio), length, deposit and cancellation fee
OpenTableopentable.com/r/carbone-new-yorklisting, urlOpen times near the time you ask for, with seating and table type. Needs fetch.accept_site_terms: true. Fails with site_unavailable until OpenTable is connected
Tockexploretock.com/alinealisting, urlEach time and experience for your party, with tickets left and the prepaid price. Cloudflare blocks us for now

Tickets and events

Sitewhere.sitesmatchGives
Eventbriteeventbrite.com/e/snakehips-twofaced-tickets-1993478754618url, listingLowest and highest price, on sale, sold out or not yet on sale, sales start (opens_at) and end, start time, venue. Ticket classes when Eventbrite is connected
Lumaluma.com/u9l1w54q (an event), luma.com/genai-sf (a calendar), luma.com/sf (a city)url, listing, lineEvents with exact price, open, low, sold out or waitlist, spots left, start time, place. Event pages add each ticket type
Ticketmasterticketmaster.comurl, line, category, listing (with attributes.venue_id or attraction_id)Face-value price range, on-sale state and time (opens_at), start time, venue. Fails with site_unavailable until Ticketmaster is connected
SeatGeekseatgeek.comurl, line, category, listing (with attributes.performer or venue_id)Lowest resale price, listing count, average and median price. Needs fetch.accept_site_terms (its API's robots.txt disallows everything). Fails with site_unavailable until SeatGeek is connected
DICEdice.fm/event/6a0b2ac3c6b6f90001b98470 (an event), dice.fm/browse/london (a city), dice.fm/venue/earth-76a7 (a venue)url, listing, lineEvents with exact price (or the lowest, as price_from, when there are several ticket types), on sale, sold out or announced, waiting list, sale and presale start (opens_at), start time, venue, promoter. Ticket types aren't read (ticket_types_not_read)
Meetupmeetup.com/torontojs (a group's upcoming events), meetup.com/torontojs/events/317276003 (an event)url, listing, lineEvents with the fee and how it's paid, spots left against the limit, going and waitlist counts, full or RSVPs closed, RSVP opening (opens_at, event pages only), start time, venue, group
Titoti.to/clearleft/ux-london-2027 (an event), ti.to/clearleft (an organizer's events)url, listing, lineAn event gives each ticket type with exact price and state (available, low, sold out, waiting list, not yet on sale), tickets left when shown, sale start (opens_at) and end, tax note and event date. A time without a zone keeps only the date, with the clock time in local_start. An organizer gives its upcoming events, without prices. In conditions, give GBP and EUR prices as a bare number
Partifulpartiful.com/e/AWx1DaE9DfUoBmqFTaVUurlPublic events only, with open, low, full or closed, spots left, capacity, waitlist, RSVP deadline, start time, place, guest counts (no names). Partiful's own status decides, so an event cancelled only in its description reads as open
Sessionizesessionize.com/ndc-oslo-2027 (a call for speakers), sessionize.com/api/v2/jl4ktls0 (a published schedule)url, listing, lineA call gives open, upcoming or closed, exact closing time, opening time (opens_at), event days, venue, travel and fee offers. A schedule gives one listing per session with start, end, room, speakers, track
start.ggstart.gg/tournament/genesis-x2, or listing with attributes.videogame_idurl, listing, lineRegistration open or closed, closing time, attendees, start time, venue, and each event's entrants and game. Fails with site_unavailable until start.gg is connected
AXSaxs.comurl, listingNot available. AXS sends our server a bot check, and its API is partners only
See Ticketsseetickets.com, seetickets.usurl, listingNot available. Both send our server a bot check (seetickets.us is now eventim.us)
SignUpGeniussignupgenius.com/go/<id>, or signupgenius.com for the account's active signupsurl, listingOpen slots on a signup you own, with spots left (myqty), item, start time. A bare signupgenius.com lists your active signups, not every slot. Fails with site_unavailable until SignUpGenius is connected. Names, emails and the public /go/ page aren't read
Challongechallonge.comurl, listingNot available. Cloudflare's human check blocks our server, and the API needs a key
Battlefybattlefy.comurl, listingNot available. The API refuses direct access, and the site is an empty app shell

Classes, courts and tee times

Sitewhere.sitesmatchGives
ActiveNet (city recreation)activecommunities.com/denverurl, line, category, listingOne slot per program, with spots left, class size, fee, first session time, place, and opens_at before registration opens. Full, closed and cancelled count as out of stock
CivicRec (Rec1) city recreationsecure.rec1.com/CA/temecula-calisting, line, categoryOne slot per program session, with exact start and the agency's offset, fee, days, ages, place, and opens_at before registration opens. Full, closed and cancelled count as out of stock. Rec1 doesn't publish spots left, so quantity is empty. Pick tabs with tab ("Swim Lessons"). kind: facilities gives one slot per facility and day (from to to, 14 days by default, 31 at most): available, limited or unavailable, with rates. Facility days have the date, not the hours (hours_not_read)
Amilia program registrationapp.amilia.com/store/en/example-orgurl, listingOne slot per activity, with spots left as quantity, class size, start, schedule, ages, waitlist, and price when you set currency. drop_in: true adds a slot per drop-in session with its own spots left. Watch by activity link (url) or activity_ids (listing). Fails with site_unavailable until Amilia is connected (it takes a partner agreement). Not yet checked against a live answer
Mindbodymindbodyonline.com/explore/locations/dear-pilatesurl, line, category, listingOne slot per class from from to to (7 days by default, 14 at most), with spots open, capacity, waitlist, instructor, start time, and opens_at before booking opens
foreUP golf tee timesforeupsoftware.com/index.php/booking/19765/2431listing, urlOne slot per open tee time, with spots left, holes, green fee, cart fee. booking_class, holes and party are optional. Set fetch.accept_site_terms
GolfNow tee timesgolfnow.com/tee-times/facility/1225-audubon-park-golf-clublisting, urlOne slot per open tee time from from to to (3 days by default, 7 at most), with per-player rate, holes, rate name, Hot Deal, transaction fee, and the largest group as quantity. holes (9 or 18), party (1 to 4) and time_zone are optional. Without time_zone, times use the course's offset on the check day, so times past a clock change in the window are an hour off
Chronogolf tee timeschronogolf.com/club/miami-beach-golf-clublisting, urlOne slot per open tee time at every course of the club (3 days by default, 7 at most), with spots left, holes, starting hole, public green fee, deals. A held time is out_of_stock. holes and party are optional. Chronogolf's firewall blocks our tee-time requests for now, so checks report blocked
Mariana Tek class schedulesbarrysbootcamp.marianatek.com?location=10462url, line, category, listingOne slot per class (7 days by default, 14 at most), with spots left (when shown), class size, waitlist, instructors, room, and opens_at before booking opens. Set fetch.accept_site_terms
Playtomic courtsplaytomic.com/clubs/02-padel-sport-academylisting, urlOne slot per free court time (3 days by default, 7 at most), with court, length, price, indoor or outdoor. sport (PADEL or TENNIS), duration, court and indoor are optional. Set fetch.accept_site_terms
CourtReserve court bookingsapp.courtreserve.com/Online/Reservations/Bookings/12345listing, urlNot read yet. The court grid's API isn't verified, and most clubs need a member login
Zen Planner class schedules<gym>.sites.zenplanner.com/calendar.cfmlisting, urlOne slot per class on the public list, with time, spots left, enrolled/capacity. Set fetch.accept_site_terms. time_zone adds an offset to the times
ClassPass studiosclasspass.com/studios/<studio>listing, urlNot available. ClassPass challenges every request from our server, so watch the studio's own booking page (Mindbody, Mariana Tek)
CampMindersystem.campminder.comurl, listingNot available. The portals return an error page and publish no session list (campminder.com is the marketing site)
PetDeskapi.petdesk.comurl, listingNot available. The API answers 403, and no public booking host resolved

Jobs

Each open job is a listing, and a closed one drops out. Use match: line with words from the title.

Sitewhere.sitesmatchGives
Greenhouseboards.greenhouse.io/discordlisting, line, categorytitle, link, date posted, location, department, team, remote, req id
Leverjobs.lever.co/spotifylisting, line, categorytitle, link, date, location, department, team, employment type, remote, salary when published
Ashbyjobs.ashbyhq.com/notionlisting, line, categorytitle, link, date, location, department, team, employment type, remote, salary when published
Workdaynvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSitelisting, line, categorytitle, link, posting date (from "Posted N days ago"), location, remote, req id. The first 100 jobs per run
SmartRecruiterscareers.smartrecruiters.com/BoschGrouplisting, line, categorytitle, link, date, location, department, employment type, remote. Set fetch.accept_site_terms

Software, games and status pages

Sitewhere.sitesmatchGives
GitHub releasesgithub.com/nodejs/nodeurl, lineversion (major), status, date
GitHub commitsgithub.com/nodejs/node/commits/main, github.com/nodejs/node/blob/main/doc/api/fs.md, github.com/nodejs/node/tree/main/liblisting, line, urlNew commits on a branch, or only those touching a file or folder, with sha, title, date, author login, message, verified, parents. Put a branch name with a slash in attributes.branch. A path no commit has touched reads as no commits, so check the path
GitHub repository statsgithub.com/nodejs/node/stargazersurl, listingOne record with stars, forks, watchers, open_issues and size_kb as numbers (so stars >= 10000 or compare_to works), plus default branch, language, license, archived, pushed_at
GitHub issues and pull requestsgithub.com/nodejs/node/issues, github.com/nodejs/node/pullslisting, line, urlThe newest 100 open issues or PRs (closed with attributes.state), with number, author, labels, comments, reactions, thumbs_up, draft, merged_at. Words match the title or labels. attributes.label is sent to GitHub
GitHub security advisoriesgithub.com/advisories/npm/lodash, github.com/advisories/pip/jinja2listing, line, urlReviewed advisories for one package, newest first, with GHSA and CVE ids, severity, cvss_score, vulnerable range, patched version, withdrawn. attributes.affects_version and severity are sent to GitHub. A misspelled package reads as one with no advisories
OSV.devosv.dev/PyPI/jinja2, osv.dev/Go/github.com/gin-gonic/gin, osv.dev/vulnerability/GHSA-35jh-r3h4-6jhmlisting, line, urlEvery known vulnerability for a package in any OSV ecosystem, or one by id, with aliases (CVE, GHSA), fixed versions, severity, cvss_score (from CVSS 3.x), modified. attributes.affects_version keeps those affecting that version. Only the first page is read (partial_results says so). No key needed
NVD (National Vulnerability Database)nvd.nist.gov plus words, nvd.nist.gov plus attributes.cpe, nvd.nist.gov/vuln/detail/CVE-2021-44228listing, line, urlCVEs published in the last 119 days matching the words or CPE (newest 200), or one CVE, with cvss_score, cvss_version, severity, status, CWEs, kev (on CISA's exploited list). Checked every 2 hours at most
npmnpmjs.com/package/react, npmjs.com/package/@types/nodeurl, lineVersions, with the major as version, semver, prereleases (announced), dist-tags, deprecated, publish date
crates.iocrates.io/crates/serdeurl, lineThe 30 newest versions, with semver, yanked, downloads, publish date
Docker Hubhub.docker.com/_/node, hub.docker.com/r/grafana/grafanaurl, lineThe 100 most recently pushed tags (or those in attributes.tag), with digest (watch it with changed), version, size, platforms, push time
PyPIpypi.org/project/requestsurl, lineVersions, prereleases, yanked, upload date. Set fetch.accept_site_terms (robots.txt disallows the JSON API)
Steamstore.steampowered.com/app/1245620url, skuExact price in the location.country currency, price before discount, discount_percent, coming soon, release date
Statuspage (and incident.io)statuspage:www.cloudflarestatus.com, statuspage:status.clerk.comlisting, url, line, categoryOne record per component (status: operational, degraded_performance, partial_outage, major_outage, under_maintenance), plus overall state, open incidents and scheduled maintenances. Many Statuspage-hosted pages (*.statuspage.io, githubstatus.com) disallow /api/ in robots.txt and fail with robots_disallowed
Instatusinstatus:status.zed.dev, railway.instatus.comlisting, url, line, categorySame records and status values as Statuspage

Feeds and posts

Sitewhere.sitesmatchGives
Any RSS, Atom or JSON feedblog.rust-lang.org/feed.xml, example.substack.com, or feed:blog.rust-lang.org for a page that links its feedurl, listing, line, categoryThe newest 100 items, with title, link, published time, author, categories, plain-text summary, and podcast audio file and length
Hacker Newsnews.ycombinator.com, news.ycombinator.com/show, /ask, /jobs, /newest, /user?id=pglisting, line, categoryStories with title, discussion link, posted time, author, points, comments, and the link and its domain
Blueskybsky.app/profile/bsky.app, or bsky.app with words in target.querylisting, line, categoryThe newest 50 posts, with text, link, time, author, likes, reposts, replies
Wikipedia page editsen.wikipedia.org/wiki/Earth (any language)url, listingThe newest 20 revisions, with editor, time, summary, size, bytes changed
YouTubeyoutube.com/@GoogleDevelopers, youtube.com/playlist?list=...url, listing, line, categoryThe latest 15 videos. Set fetch.accept_site_terms (robots.txt disallows the feed)
Stack Exchange questionsThe question link, https://stackoverflow.com/questions/<id>/... (any Stack Exchange site)url, listingA post with score, answer_count, view_count, accepted, accepted_answer_id, closed, closed_reason, tags

Books and streams

Sitewhere.sitesmatchGives
Open Libraryopenlibrary.org/authors/OL7234434A (an author), openlibrary.org/works/OL82563W (a book's editions), openlibrary.org/books/OL62447619M or openlibrary.org/isbn/9781551927282 (one edition)url, listing, lineOne release per work or edition, with title, publish date (exact day when known, else publish_year), publishers, format, pages, languages, ISBN as GTIN, scan id. Use { "field": "items", "op": "added" } for new works or editions. Ids only, since robots.txt disallows search. Reads the first 100 works or 50 editions
Open Library loansarchive.org/loans, with an archive.org/details/<id> link as the url queryurlWhether a scan can be borrowed now, copies free, waitlist size. Set fetch.accept_site_terms
Twitchtwitch.tv/somechannel, or category with a game nameurl, categoryOne profile per channel, with live or offline (in_stock while live, so back_in_stock means it went live), stream title, game, viewers, start time. Needs fetch.accept_site_terms (its API's robots.txt disallows everything). Fails with site_unavailable until Twitch is connected

Markets

Quotes are exact, so $0.000002808 is compared as that, not rounded to a cent.

Sitewhere.sitesmatchGives
DexScreenerdexscreener.com, with a pair or token link https://dexscreener.com/solana/<address> as a url match, or a symbol for lineurl, sku (solana:<address>), lineA quote with USD price, price_change_5m/1h/6h/24h, volumes, liquidity_usd, fdv, market_cap. A symbol search can rank a lookalike token first, so use the address when you can
OpenSeaopensea.io/collection/pudgypenguinsurl, sku (the slug), lineThe collection floor as a quote in its currency, plus volume, sales, owners. Fails with site_unavailable until OpenSea is connected
Magic Eden (Solana NFTs)magiceden.io/marketplace/okay_bearsurl, listing; sku only for a symbol without underscoresThe collection floor as an exact quote in SOL, with listed_count, avg_price_24h and volume_7d (SOL, as numbers), plus the 20 cheapest listings as listing items (exact SOL price, rarity ranks, traits as trait_<type>). listings: false reads only the floor. An unknown symbol reads as an empty collection (nothing_listed), so check it on magiceden.io first. No key needed. Set fetch.accept_site_terms (Magic Eden's API terms limit passing its data on)
Etherscan (wallets and gas)etherscan.io/address/0x..., polygonscan.com/address/0x..., arbiscan.io/address/0x..., etherscan.io/gastrackerlisting, urlAn address gives a record per recent transaction and ERC-20 transfer (50 of each, newest first), with direction (in, out, self), value in the coin or token as a number (value_wei exact), fee, block, method, failed, token symbol, contract and reputation. direction, token and include narrow it. A gas tracker gives a quote in gwei with safe_gwei, propose_gwei, fast_gwei. Free-tier chains only (Ethereum, Polygon, Arbitrum, Linea and others). Base, BNB, Optimism and Avalanche fail with paid_plan_required. A wallet can belong to a private person, so the address you pick is your responsibility. Needs fetch.accept_site_terms, and anything you show must say Powered by Etherscan.io APIs. Fails with site_unavailable until Etherscan is connected
CoinGeckocoingecko.com/en/coins/bitcoinurl, sku (the coin id), lineCoin price in attributes.vs_currency (usd by default), market cap, 24h volume and change. Set fetch.accept_site_terms (robots.txt disallows /api/v3)
Kalshikalshi.com (a market ticker as the sku query), kalshi.com/markets/kxfed/fed-meeting/kxfed-27apr (an event), kalshi.com/markets/kxfed (a series)sku, url, listingOne quote per market. price is the last Yes trade in dollars (0.56 = 56%), empty if never traded. yes_bid, yes_ask, no_bid, no_ask, volume, volume_24h, open_interest and strikes are numbers. No key needed
Polymarketpolymarket.com/event/<event-slug>, polymarket.com/event/<event>/<market>, or polymarket.com with a market slug (sku) or words (line)url, sku, lineOne quote per outcome (just Yes for a Yes/No market), at the exact price in dollars. best_bid, best_ask, spread, last_trade_price, volume, volume_24h, liquidity, price_change_1d and price_change_1w are numbers. No key needed. Checked every 5 minutes at most, since Polymarket caches for 5 minutes
Stock quotes (Finnhub)finnhub.io with a symbol as the sku query (AAPL)skuOne quote per U.S. symbol, with USD price, change, change_percent, open, high, low, previous_close, updated_at. Needs fetch.accept_site_terms and a licence that allows passing quotes on. Fails with site_unavailable until Finnhub is connected
Crypto exchange listings (Coinbase, Kraken)coinbase.com/listings, kraken.com/listings, with currencies as the query ("SOL", "usd")listingA record per trading pair, with pair, base, quote, status (online, auction, post_only, limit_only, cancel_only, disabled, delisted) and trading. A new listing is items added. No prices

Rates and economic data

Sitewhere.sitesmatchGives
U.S. Treasury Fiscal Datafiscaldata.treasury.gov/v2/accounting/od/debt_to_penny, or any Fiscal Data API endpoint, with the API's ?filter= if you want oneurl, lineOne record per row of the newest release (date is its record_date). Numeric columns are numbers under their API names (tot_pub_debt_out_amt, avg_interest_rate_amt, exchange_rate). Pin text columns with target attributes. Releases over one page are read in full, up to 10,000 rows. No key needed
FREDfred.stlouisfed.org/series/UNRATE, or fred.stlouisfed.org with the series id as the sku queryurl, skuOne record per series, with value, previous_value, change, change_percent, period, units, frequency, and date as the newest observation. Fails with site_unavailable until FRED is connected
BLSdata.bls.gov/timeseries/CUUR0000SA0, or bls.gov with the series id as the sku queryurl, skuOne record per series, with value, previous_value, change, change_percent, period_name, and date as the first day of the newest period. Set fetch.accept_site_terms (api.bls.gov's robots.txt disallows everything)
European Central Bankecb.europa.eu with a currency code (line: USD), data.ecb.europa.eu/data/datasets/EXR/EXR.D.USD.EUR.SP00.A, or FLOW.KEY as the sku queryline, url, skuA euro reference rate is a quote with the exact price of one euro in that currency. Other series (deposit rate, €STR) are records. Both have value, previous_value, change, change_percent, unit. A + in the key gives several series at once. No key needed
U.S. economic release calendars (BLS, BEA)bls.gov/schedule, bea.gov/schedule, with words as the query ("consumer price index", "GDP")listingA release for each scheduled release from the past week onward, with date (Eastern time offset), release_date, release_status (announced, then available once the time passes), agency, release, period. A rescheduled release keeps its key, so date changed catches it. Dates only, not the numbers

Government

Sitewhere.sitesmatchGives
SEC EDGAR filingssec.gov/AAPL, sec.gov/cik/320193listing, urlThe newest 100 filings, with form, filing date, archive link and 8-K items
Companies House (UK companies)find-and-update.company-information.service.gov.uk/company/00445790 (add /filing-history or /officers to read only that part)listing, urlA record for the company (status, accounts and confirmation statement due dates, accounts_overdue, has_insolvency_history, SIC codes, filings_total, active_officers), one per filing (form like CS01 or AA, category, description, processed date and PDF link), and one per officer appointment (name, role, appointed_on, resigned_on, active). form: ["AA"] or category: ["accounts"] narrows the filings. Officers' birth dates and addresses are never read. Watching officers means watching private people, which is your call. Fails with site_unavailable until Companies House is connected (Server keys). Not yet checked against a live answer
NHTSA vehicle recallsnhtsa.gov/recalls/honda/accord/2018listing, line, categoryEach recall campaign, with component, summary, consequence, remedy and park-it warnings
CPSC product recallscpsc.gov/recalls, with words in target.querylisting, line, category, urlRecalls from the last 60 days that have every word, with hazard, remedy, units and where they were sold
openFDA drug shortagesopen.fda.gov/drug/shortages, with the drug in target.querylisting, line, category, skuShortage entries per package, with availability, status and reason
openFDA recallsopen.fda.gov/drug/enforcement, /food/enforcement, /device/enforcementlisting, line, categoryEnforcement reports, with class, status, reason, firm and lots
Federal Registerfederalregister.gov, federalregister.gov/agencies/<slug>listing, line, categoryRules, proposed rules and notices, with type, comment deadline and effective date
Grants.govgrants.gov, grants.gov/agencies/HHS-NIH11listing, line, categoryOpen and forecast funding opportunities, with open and close dates
City and county meetings (Legistar)legistar:seattle, oakland.legistar.comlisting, line, categoryUpcoming meetings as records (start time with offset, agenda and minutes status, cancelled, agenda PDF), or with words, matching agenda items from the next 12 agendas (matter file, type, status, action). For a city with no known time zone, set attributes.time_zone, or you get the day only and time_zone_unknown. An unknown client name answers 500 and reads as unreachable
US wildfires (NIFC)wildfire:CA, wildfire:2026-CALPF-002271, wildfire with a locationlisting, line, categoryCurrent incidents, with name, acres, percent_contained, personnel, county, cause, discovered, contained and out dates, and point of origin. attributes.include_rx adds prescribed burns
InciWebinciweblisting, line, categoryIncident team updates. Set fetch.accept_site_terms. wildfire:<state> reads NIFC's open incident data without it
Road work and closures (WZDx)roadwork:WA, roadwork:MN, wzdx:https://<feed>listing, line, categoryWork zones and detours, with road, direction, vehicle_impact, start and end, active and mileposts. Narrow with attributes.closed_only, attributes.road or a location radius (distance_miles). On a feed that marks every impact unknown (WSDOT), closed_only adds impact_unknown. A feed that stopped updating adds feed_stale
AirNow air qualityairnow:94110, or airnow with a locationlistingCurrent AQI per pollutant plus an overall record, and forecasts with category and action_day. Fails with site_unavailable until AirNow is connected. Not yet checked against a live answer
USGS earthquakesearthquake.usgs.gov/m4.5, earthquake.usgs.gov/near/34.05,-118.24/200kmlisting, line, categoryEvents in the last 7 days, with magnitude, depth, alert level and tsunami flag
CourtListenercourtlistener.com/opinions/scotus, courtlistener.com/dockets/candlisting, line, categoryNew opinions or dockets. Fails with site_unavailable until CourtListener is connected
NWS weather alertsweather.gov/alerts/area/KS, weather.gov/alerts/zone/KSZ009listing, line, categoryActive alerts, with event, severity, urgency, onset and expiry. Set fetch.accept_site_terms
USAspending.govusaspending.gov, https://www.usaspending.gov/award/CONT_AWD_.../listing, line, category, urlFederal contract, IDV or grant awards from the last 90 days (days, up to 365), or one award, with the exact amount as the price, recipient, agency, signed date, place of performance, NAICS and PSC. award_type, agency, sub_agency, recipient, state, naics, psc, min_amount and max_amount narrow the search. Defense contracts show up about 90 days late, so use days: 180 or more for DoD. An IDV's amount is often $0 because the money sits on its orders
SAM.govsam.gov, https://sam.gov/opp/<notice id>/viewlisting, line, category, urlContract opportunities posted in the last 30 days, with type, agency, NAICS, set-aside, the response deadline and its offset, award amount and awardee. Fails with site_unavailable until SAM.gov is connected. Checked every 3 hours at most, because a personal key gets 10 requests a day
USAJOBSusajobs.gov, https://www.usajobs.gov/search/results/?k=nurse&l=Denverlisting, line, category, urlOpen federal job announcements, with agency, place, posted and closing dates, salary range and grades. Fails with site_unavailable until USAJOBS is connected. Set fetch.accept_site_terms
NOAA space weather (Kp index)swpc.noaa.gov/kplistingA record per 3-hour step from the last 24 hours onward, with kp (0 to 9, where 5 and up is a storm), status (observed, estimated, predicted) and noaa_scale (G1 to G5)
Hospital price filesuchealth.org/cms-hpt.txt (the hospital's site plus /cms-hpt.txt), or that link as a url matchlisting, urlA record per location from the hospital's CMS index, with mrf_url (the price file), source_page_url, file_type and ein. It catches new locations and moved files, and doesn't read the prices inside them
US House periodic transaction reportsdisclosures-clerk.house.gov/ptr, with a member's name or seat as the query ("Pelosi", "CA11")listingNot available yet, pending a review of the commercial-use limit in 5 U.S.C. §13107(c). Then a record per stock-trade report, with filing date, PDF link, member, state_district, state and district. The trades are inside the PDF

Bills and regulations

Sitewhere.sitesmatchGives
Congress.govcongress.gov (the current Congress), congress.gov/bill/119th-congress/senate-bill, or a bill congress.gov/bill/119th-congress/house-bill/1url, listing, lineOne record per bill, with the bill number, the latest action and its date (with Washington's offset when there's a time), and a status read from that action (introduced, reported, passed, to_executive, became_law, vetoed, failed, in_progress). A single bill adds the public law, sponsor, policy area, and action and cosponsor counts. Lists cover the 250 most recently updated bills, and search words keep titles with every word. Watch latest_action with changed. Fails with site_unavailable until Congress.gov is connected (Server keys)
Open States (state bills)openstates.org/ca (one state), openstates.org with search words (every state), or a bill openstates.org/nc/bills/2025/HB1105url, listing, lineOne record per state bill, with the latest action and its date, latest passage date, status, state, session, chamber and subjects. A single bill adds the latest action's classification, the action count and the primary sponsor. session, chamber and classification narrow the list. 20 bills per check, latest action first. Fails with site_unavailable until Open States is connected. Not yet checked against a live answer
Regulations.govregulations.gov with search words (documents open for comment), regulations.gov/docket/FAA-2026-10694 (everything filed in a docket), or regulations.gov/document/<id>url, listing, lineOne record per rulemaking document, with type (Proposed Rule, Rule, Notice...), docket, agency, posted time, and the exact times comments open and close. open_for_comment is true while comments are open, and comments_open_ended when there's no deadline. A single document adds the page count and PDF link. agency, document_type and open_for_comment narrow the search (use [true, false] for both). Fails with site_unavailable until Regulations.gov is connected

status is our reading of the action's wording, not an official field.

Open data

Sitewhere.sitesmatchGives
Socrata open datasocrata:data.cityofnewyork.us/erm2-nwe9, data.cityofchicago.org/resource/ydr8-5enu.jsonlisting, line, category, urlNew rows of any dataset on a Socrata portal (like 311 requests, permits or WARN notices), newest first, with every column as an attribute, plus date and place.
ArcGIS feature layersarcgis:https://maps2.dcgis.dc.gov/dcgis/rest/services/FEEDS/DCRA/FeatureServer/4listing, line, category, urlNew rows of any public FeatureServer or MapServer layer (like permits, zoning cases or code violations), newest first. A day-only date (like an issue date) stays a day, like 2026-11-02, and other times are exact UTC. Set key_field for layers reloaded every night

Both take these attributes: where (a SoQL or SQL condition), q or search words (Socrata's full-text search), order, date_field, key_field, title_fields, place_field, location_field (Socrata), time_zone (Socrata, to turn the portal's local times into exact ones), and limit (1 to 1000, 200 by default). location.zip with location.radius keeps rows within that distance.

Research

Sitewhere.sitesmatchGives
ClinicalTrials.govclinicaltrials.gov with words in target.query, clinicaltrials.gov/search?cond=glioblastoma, clinicaltrials.gov/study/NCT00083512listing, url, line, categoryStudies, most recently updated first, with status (watch it with changed), recruiting as in_stock, phase, sponsor, enrollment, conditions, interventions, and the nearest site when location.zip and radius are set. condition, sponsor and intervention narrow the search. Set fetch.accept_site_terms, because robots.txt disallows /api/
arXiv category feedsarxiv.org/list/cs.CL, rss.arxiv.org/rss/cs.CL+cs.AIlisting, line, categoryToday's announcement, with title, abstract, authors, categories, announce_type (new, cross, replace) and arxiv_version. Words in target.query keep papers whose title or abstract has every word
arXiv searcharxiv.org with words (and attributes.category), or https://arxiv.org/abs/2611.07928 as a url matchlisting, url, line, categoryThe newest 100 submissions, with abstract, authors, categories, DOI, journal ref and version. Set fetch.accept_site_terms, because robots.txt disallows the API
PubMedpubmed.ncbi.nlm.nih.gov with a search in target.query, pubmed.ncbi.nlm.nih.gov/?term=semaglutidelisting, url, line, categoryPapers added in the last 30 days, with journal, publication date, DOI, PMCID and article types. Set fetch.accept_site_terms, because robots.txt disallows E-utilities. Without it, use Europe PMC
Europe PMCeuropepmc.org with a search in target.query, or https://europepmc.org/article/MED/42958891 as a url matchlisting, url, line, categoryPapers indexed in the last 14 days, including PubMed's (attributes.source: "MED" for PubMed only), with journal, DOI, PMID, open access and cited_by
Semantic Scholar citationssemanticscholar.org/DOI:10.1056/NEJMoa2032183, semanticscholar.org/paper/<id>listing, url, line, categoryThe 100 most recent papers citing it, with venue, year, DOI and citation_count.

Names and domains

Sitewhere.sitesmatchGives
Username and package names (GitHub, Bluesky, npm)handle:github/<name>, handle:bsky/<handle>, handle:npm/<package>listingOne record with available (true when the platform has no such account or package), plus account_type and created_at (GitHub), did (Bluesky) or version (npm). Available means the lookup came back empty, which doesn't guarantee you can register it
Domain registration (RDAP)rdap:example.comlistingavailable (true when the registry has no record), expires, days_to_expiry, EPP status, registrar, nameservers, dnssec, pending_delete and redemption. Reserved and premium names can have no record too. A host under a registered name (www.example.com) fails with rdap_not_registrable. TLDs with no RDAP server (.de, .io) fail with rdap_unsupported_tld, and .uk and .xyz fail with robots_disallowed because their robots.txt blocks us
TLS certificates for a domain (Cert Spotter)certs:example.com (include_subdomains: false for the domain alone)listingA record per unexpired certificate for the domain and its subdomains, with dns_names, issuer, not_before, not_after, wildcard and revoked. A new certificate is items added. Checked hourly at most. Lookalike domains outside yours aren't searched

Any page

For a page nothing above reads, we learn a recipe for it or watch it for changes. Both are looser than the rows above, so check the preview first.

Sitewhere.sitesmatchGives
Any page, read by a learned recipethe page with its path: books.toscrape.com/catalogue/category/books/travel_2/index.htmlurl, listing, line, categoryWhatever the page lists (like products, slots or posts), with title, link, exact price, stock or open and taken, and date. The first preview or spawn learns the recipe, and later runs reuse it
Any page with several pages (learned recipe)quotes.toscrape.com/page/1/listing, line, category, urlEvery item across up to 10 pages. Paging stops at an empty page, a repeated page, a "not found" past the last page, or when the site's own "more" flag says it's done
Any store page (learned recipe), with variantsshop.example.com/p/trail-runnerurl, listing, lineOne result per size or color, each with its own stock, price and SKU, so a size restock works on stores we don't support by platform
Any page with a form (learned recipe), with stepsshop.example.com/stores/pickup plus target.attributes.steps and location.ziplisting, line, urlWhat the page shows after typing a ZIP, picking a party size or clicking. The steps are saved in the recipe and replayed with each watcher's values
Any page, watched for changeschange:https://www.mozilla.org/en-US/about/legal/terms/mozilla/urlOne region with the text, text_hash, length, contains, and what changed last time (last_change, previous_hash, paragraph counts). With seo: true it also has the title, meta description, canonical, robots meta, noindex, h1, script hosts and seo_hash. See Conditions
Any PDF, watched for changeschange:https://www.irs.gov/pub/irs-pdf/fw9.pdfurlOne document with text (the first 2,000 characters), text_hash, pages, pdf_title, pdf_created, pdf_modified, contains, the same last_change, and change_excerpt (the changed lines with context, which a judge condition reads wherever they are in the file). If a link starts returning a web page, it keeps its key and adds format_changed
A list on any pagechange:https://www.cisa.gov/news-events/news with target.attributes.list: "article.c-teaser"urlOne record per item (like a press release, a product or a list entry), keyed by its link or its text, with title, href, position, date from the item's <time>, and contains for each item. Use { "field": "items", "op": "added" } or "removed". After the first read, an empty list is ok and flagged list_empty
Any page, after form steps (browser)change:https://www.weather.gov/ with browser: true, steps and location.zipurlThe page the steps led to, read like any change watch. See Steps and location
Any page, from a location (browser)change:https://www.target.com/p/... with browser: true, cookies, locale, timezone and location.zip or lat/lngurlThe page as a visitor there sees it. Many stores go by IP or their own cookie, not the browser's position, so set the site's location cookie

A page no listed site covers, given with its path (like example.com/book/haircut), is read with a learned recipe. A new page's recipe is learned only at preview or spawn, and a spawn that can't learn it is refused with recipe_not_learned. If the page's layout changes and the recipe fails 3 runs in a row, it's learned again, at most once every 6 hours.

For page changes, set what to read in target.attributes: selector (a CSS selector, the page's main text by default), contains (a phrase or a list), ignore (a CSS selector to drop, like a ticker), strip_noise: false to keep times with seconds, and browser: true for a page that builds its text with scripts.

  • list is a CSS selector for repeated items, one result each (see the row above). It reads at most 500 items, and more is flagged list_truncated.
  • pdf: true reads a link as a PDF. Links ending in .pdf are read as PDFs anyway.
  • seo: true adds the page's tags (see the row above), which is useful for watching your own site.
  • summarize: true adds a one or two sentence summary of the latest change, written by an LLM.

A selector or list that matches nothing on the first read fails with selector_not_found and names close selectors that do match. To wait for an element to appear, watch a larger region with contains. Each change also has change_excerpt, the changed lines with a line of context, which judged conditions read. The last version is stored, so a deploy doesn't lose what changed. PDFs are always fetched directly (never through the proxy), at most one request a second per host, and robots.txt is checked on every redirect.

A url match with any of these attributes, or a link ending in .pdf, is a page-change watch even without the change: prefix. Timestamps, relative times like "5 minutes ago" and long tokens are dropped before hashing, so they don't count as a change.

Steps and location

With browser: true, target.attributes.steps runs steps in the page before we read it, one per string:

StepDoes
fill <selector> => <value>Types the value into a field
select <selector> => <value or label>Picks an option
click <selector>Clicks the element
press <Key>, press <selector> => <Key>Presses Enter, Tab, Escape, an arrow key and a few others
wait <selector>, wait network, wait 2sWaits for an element, for the network to go quiet, or up to 5 seconds

End a step with ? to make it optional. {{name}} is filled from target.attributes, and {{zip}} from location.zip. You get at most 10 steps, 5 seconds each and 25 seconds in all (less when the page loads quicker). Steps never type into password, file or card fields. A step that can't be done fails the check with step_failed, naming the step, and a bad step is refused at spawn with bad_steps. If a captcha guards the form and can't be solved (Blocks), the run is blocked.

Each account gets 200 browser runs with steps and 100 captcha solves a day (previews count), reset at 00:00 UTC. Past that, a run with steps is skipped (its check says usage_limit with the reset time), and a captcha is left unsolved (the check says usage_limit, not blocked). Steps that would send a contact, sign-up, registration, subscription, checkout or application form, or post a form after filling more than one field, are refused at spawn with steps_submit_refused. If only the live page shows that, the step fails with the same reason. Search forms, a single field followed by Enter, and GET forms are allowed.

For location, the browser gets the position (the ZIP's center without lat/lng), locale (like "en-US") and timezone. cookies (up to 5, as name=value) are set on the page's host before it loads, like "GuestLocation=48104|42.270|-83.730|MI|US" on Target.

Blocks

We fetch pages directly, and use a residential proxy only after a store blocks us. Sites that wall bots off on purpose (booking, restaurant, ticket and job sites, and every API above) never go through the proxy, because a block there is the site saying no.

When a page we load in a browser shows reCAPTCHA (v2 or v3) or Cloudflare Turnstile, we solve it through a captcha-solving service and carry on, when captcha solving is connected (Server keys). After a solve, your account keeps that site's cookies for 6 hours, so later checks there usually get through without solving again. If the site walls a check anyway, the cookies are dropped, or replaced if that check solves the captcha. hCaptcha is reported as blocked. If solving is limited to reviewed sites, a captcha anywhere else stays unsolved and the check says captcha_not_allowed.

We never fetch a page whose address is on a private or local network (private_address). If a site can't be read 3 runs in a row (it's blocked, unreachable, or the page or subreddit is gone), we send a blocked event saying why, and its conditions stay unknown until it answers again.

robots.txt

We read each host's robots.txt once a day and don't fetch paths it disallows. A check that needs a disallowed path fails with robots_disallowed, and the failing event goes out on that first run. A host whose robots.txt can't be read (a timeout, a server error, or a bot check in its place) counts as fully disallowed until the next read.

Some sites disallow the paths that hold the data, and a few forbid automated access in their terms. Craigslist's terms forbid automated searching, for one. We turn those sites on one at a time, and watchers on them must set:

JSON
"fetch": { "accept_site_terms": true }

By setting it, you confirm you're allowed to fetch that site under its terms, and that you, not Snag, are responsible for doing so. Snag doesn't read those terms for you.

If preview or spawn can't use a site, it answers 422 with one of these codes:

ErrorMeans
site_unknownThat site can't be read yet
site_unavailableThe site blocks automated access, or needs a key that isn't connected yet. The message says which, and the watcher isn't created
site_cant_matchThe site doesn't support that match mode. The error lists the ones it does
site_not_reviewedNot read yet, because a check would write to someone else's account, a statute restricts it, or support isn't built
site_terms_not_acceptedThe site needs fetch.accept_site_terms

Server keys

Some sites need a key we connect on our side. If a site's key isn't connected yet, creating the watcher fails with site_unavailable, or for an optional key, the site reads less (usually at a lower rate limit). The table shows which key each site uses, and it's the list to set if you run the server yourself.

VariableFor
REDDIT_CLIENT_ID, REDDIT_CLIENT_SECRET, REDDIT_USER_AGENTReddit through its own API (REDDIT_USER_AGENT is optional). Without them, Reddit goes through Bright Data, and without either it isn't read
BRIGHTDATA_API_KEY, BRIGHTDATA_WEB_UNLOCKER_ZONE, BRIGHTDATA_MONTHLY_RECORDSZocdoc (through Bright Data's Web Unlocker, with the zone web_unlocker1 by default) and Reddit when it has no keys of its own. Both draw on one monthly cap, 5,000 by default
RESY_API_KEYResy, optional. Without it, we use the key Resy's widget publishes
BRIGHTDATA_BROWSERAPI_USERNAME, BRIGHTDATA_BROWSERAPI_PASSWORDOpenTable, read in a real browser (the full username and password of a Bright Data Browser API zone)
OPENTABLE_AVAILABILITY_HASHOptional, for OpenTable's legacy direct read. The browser read doesn't need it
EVENTBRITE_TOKENEventbrite ticket classes
TICKETMASTER_API_KEYTicketmaster
SEATGEEK_CLIENT_IDSeatGeek, after its review
OPENSEA_API_KEYOpenSea
COINGECKO_DEMO_API_KEYCoinGecko's higher limit
EBAY_CLIENT_ID, EBAY_CLIENT_SECRETeBay
DISCOGS_TOKENDiscogs' higher limit
ETSY_API_KEY, ETSY_SHARED_SECRETEtsy
OPENFDA_API_KEYopenFDA's higher daily limit
COURTLISTENER_TOKENCourtListener
GITHUB_TOKENA higher limit for every GitHub site (5,000 requests an hour instead of 60, shared), and GitHub name lookups
NVD_API_KEYNVD's higher limit
SOCRATA_APP_TOKENSocrata's higher limit
CONGRESS_API_KEY, REGULATIONS_API_KEYCongress.gov and Regulations.gov (one free api.data.gov key works for both)
OPENSTATES_API_KEYOpen States
SAM_API_KEYSAM.gov
USAJOBS_API_KEY, USAJOBS_USER_AGENT_EMAILUSAJOBS, after its review
FINNHUB_API_KEYStock quotes, after their review
FRED_API_KEYFRED
BLS_API_KEYBLS's higher limit, after its review
NCBI_API_KEYPubMed's higher limit, after its review
SEMANTIC_SCHOLAR_API_KEYSemantic Scholar's own limit
STACKEXCHANGE_KEYStack Exchange's higher daily limit
AIRNOW_API_KEYAirNow air quality
COMPANIES_HOUSE_API_KEYCompanies House
STARTGG_TOKENstart.gg
TWITCH_CLIENT_ID, TWITCH_CLIENT_SECRETTwitch, after its review
ETHERSCAN_API_KEYEtherscan, after its review
AMILIA_API_TOKENAmilia (a partner token from Amilia)
SIGNUPGENIUS_API_KEYSignUpGenius open slots (the account's user key)
OPENROUTER_API_KEY, OPENROUTER_MODEL, WATCHER_LLM_MONTHLY_USD, WATCHER_LLM_OWNER_MONTHLY_USDLearning recipes, judged conditions, page-change summaries and suggestions on failed requests. The server's monthly cap is $20 by default, and each account's is $2
WATCHER_JUDGE_CALLS_PER_DAYHow many judged checks a watcher may make a day (20 by default)
CAPTCHA_PROVIDER, CAPTCHA_API_KEYSolving captchas, with 2captcha, capsolver or anticaptcha
WATCHER_CAPTCHA_UNREVIEWED_HOSTSon (the default) solves captchas on any site, and off solves them only on WATCHER_REVIEWED_HOSTS
WATCHER_REVIEWED_HOSTSReviewed hosts, comma-separated. Steps may submit forms there, and with the switch off, only these hosts get captcha solves
WATCHER_STEP_RUNS_PER_DAY, WATCHER_CAPTCHA_SOLVES_PER_DAYEach account's daily caps on browser runs with steps (200) and captcha solves (100). 0 turns it off
WATCHER_SOLVED_SESSION_HOURSHow long an account reuses a site's cookies after solving a captcha there (6 by default). 0 turns it off