Guide: catalog crawler

Goal: index or monitor products across every Artos store — for a price tracker, a shopping search engine, a comparison feed, or to ground an assistant’s product knowledge. Read-only: no cart, no checkout, no payments.

Who this is for: you ingest the catalog and surface it elsewhere. If you also need to buy, layer the Shopping agent (SDK) guide on top.

Catalog discovery runs at the anonymous trust tier — a platform key is not strictly required. Provide one (and your profile URL) for production stability; see Platform API keys.

The endpoints

Cross-store catalog is three global operations (no /s/:slug):

Each product carries metadata.artos_seller (slug, name, endpoint) so you can attribute every row to its store. See Catalog for the filter vocabulary and seller routing.

Crawl with the CLI

The CLI is the quickest way to pull catalog data — no code:

# Search the cross-store catalog (price in major units / dollars).
artos catalog search "running shoes" --max 150 --currency USD --json
 
# Resolve specific products globally (anonymous — no key needed).
artos product get --id product_123 --json
artos product lookup product_123 product_456 --json

Add --json to every command for machine-readable output you can pipe into your pipeline. Set UCP_PLATFORM_API_KEY and PLATFORM_PROFILE_URL (or pass --key / --profile) to crawl as an identified platform.

Crawl with the SDK

For a real ingestion loop, drive the SDK directly. The response’s pagination block is { has_next_page, cursor, total_count } — request the next page with cursor while has_next_page is true:

import { UcpClient } from '@artos-commerce/ucp-client';
 
const client = new UcpClient({
  artosBaseUrl: 'https://api.artos.sh',
  platformApiKey: process.env.UCP_PLATFORM_API_KEY!, // optional, but recommended
  platformProfileUrl: 'https://your-app.example/.well-known/ucp',
});
 
interface SearchPage {
  products?: Array<{
    id: string;
    title: string;
    metadata?: { artos_seller?: { slug?: string } };
    variants?: Array<{ price?: number }>;
  }>;
  pagination?: { has_next_page?: boolean; cursor?: string; total_count?: number };
}
 
async function* crawl(query: string) {
  let cursor: string | undefined;
  do {
    const page = (await client.globalSearch({
      query,
      // Filters take MAJOR units (dollars); the SDK converts to minor units.
      filters: { price: { max: 500, currency: 'USD' } },
      pagination: { limit: 50, ...(cursor ? { cursor } : {}) },
    })) as SearchPage;
    for (const product of page.products ?? []) {
      yield {
        id: product.id,
        title: product.title,
        seller: product.metadata?.artos_seller?.slug,
        price: product.variants?.[0]?.price, // integer minor units
      };
    }
    cursor = page.pagination?.has_next_page ? page.pagination.cursor : undefined;
  } while (cursor);
}
 
for await (const row of crawl('running shoes')) {
  console.log(row.seller, row.title);
}

globalSearch needs a query and/or filters — calling with only pagination returns nothing. Read facets.categories / facets.sellers off the response to drive faceted re-crawls.

Substitute real values

Replace product_123, product_456, and YOUR_STORE_SLUG with live ids from your own search results (metadata.artos_seller.slug). The illustrative ids in these snippets will not resolve on a real API.

Be a good citizen

• Throttle. Page with cursor rather than hammering large limits; keep a modest concurrency.
• Cache by id. Use lookup_catalog / get_product to refresh known products instead of re-searching everything.
• Money is integer minor units on the wire — never parse prices as floats.

Next steps

• Catalog — full filter vocabulary, facets, multi-store routing
• Platform API keys — when and how to get one
• Shopping agent (SDK) — add checkout on top