ApiSkills

Data & ETL · 10 min

Pagination — all four styles

Big datasets arrive in pages. Learn every style and build a "fetch all pages" loop.

APIs never return a million rows at once — they page. There are five common styles, and the same dataset here supports all of them so you can compare.

stylehow to askhow to follow
page-number?page=2&page_size=50increment page
offset?offset=100&limit=50add limit to offset
cursor?cursor=<opaque>follow the relative "next" blindly
page token?paginate=tokenread nextPageToken, resend as ?pageToken=
Link header?paginate=linkread rel="next" in the Link header

Page tokens — the YouTube / Google style

Start a run with ?paginate=token and the response carries a nextPageToken — an opaque string, with NO page number and NO full URL. To get the next page you send it straight back as ?pageToken=<that value>. When the response has no nextPageToken, you have reached the end. Every Google API (YouTube, Gmail, Drive, Ads) works exactly this way, so it is worth practising.

# First page
curl "https://api.ifsjaipur.cloud/playground/data/stocks/candles?paginate=token&page_size=50" -H "x-api-key: YOUR_API_KEY"
# response: { ..., "nextPageToken": "eyJvIjo1MCwibCI6NTB9", "data": [ ... ] }

# Next page — resend that token
curl "https://api.ifsjaipur.cloud/playground/data/stocks/candles?pageToken=eyJvIjo1MCwibCI6NTB9" -H "x-api-key: YOUR_API_KEY"
# keep going until the response has no nextPageToken

The response body always includes a pagination object with total, page info, and both a relative next and an absolute next_url. Cursor pagination is the modern default: you never build the next URL yourself, you just follow the one the server gives you.

curl "https://api.ifsjaipur.cloud/playground/data/stocks/candles?paginate=cursor&page_size=50" -H "x-api-key: YOUR_API_KEY"
# response.pagination.next -> follow it until it is null

Which style should you use?

StyleBest whenWatch out for
CursorLarge or frequently-changing data; the safe default. Stable — no skipped or duplicated rows.You can't jump to an arbitrary page; you must walk from the start.
Link headerThe API is GitHub-style and puts next in the Link header. Same "follow the server" model as cursor.Harder to read in some tools (e.g. Power Query can't access response headers).
Page-numberSmall/medium, mostly-static data; or a UI that shows page numbers.If rows are inserted/deleted between requests you can skip or duplicate rows.
Offset/limitYou need to jump to an arbitrary position (offset). Simple maths.Same drift problem as page-number on changing data; slow on huge offsets in real DBs.

Rule of thumb: if the API offers a cursor or a next link, follow it. Reach for page-number/offset only when the API has nothing better, or when a human needs to pick a page.

Tip · In n8n's HTTP Request node, turn on Pagination → "Response Contains Next URL", set the next URL to {{ $response.body.pagination.next_url }} (the ABSOLUTE URL — n8n needs a full URL, not the relative one), and complete when it is null. For the page-token style, instead use "Update a Parameter in Each Request": set the pageToken query param to {{ $response.body.nextPageToken }} and complete when {{ !$response.body.nextPageToken }}. The Cookbook has a ready workflow for every style.