How data scientists and researchers use APITube#

Data scientists use the APITube News API to build labeled news datasets without scraping or hand-annotating. The workflow is four steps: size the set with GET /v1/news/count, check label balance with the facet parameter, pin a reproducible date window with published_at.start and published_at.end, then pull the corpus out in bulk with export=jsonl or export=parquet, paging through per_page up to 250 records at a time. Every article already ships with structured NLP labels — sentiment, entities, categories, topics and keywords — so the corpus is labeled the moment you download it.

curl "https://api.apitube.io/v1/news/count?language.code=en&published_at.start=2026-01-01&published_at.end=2026-03-31&api_key=YOUR_API_KEY"

You can send the key as a header (X-API-Key: YOUR_API_KEY) instead of the api_key query parameter. Start with the count, because it tells you how big the download will be before you commit to it.

How do I size a news dataset before downloading it?#

Call GET /v1/news/count with the exact filters your dataset needs. It runs your query and returns only the total — no article bodies — so you learn the row count for one request instead of paging through everything:

{ "status": "ok", "count": 48210, "request_id": "..." }

A count call charges one request against your plan regardless of how large the total is, which makes it the cheap way to plan a pull. Two caveats matter for research: on a test key the count is capped at 100 so it cannot reveal the real catalog size, and an unfiltered count returns a catalog-wide figure rather than your dataset — always scope it with the same filters (language, date, category, entity) you will export. See counting matching articles without downloading them for the full endpoint behavior.

What labels come with each article?#

Each article record is pre-annotated, so you rarely need a separate labeling step. Alongside the text (title, body, and readability counts like read_time, sentences_count, paragraphs_count, words_count, characters_count), every record carries nested NLP fields: sentiment (overall, title and body polarity plus a score from -1 to 1), entities (people, organizations, brands, locations with per-entity sentiment), categories, topics, industries, keywords, media, and the source with its rank. The boolean flags is_duplicate, is_free and is_breaking are included too.

That is the difference between APITube and a raw scrape: sentiment classification, named-entity recognition and topic labels are already attached, so you can train or evaluate on them directly instead of running your own annotation pipeline.

How do I check class balance before I train?#

Skewed classes wreck a model, so inspect the label distribution before downloading. The facet parameter groups the whole matching set by a field and returns counts, without pulling articles. Facet by the labels you plan to model — sentiment polarity, category, language, source country — in a single request:

curl "https://api.apitube.io/v1/news/everything?published_at.start=2026-01-01&published_at.end=2026-03-31&facet=true&facet.field=sentiment.overall.polarity,category.id,language.id&per_page=1&api_key=YOUR_API_KEY"

The response adds a facets object where each field is an array of { value, count } pairs computed over every matching article, not just the page. You can now see, for example, that negative articles are a fifth of the positive ones and adjust your sampling before you spend requests downloading. Faceting accepts up to five fields per call and its own facet.limit (default 10, cap 100); the faceting guide lists every groupable field.

How do I build a reproducible corpus?#

Reproducibility comes from a closed, absolute date window. Set both published_at.start and published_at.end to fixed dates in the past (YYYY-MM-DD, or a full timestamp), rather than a relative value like NOW-30DAYS that moves every run. Because a past window stops matching newly published articles, the same filters return the same corpus each time you run the pull:

curl "https://api.apitube.io/v1/news/everything?published_at.start=2026-01-01&published_at.end=2026-01-31&language.code=en&category.id=medtop:04000000&per_page=250&api_key=YOUR_API_KEY"

Add language.code to fix the language of the set — repeat the pull per language for a multilingual corpus — and layer any classification filter (category.id, topic.id, industry.id, entity.id) to scope the domain. Record the exact query string alongside your dataset so the corpus can be regenerated or extended later with the same parameters.

How do I pull the corpus out in bulk?#

Add export=jsonl for one full article object per line, or export=parquet for a typed columnar file that loads straight into pandas, DuckDB or Spark. The export runs on top of your search, so the file contains exactly the filtered articles:

curl "https://api.apitube.io/v1/news/everything?published_at.start=2026-01-01&published_at.end=2026-01-31&language.code=en&per_page=250&page=1&export=jsonl&api_key=YOUR_API_KEY" -o corpus_p1.jsonl

Each request exports the current page, up to 250 records (per_page max is 250; asking for more returns HTTP 400 ER0171), so a large dataset is a loop over page=1, page=2, … appended together. To trim each record to the columns your model needs, add fl with a comma-separated field list — for example fl=id,title,sentiment,entities,categories — which slims both the JSON response and the export. The data-pipeline export guide covers the JSONL and Parquet formats and page-by-page downloads in detail.

Common Questions#

• Can I download more than 250 articles in one request?
• Are the article bodies full text for training?
• Does building a dataset use a lot of requests?

Can I download more than 250 articles in one request?#

No. per_page is capped at 250, and a higher value returns HTTP 400 with error ER0171. Build a bigger dataset by paging: keep per_page=250 and increment page until a page comes back short. One more limit to plan around — the Free plan can only reach the first 5 pages (error ER0173 beyond that), so a large corpus needs a paid plan that lifts the pagination cap.

Are the article bodies full text for training?#

Only on a live, paid key. On a Free plan or a test key the article body is truncated to a preview and masked with an upgrade marker — the same content masking applied across the News API — while the labels, entities and sentiment are still returned. For a training corpus that needs the complete text, use a live key on a paid plan.

Does building a dataset use a lot of requests?#

Each API call is a single request against your plan, and exporting does not cost extra — an exported page counts the same as a normal search page. Sizing with count and checking balance with facet are one request each, so the planning phase is cheap; the bulk cost is the download itself, which is one request per 250-record page. For the exact parameter names and limits, the News API parameters reference is canonical.