How to export news for data pipelines (JSONL, Parquet)#

To load APITube News API results into a data pipeline, add export=jsonl for line-delimited JSON or export=parquet for a typed columnar Parquet file. Both run on top of any /v1/news/everything search, so the file contains exactly the articles your filters select — ready to stream into a warehouse, lake, or dataframe instead of parsing a paginated JSON response.

curl "https://api.apitube.io/v1/news/everything?title=tesla&export=jsonl&api_key=YOUR_API_KEY" -o results.jsonl

You can send the key as a header (X-API-Key: YOUR_API_KEY) instead of the api_key parameter. The -o flag saves the download to a file, because both formats are served as attachments.

How do I export to JSONL (JSON Lines)?#

Add export=jsonl to your search. The API responds with Content-Type: application/x-ndjson and the filename results.jsonl, and writes one full article object per line, separated by newlines. Each line is the same complete, nested article you get inside the results array of a normal JSON response — nothing is flattened or dropped — so JSONL is the format to use when you want the full structure, not a spreadsheet-style table.

curl "https://api.apitube.io/v1/news/everything?language.code=en&export=jsonl&api_key=YOUR_API_KEY" -o news.jsonl

The alias export=ndjson produces the exact same output. Line-delimited JSON streams cleanly into tools like jq, and loads directly into BigQuery, Snowflake, and most log/ETL systems that expect one JSON record per line.

How do I export to Parquet?#

Use export=parquet to get a compact, columnar binary file. The API responds with Content-Type: application/octet-stream and the filename results.parquet. Unlike JSONL, Parquet uses a fixed, typed schema of 28 columns: text fields are stored as strings, the flags is_duplicate, is_free, and is_breaking as booleans, and read_time, sentences_count, paragraphs_count, words_count, and characters_count as integers.

curl "https://api.apitube.io/v1/news/everything?category.id=medtop:04000000&export=parquet&api_key=YOUR_API_KEY" -o financial_news.parquet

Nested fields — categories, topics, industries, entities, sentiment, media, links, and keywords — are stored as JSON strings inside their column, so the structure survives the conversion. Parquet loads natively into pandas, DuckDB, Apache Spark, and lakehouse engines like Athena, where the typed columns and column pruning make analytical queries fast.

JSONL vs Parquet: which should I use?#

Pick JSONL when you want the full nested article record per line and a format that streams and appends easily — log pipelines, BigQuery/Snowflake line-delimited loads, or quick jq filtering. Pick Parquet when you want a typed, columnar file for analytics and storage — smaller on disk, fast to scan one column at a time, and a first-class citizen in data-lake and dataframe tooling. Both reflect the same filtered search; only the shape on disk differs. For a flat spreadsheet instead, see how to export news to CSV or Excel.

How many articles does one export return?#

Each request exports the current page of results, because the export runs after your normal search and pagination. Use page and per_page to control the file size. One request returns up to 250 records (per_page max 250); asking for more returns HTTP 400 with error ER0171. The default page size is 100.

curl "https://api.apitube.io/v1/news/everything?title=tesla&per_page=250&page=2&export=jsonl&api_key=YOUR_API_KEY" -o news_page2.jsonl

To build a larger dataset, page through the results (page=1, page=2, …) and append each export to your pipeline. For the full rules on page and per_page, see how to paginate through results. Each request — exported or not — counts as a single request against your plan.

Common Questions#

• Which endpoints support JSONL and Parquet export?
• What is the difference between jsonl and ndjson?
• Does the export keep nested fields like entities and sentiment?
• What other export formats are available?

Which endpoints support JSONL and Parquet export?#

The export parameter works on the article-returning news endpoints: /v1/news/everything, /v1/news/top-headlines, /v1/news/story, /v1/news/article, /v1/news/category, /v1/news/topic, /v1/news/industry, and /v1/news/entity. The same filters apply on each. The canonical list is in the official News API parameter reference.

What is the difference between jsonl and ndjson?#

There is no difference in output. export=jsonl and export=ndjson are aliases that produce the identical line-delimited JSON file (Content-Type: application/x-ndjson). Use whichever name your tooling expects.

Does the export keep nested fields like entities and sentiment?#

Yes, but differently per format. JSONL keeps every field as a real nested JSON object on each line. Parquet keeps the same fields, but nested ones (entities, categories, sentiment, media, and similar) are stored as JSON strings inside their column, so you parse that string after loading.

What other export formats are available?#

Alongside jsonl (alias ndjson) and parquet, the export parameter accepts csv, xlsx, tsv, xml, and rss. Omit export, or set export=json, to get the default JSON response.