PDF Blocks
PricingSupport
Start Free
Go to Page

Working with large files

Stream uploads and downloads, set sensible timeouts, retry transient failures, and stay robust when processing large PDFs.

Large PDFs are the same request as small ones — one file in, one file out — but the naive approach of reading the whole document into memory and waiting on a default timeout falls over as files grow. Stream the upload and the download, give the request room to finish, and retry transient failures. Because the API is stateless, a request that fails leaves nothing behind, so retries are safe.

Stream, don’t buffer

Read the input from disk in chunks as you send it, and write the response to disk in chunks as it arrives. Never hold the entire document in memory. Every example below streams both directions and watermarks big.pdf as a stand-in for any single-output action.

cURLbash
# curl streams the upload from disk and the response to disk by default.
curl https://api.pdfblocks.com/v1/add_text_watermark \
  -H 'X-API-Key: your_api_key' \
  -F file=@big.pdf \
  -F line_1='CONFIDENTIAL' \
  --max-time 600 \
  --retry 3 --retry-delay 2 --retry-all-errors \
  -o watermarked.pdf
Pythonpython
# pip install requests requests-toolbelt
import requests
from requests_toolbelt.multipart.encoder import MultipartEncoder

with open('big.pdf', 'rb') as file:
    form = MultipartEncoder(fields={
        'file': ('big.pdf', file, 'application/pdf'),  # streamed from disk
        'line_1': 'CONFIDENTIAL',
    })
    with requests.post(
        'https://api.pdfblocks.com/v1/add_text_watermark',
        headers={'X-API-Key': 'your_api_key', 'Content-Type': form.content_type},
        data=form,
        stream=True,
        timeout=(10, 600),  # 10s to connect, 600s to read
    ) as response:
        response.raise_for_status()
        with open('watermarked.pdf', 'wb') as output:
            for chunk in response.iter_content(chunk_size=65536):
                output.write(chunk)
Node.jsjavascript
// npm install form-data
import { createReadStream, createWriteStream } from 'node:fs';
import { request } from 'node:https';
import FormData from 'form-data';

const form = new FormData();
form.append('file', createReadStream('big.pdf')); // streamed from disk
form.append('line_1', 'CONFIDENTIAL');

const req = request(
  'https://api.pdfblocks.com/v1/add_text_watermark',
  {
    method: 'POST',
    headers: { 'X-API-Key': 'your_api_key', ...form.getHeaders() },
    timeout: 600000,
  },
  (response) => {
    if (response.statusCode !== 200) {
      throw new Error(`Request failed: ${response.statusCode}`);
    }
    response.pipe(createWriteStream('watermarked.pdf')); // streamed to disk
  },
);

form.pipe(req); // stream the multipart body
Gogo
package main

import (
	"io"
	"mime/multipart"
	"net/http"
	"os"
	"time"
)

func main() {
	// Generate the multipart body on the fly with io.Pipe — the file is never
	// held in memory in full.
	pr, pw := io.Pipe()
	form := multipart.NewWriter(pw)

	go func() {
		defer pw.Close()
		defer form.Close()
		file, _ := os.Open("big.pdf")
		defer file.Close()
		part, _ := form.CreateFormFile("file", "big.pdf")
		io.Copy(part, file) // streamed, not buffered
		form.WriteField("line_1", "CONFIDENTIAL")
	}()

	req, _ := http.NewRequest("POST",
		"https://api.pdfblocks.com/v1/add_text_watermark", pr)
	req.Header.Set("Content-Type", form.FormDataContentType())
	req.Header.Set("X-API-Key", "your_api_key")

	client := &http.Client{Timeout: 10 * time.Minute}
	res, err := client.Do(req)
	if err != nil {
		panic(err)
	}
	defer res.Body.Close()

	out, _ := os.Create("watermarked.pdf")
	defer out.Close()
	io.Copy(out, res.Body) // streamed to disk
}
C#csharp
using var client = new HttpClient { Timeout = TimeSpan.FromMinutes(10) };
client.DefaultRequestHeaders.Add("X-API-Key", "your_api_key");

using var fileStream = File.OpenRead("big.pdf");
using var form = new MultipartFormDataContent
{
    { new StreamContent(fileStream), "file", "big.pdf" }, // streamed from disk
    { new StringContent("CONFIDENTIAL"), "line_1" },
};

// ResponseHeadersRead returns as soon as the headers arrive, so the body isn't
// buffered before you read it.
using var response = await client.PostAsync(
    "https://api.pdfblocks.com/v1/add_text_watermark",
    form,
    HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();

await using var output = File.Create("watermarked.pdf");
await response.Content.CopyToAsync(output); // streamed to disk

Set generous timeouts

A large document takes longer to upload and to process, so a default client timeout — often 30 seconds or less — will cut a healthy request short. Split the budget:

  • A short connect timeout (a few seconds) fails fast when the host is unreachable.
  • A long read timeout gives processing time to finish. The examples above use ten minutes; size it to your largest expected document.

Don’t set an unbounded timeout — you still want a stuck connection to give up eventually so a retry can take over.

Retry transient failures

Network resets, connection timeouts, and (once enforced) 429 or 5xx responses are transient — the same request may well succeed a moment later. Retry with exponential backoff and a little jitter. Retrying is safe here precisely because the API is stateless: each call is a pure function of its inputs, with no partial writes to undo.

Pythonpython
import random
import time
import requests


def send_with_retries(build_request, attempts=4):
    for attempt in range(attempts):
        try:
            response = build_request()
            # Retry only on server-side / throttling responses.
            if response.status_code < 500 and response.status_code != 429:
                return response
        except requests.RequestException:
            if attempt == attempts - 1:
                raise
        # Back off: 1s, 2s, 4s, … plus jitter.
        time.sleep(2 ** attempt + random.uniform(0, 1))
    return response

Don’t retry a 400 — a validation error won’t fix itself, and the application/problem+json body tells you which field to correct. See Errors for the catalog. From the command line, cURL has this built in: --retry 3 --retry-delay 2 --retry-all-errors.

Size and rate limits

Request-size limits, rate limits, and their 413, 429, and related status codes are a reserved part of the API contract and are not enforced yet. No fixed thresholds are published — design for the contract rather than a specific number.

To stay robust as limits come online, keep individual requests to a sensible size, handle 413 and 429 as retryable-with-backoff (or, for 413, as a signal to split the input), and honor a Retry-After header when present. See Rate limits & usage for how usage is metered.

Split very large files first

If a single document is too large or too slow to process in one call, break it into smaller parts, process each independently, and — if you need one file back — merge the results. This caps the memory and time any single request needs, and lets you process the parts in parallel.

Split into parts

Use Split by file size or Split by page count to produce manageable chunks. See Splitting a PDF for how to unpack the output.

Process each part

Run your action on each part — in parallel if you like, since the calls are independent.

Merge if needed

Recombine the processed parts with Merge PDF documents.

Rate limits & usage

How usage is metered and the reserved limit contract.

Errors

Status codes, the problem+json shape, and what to retry.

Splitting a PDF

Break a huge document into smaller parts first.

Chaining actions

Stream a multi-step pipeline without temp files.