# Working with large files

Stream uploads and downloads, set sensible timeouts, retry transient failures, and stay robust when processing large PDFs.

Large PDFs are the same request as small ones — one file in, one file out — but
the naive approach of reading the whole document into memory and waiting on a
default timeout falls over as files grow. Stream the upload and the download,
give the request room to finish, and retry transient failures. Because the API
is stateless, a request that fails leaves nothing behind, so retries are safe.

## Stream, don't buffer

Read the input from disk in chunks as you send it, and write the response to
disk in chunks as it arrives. Never hold the entire document in memory. Every
example below streams both directions and watermarks `big.pdf` as a stand-in for
any single-output action.

<CodeGroup>

```bash title="cURL"
# curl streams the upload from disk and the response to disk by default.
curl https://api.pdfblocks.com/v1/add_text_watermark \
  -H 'X-API-Key: your_api_key' \
  -F file=@big.pdf \
  -F line_1='CONFIDENTIAL' \
  --max-time 600 \
  --retry 3 --retry-delay 2 --retry-all-errors \
  -o watermarked.pdf
```

```python title="Python"
# pip install requests requests-toolbelt
import requests
from requests_toolbelt.multipart.encoder import MultipartEncoder

with open('big.pdf', 'rb') as file:
    form = MultipartEncoder(fields={
        'file': ('big.pdf', file, 'application/pdf'),  # streamed from disk
        'line_1': 'CONFIDENTIAL',
    })
    with requests.post(
        'https://api.pdfblocks.com/v1/add_text_watermark',
        headers={'X-API-Key': 'your_api_key', 'Content-Type': form.content_type},
        data=form,
        stream=True,
        timeout=(10, 600),  # 10s to connect, 600s to read
    ) as response:
        response.raise_for_status()
        with open('watermarked.pdf', 'wb') as output:
            for chunk in response.iter_content(chunk_size=65536):
                output.write(chunk)
```

```javascript title="Node.js"
// npm install form-data
import { createReadStream, createWriteStream } from 'node:fs';
import { request } from 'node:https';
import FormData from 'form-data';

const form = new FormData();
form.append('file', createReadStream('big.pdf')); // streamed from disk
form.append('line_1', 'CONFIDENTIAL');

const req = request(
  'https://api.pdfblocks.com/v1/add_text_watermark',
  {
    method: 'POST',
    headers: { 'X-API-Key': 'your_api_key', ...form.getHeaders() },
    timeout: 600000,
  },
  (response) => {
    if (response.statusCode !== 200) {
      throw new Error(`Request failed: ${response.statusCode}`);
    }
    response.pipe(createWriteStream('watermarked.pdf')); // streamed to disk
  },
);

form.pipe(req); // stream the multipart body
```

```go title="Go"
package main

import (
	"io"
	"mime/multipart"
	"net/http"
	"os"
	"time"
)

func main() {
	// Generate the multipart body on the fly with io.Pipe — the file is never
	// held in memory in full.
	pr, pw := io.Pipe()
	form := multipart.NewWriter(pw)

	go func() {
		defer pw.Close()
		defer form.Close()
		file, _ := os.Open("big.pdf")
		defer file.Close()
		part, _ := form.CreateFormFile("file", "big.pdf")
		io.Copy(part, file) // streamed, not buffered
		form.WriteField("line_1", "CONFIDENTIAL")
	}()

	req, _ := http.NewRequest("POST",
		"https://api.pdfblocks.com/v1/add_text_watermark", pr)
	req.Header.Set("Content-Type", form.FormDataContentType())
	req.Header.Set("X-API-Key", "your_api_key")

	client := &http.Client{Timeout: 10 * time.Minute}
	res, err := client.Do(req)
	if err != nil {
		panic(err)
	}
	defer res.Body.Close()

	out, _ := os.Create("watermarked.pdf")
	defer out.Close()
	io.Copy(out, res.Body) // streamed to disk
}
```

```csharp title="C#"
using var client = new HttpClient { Timeout = TimeSpan.FromMinutes(10) };
client.DefaultRequestHeaders.Add("X-API-Key", "your_api_key");

using var fileStream = File.OpenRead("big.pdf");
using var form = new MultipartFormDataContent
{
    { new StreamContent(fileStream), "file", "big.pdf" }, // streamed from disk
    { new StringContent("CONFIDENTIAL"), "line_1" },
};

// ResponseHeadersRead returns as soon as the headers arrive, so the body isn't
// buffered before you read it.
using var response = await client.PostAsync(
    "https://api.pdfblocks.com/v1/add_text_watermark",
    form,
    HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();

await using var output = File.Create("watermarked.pdf");
await response.Content.CopyToAsync(output); // streamed to disk
```

</CodeGroup>

## Set generous timeouts

A large document takes longer to upload and to process, so a default client
timeout — often 30 seconds or less — will cut a healthy request short. Split the
budget:

- A short **connect** timeout (a few seconds) fails fast when the host is
  unreachable.
- A long **read** timeout gives processing time to finish. The examples above use
  ten minutes; size it to your largest expected document.

Don't set an unbounded timeout — you still want a stuck connection to give up
eventually so a retry can take over.

## Retry transient failures

Network resets, connection timeouts, and (once enforced) `429` or `5xx`
responses are transient — the same request may well succeed a moment later.
Retry with exponential backoff and a little jitter. Retrying is safe here
precisely because the API is stateless: each call is a pure function of its
inputs, with no partial writes to undo.

```python title="Python"
import random
import time
import requests


def send_with_retries(build_request, attempts=4):
    for attempt in range(attempts):
        try:
            response = build_request()
            # Retry only on server-side / throttling responses.
            if response.status_code < 500 and response.status_code != 429:
                return response
        except requests.RequestException:
            if attempt == attempts - 1:
                raise
        # Back off: 1s, 2s, 4s, … plus jitter.
        time.sleep(2 ** attempt + random.uniform(0, 1))
    return response
```

Don't retry a `400` — a validation error won't fix itself, and the
`application/problem+json` body tells you which field to correct. See
[Errors](/docs/api/errors) for the catalog. From the command line, cURL has this
built in: `--retry 3 --retry-delay 2 --retry-all-errors`.

## Size and rate limits

<Note>
  Request-size limits, rate limits, and their `413`, `429`, and related status
  codes are a reserved part of the API contract and are **not enforced yet**. No
  fixed thresholds are published — design for the contract rather than a specific
  number.
</Note>

To stay robust as limits come online, keep individual requests to a sensible
size, handle `413` and `429` as retryable-with-backoff (or, for `413`, as a
signal to split the input), and honor a `Retry-After` header when present.
See [Rate limits & usage](/docs/api/rate-limits-and-usage) for how usage is
metered.

## Split very large files first

If a single document is too large or too slow to process in one call, break it
into smaller parts, process each independently, and — if you need one file back
— merge the results. This caps the memory and time any single request needs, and
lets you process the parts in parallel.

<Steps>

<Step title="Split into parts">
  Use [Split by file size](/docs/api/split-pdf-by-file-size) or [Split by page
  count](/docs/api/split-pdf-by-page-count) to produce manageable chunks. See
  [Splitting a PDF](/docs/api/splitting-a-pdf) for how to unpack the output.
</Step>

<Step title="Process each part">
  Run your action on each part — in parallel if you like, since the calls are
  independent.
</Step>

<Step title="Merge if needed">
  Recombine the processed parts with [Merge PDF
  documents](/docs/api/merge-pdf-documents).
</Step>

</Steps>

## Related

<CardGroup cols={2}>

<Card title="Rate limits & usage" href="/docs/api/rate-limits-and-usage">
  How usage is metered and the reserved limit contract.
</Card>

<Card title="Errors" href="/docs/api/errors">
  Status codes, the problem+json shape, and what to retry.
</Card>

<Card title="Splitting a PDF" href="/docs/api/splitting-a-pdf">
  Break a huge document into smaller parts first.
</Card>

<Card title="Chaining actions" href="/docs/api/chaining-actions">
  Stream a multi-step pipeline without temp files.
</Card>

</CardGroup>