# requests

> Send HTTP requests and read responses as text, typed JSON, lines or server-sent events, with timeouts and honest error handling.

This is the Markdown version of https://osl-lang.org/packages/requests. Every page of the OSL docs has one: add .md to its path. https://osl-lang.org/llms.txt lists them all.

```osl
import "std:requests"
```

`std:requests` is the HTTP client. Use it to call web APIs, download pages, post data and read streaming responses. It is the counterpart of [serve](/packages/serve.md), and the examples here call a small server they start themselves on `127.0.0.1`, so every one runs as written. Replace the local address with a real URL in your own code.

## A first request

`requests.get(url)` returns a result. On success you get a response with `status`, `ok`, `statusText`, `url`, `headers` and `body`:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/hello", def(*serve.Context c) (
  c.setHeader("X-Served-By", "osl")
  c.text(200, "Hello, client")
))
app.start("127.0.0.1:8790").unwrap()

auto reply = requests.get("http://127.0.0.1:8790/hello").unwrap()
log reply.status reply.statusText
log reply.ok
log reply.headers["X-Served-By"]
log reply.body

app.stop().unwrap()
```

Output:

```text
200 OK
true
osl
Hello, client
```

Try a path the server doesn't have, such as `/nothing`, and look at `status` and `ok`.

Header names use their canonical capitalisation, such as `Content-Type` and `X-Served-By`, whatever the server sent.

## Other methods

Each HTTP method has a function: `get`, `post`, `put`, `patch`, `delete`, `head` and `options`. All take a URL and an optional options object. The `json` option encodes a value as the request body and sets `Content-Type: application/json`. The `body` option sends a string as it is:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.post("/items", def(*serve.Context c) (
  auto item = c.bodyJSON().unwrap()
  c.json(201, {id: 1, name: item["name"]})
))
app.put("/items/:id", def(*serve.Context c) (
  c.text(200, "replaced " ++ c.param("id") ++ " with " ++ c.body())
))
app.delete("/items/:id", def(*serve.Context c) (
  c.noContent()
))
app.start("127.0.0.1:8791").unwrap()

auto base = "http://127.0.0.1:8791/items"
auto made = requests.post(base, {json: {name: "lamp"}}).unwrap()
log made.status made.body.trim()
auto put = requests.put(base ++ "/1", {body: "plain text"}).unwrap()
log put.status put.body
auto gone = requests.delete(base ++ "/1").unwrap()
log gone.status gone.body.len

app.stop().unwrap()
```

Output:

```text
201 {"id":1,"name":"lamp"}
200 replaced 1 with plain text
204 0
```

## Query strings and headers

The `query` option adds parameters to the URL and escapes them. `headers` sets request headers, which is where tokens go:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/search", def(*serve.Context c) (
  c.json(200, {
    q: c.query("q"),
    auth: c.bearer(),
    agent: c.userAgent()
  })
))
app.start("127.0.0.1:8792").unwrap()

auto reply = requests.get("http://127.0.0.1:8792/search", {
  query: {q: "hello world, osl"},
  headers: {Authorization: "Bearer abc123", "User-Agent": "docs/1.0"}
}).unwrap()
log reply.url
log reply.body.trim()

app.stop().unwrap()
```

Output:

```text
http://127.0.0.1:8792/search?q=hello+world%2C+osl
{"agent":"docs/1.0","auth":"abc123","q":"hello world, osl"}
```

## Typed JSON responses

Most APIs answer with JSON. `.parse<T>()` on a response (or on the result of a request) parses the body and checks it against a type, so a wrong shape is an error and not a surprise later. It needs `std:json` no more than you do:

```osl
import "std:serve"
import "std:requests"

struct User (
  string name = ""
  int age = 0
)

auto app = serve.new()
app.get("/user", def(*serve.Context c) (
  c.json(200, {name: "Ada", age: 36})
))
app.get("/broken", def(*serve.Context c) (
  c.json(200, {name: "Ada", age: "old"})
))
app.start("127.0.0.1:8793").unwrap()

auto base = "http://127.0.0.1:8793"
User user = requests.get(base ++ "/user").parse<User>().unwrap()
log user.name user.age

auto bad = requests.get(base ++ "/broken").parse<User>()
log bad.isErr() bad.unwrapErr().kind

app.stop().unwrap()
```

Output:

```text
Ada 36
true ParseError
```

`parse` sits on the request's result, so it also passes a failed request through: one `unwrap` or `catch` at the end covers both the network and the parsing.

## A bad status is not a failure

A request fails only when no response arrives: the host is down, the connection times out or the URL is malformed. A 404 or 500 is a successful request with `ok` set to false. Check it yourself:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/missing", def(*serve.Context c) (
  c.notFound("no such thing")
))
app.start("127.0.0.1:8794").unwrap()

auto reply = requests.get("http://127.0.0.1:8794/missing").unwrap()
log reply.ok reply.status
log reply.body.trim()

if !reply.ok (
  log "the server said no"
)

app.stop().unwrap()
```

Output:

```text
false 404
{"error":"no such thing"}
the server said no
```

## Failures that are failures

When no response arrives, you get an error whose `kind` says why: `NetworkError` (refused or reset), `NotFoundError` (unknown host), `TimeoutError`, `ParseError` (malformed URL), `UnsupportedError` (a scheme other than http and https) or `IOError` (a body over `maxBytes`). The `timeoutMs` option sets the limit and defaults to 30 seconds:

```osl
import "std:serve"
import "std:requests"
import "std:thread"

auto app = serve.new()
app.get("/slow", def(*serve.Context c) (
  thread.sleep(500)
  c.text(200, "finally")
))
app.start("127.0.0.1:8795").unwrap()

auto slow = requests.get("http://127.0.0.1:8795/slow", {timeoutMs: 100})
log slow.isErr() slow.unwrapErr().kind

auto closed = requests.get("http://127.0.0.1:1/")
log closed.unwrapErr().kind

auto odd = requests.get("gopher://example.com/")
log odd.unwrapErr().kind

switch requests.get("http://127.0.0.1:8795/slow", {timeoutMs: 2000}) (
  case ok(r)
    log "got" r.body
  case err(e)
    log "failed:" e.kind
)

app.stop().unwrap()
```

Output:

```text
true TimeoutError
NetworkError
UnsupportedError
got finally
```

## Retrying

Networks fail in passing ways, so retry the ones worth retrying. [retry](/packages/retry.md) calls a function again, with a growing delay, until it succeeds. The function fails by throwing or returning a failed result, so turn a bad status into an error:

```osl
import "std:serve"
import "std:requests"
import "std:retry"

auto app = serve.new()
int hits = 0
app.get("/flaky", def(*serve.Context c) (
  hits = hits + 1
  if hits < 3 (
    c.internalError("not yet")
    return
  )
  c.text(200, "worked on try " ++ hits)
))
app.start("127.0.0.1:8796").unwrap()

auto body = retry.call(def(int attempt) (
  auto r = requests.get("http://127.0.0.1:8796/flaky").unwrap()
  if !r.ok (
    throw error("status " ++ r.status ++ " on attempt " ++ attempt)
  )
  return r.body
), {attempts: 5, delay: 10}).unwrap()
log body

app.stop().unwrap()
```

Output:

```text
worked on try 3
```

Retry only requests that are safe to repeat. A `GET` is. A `POST` that charges a card needs an idempotency key first.

## Redirects and size limits

Redirects are followed by default. Pass `redirects: false` to receive the redirect itself, which is how you read a `Location` header. `maxBytes` (16 MiB by default) stops a server from sending you more than you want to hold:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/old", def(*serve.Context c) (
  c.redirect(301, "/new")
))
app.get("/new", def(*serve.Context c) (
  c.text(200, "the new page")
))
app.get("/big", def(*serve.Context c) (
  c.text(200, "x" * 5000)
))
app.start("127.0.0.1:8797").unwrap()

auto base = "http://127.0.0.1:8797"
auto followed = requests.get(base ++ "/old").unwrap()
log followed.status followed.url followed.body
auto raw = requests.get(base ++ "/old", {redirects: false}).unwrap()
log raw.status raw.headers["Location"]

auto big = requests.get(base ++ "/big", {maxBytes: 1000})
log big.unwrapErr().kind

app.stop().unwrap()
```

Output:

```text
200 http://127.0.0.1:8797/new the new page
301 /new
IOError
```

## Streaming lines

`requests.get` reads the whole body before returning. For a large download, or a server that keeps sending, use `requests.stream(method, url, options)`. It returns as soon as the headers arrive, and `stream.lines()` yields each line as it comes in. Close the stream when you are done:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/log", def(*serve.Context c) (
  c.text(200, "starting\nworking\ndone\n")
))
app.start("127.0.0.1:8798").unwrap()

auto stream = requests.stream("GET", "http://127.0.0.1:8798/log").unwrap()
log stream.status() stream.ok()
for line in stream.lines() (
  log "line:" line
)
stream.close().unwrap()

app.stop().unwrap()
```

Output:

```text
200 true
line: starting
line: working
line: done
```

A stream with an error status is still a stream: check `stream.ok()` before reading. The `timeoutMs` of a stream defaults to no limit and covers reading the body, so set one if the server might go quiet.

## Server-sent events

A server that answers with `text/event-stream` pushes events over one response, as many AI and live-update APIs do. `stream.events()` yields each as `{event, data, id}`. The event name is `message` unless the server names one:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/events", def(*serve.Context c) (
  c.send(200, "text/event-stream", "event: tick\ndata: 1\n\ndata: 2\nid: 7\n\ndata: [DONE]\n\n")
))
app.start("127.0.0.1:8799").unwrap()

auto stream = requests.stream("GET", "http://127.0.0.1:8799/events").unwrap()
for e in stream.events() (
  if e.data == "[DONE]" break
  log e.event e.data e.id
)
stream.close().unwrap()

app.stop().unwrap()
```

Output:

```text
tick 1 
message 2 7
```

A sentinel such as `[DONE]` comes through as an ordinary event, so end the loop yourself when you see it.

## Many requests at once

An async function runs on its own thread, so several requests can be in flight together. `waitAll` collects the results in order:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/n/:n", def(*serve.Context c) (
  thread.sleep(100)
  c.text(200, "item " ++ c.param("n"))
))
app.start("127.0.0.1:8789").unwrap()

async fetch(int n) string (
  return requests.get("http://127.0.0.1:8789/n/" ++ n).unwrap().body
)

auto jobs = [fetch(1), fetch(2), fetch(3)]
for body in thread.waitAll(jobs) (
  log body
)

app.stop().unwrap()
```

Output:

```text
item 1
item 2
item 3
```

Three sleeping requests finish in about the time of one.

**Quiz.** A server answers `404 Not Found`. What does `requests.get(url)` return?

- [ ] A failed result.
- [x] A successful result whose response has `ok` false and `status` 404.
- [ ] `null`.

Answer: A request fails only when no response arrives at all.

### Exercise: Fetch and total

The server lists prices at `/prices` as a JSON array of numbers. Fetch it with `parse<number[]>()`, then print how many prices there are and their total. Treat a failed request as zero prices and print a message instead of crashing.

Starting code:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/prices", def(*serve.Context c) (
  c.json(200, [3.5, 1.25, 10])
))
app.start("127.0.0.1:8788").unwrap()

// fetch and print here

app.stop().unwrap()
```

It should print:

```text
3 prices, total 14.75
```

Hint 1: `requests.get(url).parse<number[]>()` is a result: use `switch` with `ok` and `err` cases.

Hint 2: `prices.sum()` adds an array, and `prices.len` counts it.

A solution:

```osl
import "std:serve"
import "std:requests"

auto app = serve.new()
app.get("/prices", def(*serve.Context c) (
  c.json(200, [3.5, 1.25, 10])
))
app.start("127.0.0.1:8850").unwrap()

number[] prices = []
switch requests.get("http://127.0.0.1:8850/prices").parse<number[]>() (
  case ok(list)
    prices = list
  case err(e)
    log "could not load prices:" e.message
)
log prices.len ++ " prices, total " ++ prices.sum()

app.stop().unwrap()
```

### Exercise: Add two numbers remotely

The server adds two numbers: `POST /add` with `{"a": 2, "b": 3}` answers `{"sum": 5}`. Post those numbers, parse the answer into the `Sum` struct and print the sum.

Starting code:

```osl
import "std:serve"
import "std:requests"

struct Sum (
  int sum = 0
)

class Pair (
  int a = 0
  int b = 0
)

auto app = serve.new()
app.post("/add", def(*serve.Context c) (
  Pair p = Pair()
  c.bindJSON(p).unwrap()
  c.json(200, {sum: p.a + p.b})
))
app.start("127.0.0.1:8776").unwrap()

log "?"

app.stop().unwrap()
```

It should print:

```text
5
```

Hint 1: `requests.post(url, {json: {a: 2, b: 3}})` returns a result.

Hint 2: `.parse<Sum>().unwrap()` turns the response body into a `Sum`.

A solution:

```osl
import "std:serve"
import "std:requests"

struct Sum (
  int sum = 0
)

class Pair (
  int a = 0
  int b = 0
)

auto app = serve.new()
app.post("/add", def(*serve.Context c) (
  Pair p = Pair()
  c.bindJSON(p).unwrap()
  c.json(200, {sum: p.a + p.b})
))
app.start("127.0.0.1:8851").unwrap()

Sum answer = requests.post("http://127.0.0.1:8851/add", {json: {a: 2, b: 3}}).parse<Sum>().unwrap()
log answer.sum

app.stop().unwrap()
```

## API reference

## API reference

Every member of `std:requests`, generated from the source of OSL.

### Functions

#### `requests.delete(string url, object? options) result<{body: string, error: string (deprecated), headers: string[string], ok: boolean, raw: any (deprecated), status: int, statusText: string, success: boolean (deprecated), url: string}, error>`

delete sends a DELETE request; see requests.request for the response,
options and errors.

Deprecated option timeout: it is in seconds; pass timeoutMs (milliseconds) instead.
Deprecated option params: pass query instead.
Deprecated option max_bytes: pass maxBytes instead.

#### `requests.get(string url, object? options) result<{body: string, error: string (deprecated), headers: string[string], ok: boolean, raw: any (deprecated), status: int, statusText: string, success: boolean (deprecated), url: string}, error>`

get sends a GET request; see requests.request for the response, options
and errors.

Deprecated option timeout: it is in seconds; pass timeoutMs (milliseconds) instead.
Deprecated option params: pass query instead.
Deprecated option max_bytes: pass maxBytes instead.

#### `requests.head(string url, object? options) result<{body: string, error: string (deprecated), headers: string[string], ok: boolean, raw: any (deprecated), status: int, statusText: string, success: boolean (deprecated), url: string}, error>`

head sends a HEAD request, whose response has an empty body; see
requests.request for the response, options and errors.

Deprecated option timeout: it is in seconds; pass timeoutMs (milliseconds) instead.
Deprecated option params: pass query instead.
Deprecated option max_bytes: pass maxBytes instead.

#### `requests.options(string url, object? options) result<{body: string, error: string (deprecated), headers: string[string], ok: boolean, raw: any (deprecated), status: int, statusText: string, success: boolean (deprecated), url: string}, error>`

options sends an OPTIONS request; see requests.request for the response,
options and errors.

Deprecated option timeout: it is in seconds; pass timeoutMs (milliseconds) instead.
Deprecated option params: pass query instead.
Deprecated option max_bytes: pass maxBytes instead.

#### `requests.patch(string url, object? options) result<{body: string, error: string (deprecated), headers: string[string], ok: boolean, raw: any (deprecated), status: int, statusText: string, success: boolean (deprecated), url: string}, error>`

patch sends a PATCH request; see requests.request for the response,
options and errors.

Deprecated option timeout: it is in seconds; pass timeoutMs (milliseconds) instead.
Deprecated option params: pass query instead.
Deprecated option max_bytes: pass maxBytes instead.

#### `requests.post(string url, object? options) result<{body: string, error: string (deprecated), headers: string[string], ok: boolean, raw: any (deprecated), status: int, statusText: string, success: boolean (deprecated), url: string}, error>`

post sends a POST request, usually with a body or json option; see
requests.request for the response, options and errors.

Deprecated option timeout: it is in seconds; pass timeoutMs (milliseconds) instead.
Deprecated option params: pass query instead.
Deprecated option max_bytes: pass maxBytes instead.

#### `requests.put(string url, object? options) result<{body: string, error: string (deprecated), headers: string[string], ok: boolean, raw: any (deprecated), status: int, statusText: string, success: boolean (deprecated), url: string}, error>`

put sends a PUT request; see requests.request for the response, options
and errors.

Deprecated option timeout: it is in seconds; pass timeoutMs (milliseconds) instead.
Deprecated option params: pass query instead.
Deprecated option max_bytes: pass maxBytes instead.

#### `requests.request(string method, string url, object? options) result<{body: string, error: string (deprecated), headers: string[string], ok: boolean, raw: any (deprecated), status: int, statusText: string, success: boolean (deprecated), url: string}, error>`

**Deprecated.** Deprecated:

request sends an HTTP request and reads the whole response as
{status, ok, statusText, url, headers, body}. A 4xx or 5xx response is
not a failure. Options: headers, query, body, json, timeoutMs (default
30000, 0 waits forever), maxBytes (default 16 MiB) and redirects (default
true; false returns a redirect as the response instead of following it).

#### `requests.stream(string method, string url, object? options) result<*requests.Stream, error>`

stream sends a request and returns as soon as the response headers
arrive, so the body can be read line by line or event by event while the
server is still sending it. Close the stream when done. It takes the
options of requests.request except maxBytes; timeoutMs defaults to 0 (no
limit) and covers reading the body too. It fails like requests.request
when no response arrives; an HTTP error status is a stream with ok()
false.

Deprecated option timeout: it is in seconds; pass timeoutMs (milliseconds) instead.
Deprecated option params: pass query instead.
Deprecated option max_bytes: pass maxBytes instead.

### `*requests.Stream`

#### `stream.close() result<unit, error>`

close stops reading and releases the connection. Closing twice is a
no-op.

#### `stream.events() iter<{data: string, event: string, id: string}>`

events returns an iterator over the body's server-sent events
(text/event-stream), each as {event, data, id}: event is the event type
("message" unless the server names one), data joins the event's data
lines with "\n", and id is the last event id seen ("" if none). Comments
and retry fields are skipped. A sentinel payload such as "[DONE]" is
yielded like any other event. Iterating fails like lines().

#### `stream.headers() string[string]`

headers returns the response headers, mapping each canonical header name
to its value; repeated headers are joined with ", ".

#### `stream.lines() iter<string>`

lines returns an iterator over the body's lines, without their line
endings, ending when the server finishes the body. Several loops and
threads can share one stream; each line goes to one of them. Iterating
throws TimeoutError or NetworkError when reading fails and StateError
once the stream is closed.

#### `stream.ok() boolean`

ok reports whether the status is 2xx.

#### `stream.readEvent() string`

**Deprecated.** Deprecated: use `for event in stream.events()` instead, which yields {event, data, id} records.

readEvent returns the data of the next server-sent event, or "" at the end
of the body, after a failure and for a "[DONE]" event alike.

#### `stream.readLine() string`

**Deprecated.** Deprecated: use `for line in stream.lines()` instead, which ends at the end of the body and throws on failures.

readLine returns the next line, or "" at the end of the body, after a
failure and for a blank line alike.

#### `stream.status() int`

status returns the response's HTTP status code, such as 200.