# cache

> Keep recent results in memory with a size limit and an expiry time, so slow work is done once.

This is the Markdown version of https://osl-lang.org/packages/cache. Every page of the OSL docs has one: add .md to its path. https://osl-lang.org/llms.txt lists them all.

```osl
import "std:cache"
```

`std:cache` is an in-memory key-value store that forgets on purpose. It holds at most a set number of entries and drops the least recently used one when it is full, and each entry can expire after a time to live. Use it to avoid repeating slow work: a network lookup, a rendered page, a heavy calculation. A cache lives only as long as the program, so use [save](/packages/save.md) or [db](/packages/db.md) for data that must survive.

## Set and get

`cache.new()` makes a cache. `set(key, value)` stores any value under a string key, and `get(key)` returns it, or `null` when there is nothing live under that key:

```osl
import "std:cache"

auto items = cache.new()
items.set("a", 1)
items.set("b", "two")
items.set("list", [3, 4])

log items.get("a") items.get("b") items.get("list")
log items.get("missing")
log items.size() items.has("a") items.has("missing")
log items.delete("a") items.delete("a")
log items.size()
```

Output:

```text
1 two [3,4]
null
3 true false
true false
2
```

`has` tells a missing key from one that holds `null`. `delete` returns whether it removed something, and `clear()` empties the cache.

## Limiting the size

The `capacity` option caps how many entries the cache holds. Reading or writing an entry marks it as recently used, and a new entry in a full cache pushes out the one used longest ago:

```osl
import "std:cache"

auto recent = cache.new({capacity: 2})
recent.set("a", 1)
recent.set("b", 2)
recent.get("a")
recent.set("c", 3)

log recent.keys().sort()
log recent.has("b")
log recent.info()
```

Output:

```text
["a","c"]
false
{"capacity":2,"size":2,"ttl":300000}
```

Reading `a` made `b` the oldest, so `b` was the one dropped. The `ttl` in `info()` is the default lifetime of an entry: 300000 milliseconds, five minutes. The default `capacity` is 100.

## Expiry

The `ttl` option, in milliseconds, makes entries expire. `set` also takes a `ttl` of its own for one entry. `ttl(key)` says how long is left, or `null` for a key that isn't there. Expired entries behave as if they were never stored:

```osl
import "std:cache"
import "std:thread"

auto c = cache.new({ttl: 100})
c.set("short", "soon gone")
c.set("long", "stays", {ttl: 60000})

log c.has("short") (c.ttl("short") ?? 0) > 0
thread.sleep(200)
log c.has("short") c.get("short")
log c.get("long")
log c.ttl("missing")

c.expire("long", 50)
thread.sleep(100)
log c.has("long")
```

Output:

```text
true true
false null
stays
null
false
```

`expire(key, ms)` gives a live entry a new lifetime. Expired entries are removed when next touched, or all at once with `cleanupExpired()`.

## Fetch: compute once

`fetch(key, loader)` is the call you will use most. It returns the live value under `key`, and when there is none it calls `loader()`, stores the answer and returns it. The loader runs only on a miss:

```osl
import "std:cache"

auto squares = cache.new()
int runs = 0

def slowSquare() int (
  runs = runs + 1
  return 12 * 12
)

log squares.fetch("12", slowSquare)
log squares.fetch("12", slowSquare)
log squares.fetch("12", slowSquare)
log "computed" runs "time"
```

Output:

```text
144
144
144
computed 1 time
```

## Caching a server's answers

Here a handler counts how often it does real work, and a cache with a short life sits in front of it. The first request is a miss, the next is served from the cache, and an expired entry is computed again:

```osl
import "std:serve"
import "std:requests"
import "std:cache"
import "std:thread"

auto app = serve.new()
auto pages = cache.new({capacity: 100, ttl: 300})
int renders = 0

def render() string (
  renders = renders + 1
  return "report #" ++ renders
)

app.get("/report", def(*serve.Context c) (
  c.text(200, pages.fetch("report", render))
))
app.start("127.0.0.1:8777").unwrap()

for _ in 1 to 2 (
  log requests.get("http://127.0.0.1:8777/report").unwrap().body
)
thread.sleep(400)
log requests.get("http://127.0.0.1:8777/report").unwrap().body

app.stop().unwrap()
```

It prints `report #1` twice and then `report #2`. The cache is safe to use from several threads at once, so handlers can share one.

> **Note**
>
> `fetch` doesn't stop two simultaneous misses from both running the loader. If the work must happen once even then, guard it with a lock from [sync](/packages/sync.md).

## Looking inside

A cache can be read as a whole: `keys()`, `values()`, `entries()` (an object), `toArray()` (with each entry's expiry time), and `forEach`, `filter` and `mapValues`, which call your function with `(key, value)`, and `reduce`, which calls it with `(result, value)`:

```osl
import "std:cache"

auto stock = cache.new()
stock.setMany({apples: 3, pears: 0, plums: 7})

log stock.entries()
log stock.pick(["apples", "kiwis"])
log stock.filter(def(string key, any value) boolean -> value > 0)
log stock.reduce(0, def(any total, any value) any -> total + value)
```

Output:

```text
{"apples":3,"pears":0,"plums":7}
{"apples":3}
{"apples":3,"plums":7}
10
```

**Quiz.** A cache has `capacity: 2` and holds `a` and `b`. You read `a`, then store `c`. Which key is dropped?

- [ ] `a`, because it is the oldest.
- [x] `b`, the least recently used.
- [ ] None. The cache grows.

Answer: Reading an entry marks it as recently used.

### Exercise: Count the misses

Look up each key with `fetch`. The loader should count how many times it had to run. Repeated keys must come from the cache.

Starting code:

```osl
import "std:cache"

auto lookups = cache.new()
int misses = 0

for key in ["a", "b", "a", "c", "a", "b"] (
  log key
)
log "misses" misses
```

It should print:

```text
a
b
a
c
a
b
misses 3
```

Hint 1: `lookups.fetch(key, loader)` calls `loader()` only when the key isn't cached.

Hint 2: The loader is `def() int ( misses = misses + 1 \n return 1 )`, written over several lines.

A solution:

```osl
import "std:cache"

auto lookups = cache.new()
int misses = 0

def load() int (
  misses = misses + 1
  return 1
)

for key in ["a", "b", "a", "c", "a", "b"] (
  lookups.fetch(key, load)
  log key
)
log "misses" misses
```

## API reference

## API reference

Every member of `std:cache`, generated from the source of OSL.

### Functions

#### `cache.create(any capacity, any ttl) *cache.Cache`

**Deprecated.** Deprecated: use cache.new({capacity: capacity, ttl: milliseconds}) instead.

create makes a cache holding capacity entries that live ttl seconds;
invalid values fall back to 100 entries and 300 seconds.

#### `cache.createDefault() *cache.Cache`

**Deprecated.** Deprecated: use cache.new() instead.

createDefault makes a cache of 100 entries that live 300 seconds.

#### `cache.new(object? options) *cache.Cache`

new creates an empty cache. Options: capacity (the most entries it holds
before evicting the least recently used one, default 100) and ttl (how
long an entry lives after it is set, in milliseconds, default 300000, five
minutes). Throws ValueError for a capacity below 1 or a ttl that isn't
above 0.

### `*cache.Cache`

#### `cache.cleanupExpired() int`

cleanupExpired removes every expired entry now instead of when it is next
read, and returns how many it removed.

#### `cache.clear()`

clear removes every entry from the cache.

#### `cache.delete(string key) boolean`

delete removes key from the cache, returning true when it held a live
entry and false when there was nothing to remove.

#### `cache.deleteMany(string[] keys) int`

deleteMany removes every key in keys, an array of strings, and returns how
many held a live entry.

#### `cache.entries() object`

entries returns the live entries as an object of key to value, without
changing their recency.

#### `cache.expire(string key, number ttl) boolean`

expire gives the live entry under key a new lifetime of ttl milliseconds
from now; a ttl of 0 removes it at once. Returns true when key held a live
entry and false when there was nothing to change. Throws ValueError for a
negative ttl.

#### `cache.fetch(string key, any loader) any`

fetch returns the live value under key, or calls loader() and stores and
returns what it gives back when key is missing or expired. Concurrent
fetches of one key wait for a single loader call. Throws StateError when
the loader fetches its own key, which would otherwise wait forever, and
lets an error the loader throws propagate without storing anything.

#### `cache.filter(any fn) object`

filter returns an object of the live entries for which fn(key, value)
returns true. The cache is unchanged.

#### `cache.forEach(any fn)`

forEach calls fn(key, value) for each live entry, from most to least
recently used.

#### `cache.get(string key) any`

get returns the value stored under key and marks it most recently used, or
null when key is missing or has expired. A stored null also reads as null;
use has to tell them apart.

#### `cache.getMany(array keys) object`

**Deprecated.** Deprecated: use cache.Cache.pick(keys) instead.

getMany returns an object holding the live entries among keys.

#### `cache.getOrSet(any key, any value) any`

**Deprecated.** Deprecated: use cache.Cache.fetch(key, loader) instead, with a loader that returns value.

getOrSet returns the value under key, storing value first when key is
missing.

#### `cache.getOrSetFunc(any key, any fn) any`

**Deprecated.** Deprecated: use cache.Cache.fetch(key, loader) instead.

getOrSetFunc returns the value under key, storing fn() first when key is
missing.

#### `cache.getTTL(any key) int`

**Deprecated.** Deprecated: use cache.Cache.ttl(key) instead, which returns milliseconds or null.

getTTL returns the seconds the entry under key has left, 0 when it has
just expired and -1 when it is missing.

#### `cache.has(string key) boolean`

has reports whether key holds a live (unexpired) entry, even one holding
null. It doesn't change the entry's recency.

#### `cache.info() {capacity: int, size: int, ttl: number}`

info describes the cache as a record with the keys size (live entries),
capacity and ttl (the default lifetime in milliseconds).

#### `cache.keys() string[]`

keys returns the keys of the live entries, in no particular order.

#### `cache.mapValues(any fn) object`

mapValues returns an object of every live key with the value fn(key,
value) returns for it. The cache is unchanged.

#### `cache.pick(string[] keys) object`

pick returns an object holding the live entries among keys, an array of
strings; missing and expired keys are left out. Each entry read becomes
most recently used.

#### `cache.reduce(any initial, any fn) any`

reduce folds the live values, from most to least recently used, into one
result: it starts from initial and replaces it with fn(result, value) for
each value.

#### `cache.set(string key, any value, object? options)`

set stores value under key, replacing any earlier value, and marks it most
recently used. When the cache is full the least recently used entry is
evicted. Option: ttl, this entry's lifetime in milliseconds (default: the
cache's ttl). Mutates the cache; throws ValueError for a negative ttl.

#### `cache.setMany(object entries)`

setMany stores every key and value of entries, as set would one at a time.

#### `cache.setTTL(any key, any ttl) boolean`

**Deprecated.** Deprecated: use cache.Cache.expire(key, milliseconds) instead.

setTTL gives the entry under key a new lifetime in seconds.

#### `cache.size() int`

size returns the number of live entries.

#### `cache.stats() object`

**Deprecated.** Deprecated: use cache.Cache.info() instead, whose ttl is in milliseconds, and cache.Cache.keys() for the keys.

stats describes the cache with the keys size, capacity, ttl (seconds) and
keys.

#### `cache.toArray() {expiresAt: int64, key: string, value: any}[]`

toArray returns the live entries, from most to least recently used, as
records with the keys key, value and expiresAt (unix milliseconds).

#### `cache.ttl(string key) int?`

ttl returns how long the entry under key has left to live, in whole
milliseconds, or null when key is missing or has expired.

#### `cache.values() array`

values returns the values of the live entries, from most to least
recently used, without changing their recency.