> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokz.si/llms.txt
> Use this file to discover all available pages before exploring further.

# Fine Tune

> Train your own decision model on your text, then run it on our servers or on your machine.

Fine Tune trains your own copy of OpenDecision-Small from a sentence and, if you have
them, your text and labels. It scores the new model against OpenDecision-Small on the
same rows and tells you which one won. After that, a call needs only the model's name and
the text, because the model already knows its question and labels.

```ts theme={null}
import { Tokz } from "@tokz/sdk";

const tokz = new Tokz({ apiKey: process.env.TOKZ_API_KEY! });

const hosted = await tokz.decide("support-triage", "I was charged twice.");
const local = await tokz.decideLocal("support-triage", "I was charged twice.");

console.log(hosted.results[0].decision.best, local.results[0].decision.best);
```

A model belongs to one organization. Any other organization's key gets the same error as
a model that does not exist.

## When it helps

Tuning paid on a few labels, with real text, when OpenDecision-Small was weak. It did not
pay with many labels, or when the labels did not follow from the text.

| Public dataset | OpenDecision-Small | Tuned | Result |
| - | - | - | - |
| Emotion, 6 labels | 47.2 | 59.8 | Better |
| Financial sentiment | 60.8 | 71.8 | Better |
| Spanish reviews, 3 labels | 57.1 | 65.8 | Better |
| Arabic tweets, 2 labels, AI labels | 40.8 | 70.7 | Better |
| Japanese reviews, 3 labels | 57.5 | 62.9 | Not clear |
| 20 Newsgroups, long text | 59.0 | 59.1 | Not clear |
| 28 emotions, multi-label | 29.1 | 30.9 | Not clear |
| Tickets, messy Excel file, 10 queues | 21.7 | 15.6 | Not clear |
| Banking, 77 labels, true labels | 70.3 | 69.1 | Not clear |
| Banking, 77 labels, AI labels | 70.9 | 66.2 | Worse |

Macro F1 out of 100 on 200 rows the model never trained on, measured on 2026-11-03. Each
job had 4 examples and up to 2,300 rows of text, and each row above is one run. Seeds move
these numbers: on the same rows across 2 seeds and our batching variants, Spanish read 64 to
66 and Financial sentiment 73 to 77. None of this is customer data, so test on your own.

### Give it answers you already know

Answer 20 rows you know. After you add a file, the form shows its rows one at a time. You can
also type answers under **Add details**, or label 20 rows in a CSV. Rows the form shows count
for more: they also tell us how common each answer is in your file, which typed rows cannot.

On web logs with no labels (CSIC 2010), scored on 1,500 hidden true answers, a model with no
answers scored 63.91 and 65.23 out of 100 on our earlier training. With 20 answered rows on the
current training, 8 runs scored 69.79 to 94.71, and half scored above 89.56. We picked those 8
to hold the 5 worst of 20 runs. In one real job through the form, a model with 20 answers caught
85% of attacks and blocked 1.8% of normal traffic. Since then we also check how often the AI's
rules give each answer against how often yours do, and the weakest of those 8 runs scored 86.68
and 87.89 in 2 of 3 new runs.

Answer carefully. With 3 of 20 answers wrong, two models scored 3.5 and 11.6 points lower than
the same runs with true answers.

Results still swing from run to run, even with the same rows, so check a model on your own rows
before it blocks anything. A model that gives the same answer to almost every row is never
marked better, and you are not charged for it. Emotion and Financial sentiment barely moved.

## Create one

In the dashboard, open **Fine Tune**, then **New fine tune**. Owners and admins can start
one. This step has no API.

| Field | What to give |
| - | - |
| **Name** | What your code calls the model: 3 to 64 lowercase letters, numbers and dashes. Names starting `opendecision-` are reserved. |
| **What should it decide?** | One question, up to 500 characters: `Which team should handle this ticket?` |
| **Files** | Optional. Up to 5 files and 20 MB in total. CSV, TSV, JSON, JSONL, Excel (`.xlsx`), Parquet and plain text work on every account. An account with a paid top-up can use any other format, which an agent converts. For a real score, include 300 labeled rows or 800 unlabeled rows. |
| **Answer rows** | Optional. With a CSV, TSV, JSON, JSONL or plain text file, the form shows its rows one at a time for you to answer. They go in as examples. Single-label only. Your file stays on the page until you start. |
| **Add details** | Optional. Labels (we work them out if you leave them empty), whether a text can have several, up to 20 typed examples, and anything else it should know. Rows you know the answer to can help more than anything else here. |

A job took 3 to 10 minutes in our tests (13 jobs on public datasets, 8 running at once).
It can take longer when GPUs are busy. You can leave the page: we email the person who
started it when it finishes. Until then you can cancel it, and nothing is charged.

### The result

| Result | What it means | Charge |
| - | - | - |
| **Better** | It beat OpenDecision-Small on rows labeled by you. | \$2.00 |
| **Looks better** | It agrees with AI labels on your text more than OpenDecision-Small does. That is not proof it is right: in one test it read 96.4 on a model that caught 32.8% of attacks. Try it on rows you know. | None |
| **No clear difference** | We cannot tell the two apart on these rows. It can still save sending labels on each call, or run on your machine. | None |
| **Worse** | Use `opendecision-small` for this. | None |
| **No real score** or **Too few rows to score** | There was no file, or we could read too few rows from your files. A score takes 300 labeled rows or 800 unlabeled rows. | None |

Each of these leaves a model you can use or delete. A win means the whole 90% range of the
gain sits above 0, after taking off anything lost by shrinking the model to the file we
host. A job can also fail or come back as **Needs input**. Neither is charged.

### Your text

Your text will be sent to RunInfra, Freestyle and Modal to build this model. It will train
only your model. RunInfra runs the AI model that labels rows, Freestyle hosts a temporary
machine for the job, and Modal runs the GPU.

We hold your files and examples in memory until that machine has them. They do not reach
our disk, our database or our logs. If our server restarts during a job, the job fails with
no charge and you upload them again. We store the name, the question, the description, the
labels and the scores.

## Run it on our servers

The model's page has a **Try it** box, and this code with your model's name filled in.

```bash cURL theme={null}
curl https://api.tokz.dev/v1/decide \
  -H "Authorization: Bearer $TOKZ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"support-triage","inputs":"I was charged twice."}'
```

```ts TypeScript theme={null}
import { Tokz } from "@tokz/sdk";

const tokz = new Tokz({ apiKey: process.env.TOKZ_API_KEY! });

const { results } = await tokz.decide("support-triage", "I was charged twice.");
console.log(results[0].decision.best);
```

```python Python theme={null}
import os
from tokz import Tokz

tokz = Tokz(api_key=os.environ["TOKZ_API_KEY"])

out = tokz.decide("support-triage", "I was charged twice.")
print(out["results"][0]["decision"]["best"])
```

```json theme={null}
{
  "id": "dec_3f9a...",
  "model": "support-triage",
  "results": [
    {
      "decision": {
        "type": "choice",
        "best": "billing",
        "selected": ["billing"],
        "policy": { "billing": 0.81, "technical": 0.19 },
        "value": { "billing": 0.93, "technical": 0.41 }
      }
    }
  ],
  "usage": { "billed_tokens": 256, "cost_usd": 0.0000064 }
}
```

The numbers above only show the shape. The answer is always under `decision`, one per
text, in the same fields as [`/decide`](https://docs.tokz.si/decisions). Do not send `decisions`: the API
refuses it for a fine tune. A multi-label model has `type: "multilabel"`, so read
`selected`, which can be empty. `best` names a label even then.

The model runs on our API server's own CPU. All hosted fine tunes share one process, which
runs one text at a time, so for steady high volume run the model on your machine. A model
unloads after 10 idle minutes, and its next call takes a moment longer. When we are busy or
a model is still loading, you get a `503` with `Retry-After` in seconds. The SDKs carry that
wait in `retryAfter` and `retry_after`, and never retry for you, because a retry could bill
twice.

```ts theme={null}
import { Tokz, TokzError } from "@tokz/sdk";

const tokz = new Tokz({ apiKey: process.env.TOKZ_API_KEY! });

try {
  await tokz.decide("support-triage", "I was charged twice.");
} catch (err) {
  if (err instanceof TokzError && err.status === 503) console.warn("wait", err.retryAfter);
  else throw err;
}
```

Time for one text grows with its labels and its length, because each label is one pass
through the model:

| Labels | 11 tokens | 111 tokens | 3,907 tokens, cut to 511 |
| - | - | - | - |
| 3 | 26 ms | 69 ms | 422 ms |
| 10 | 65 ms | 229 ms | 1,376 ms |
| 28 | 156 ms | 617 ms | 3,754 ms |

Median time through the API on one laptop (Ryzen 5 7535HS, Linux container, 4 CPUs, 2
threads), the slower of two runs, not on our servers, and with no network or queue. The
first request to a model took about 280 ms there, or about 580 ms if the process had to
start too.

The model reads at most 512 tokens of a text, and the whole text still bills. From 32 labels
up, a long text needs more memory than hosted allows, so the API answers
`413 input_too_large` with the longest text that fits. Running on your machine has no such
limit.

## Run it on your machine

One word changes, and the result has the same shape. The call is free and the text stays on
your machine.

```bash npm theme={null}
npm i @tokz/sdk onnxruntime-node@1.26.0 tokenizers@0.23.2 --onnxruntime-node-install=skip
```

```bash pip theme={null}
pip install "tokz[local]"
```

```ts TypeScript theme={null}
import { Tokz } from "@tokz/sdk";

const tokz = new Tokz({ apiKey: process.env.TOKZ_API_KEY! });

const { results } = await tokz.decideLocal("support-triage", "I was charged twice.");
console.log(results[0].decision.best);
```

```python Python theme={null}
import os
from tokz import Tokz

tokz = Tokz(api_key=os.environ["TOKZ_API_KEY"])

out = tokz.decide_local("support-triage", "I was charged twice.")
print(out["results"][0]["decision"]["best"])
```

The first call checks your API key, downloads the model into `~/.cache/tokz` and verifies
its hashes, so it needs the network once. Set `TOKZ_CACHE_DIR` to put the cache elsewhere.
After that it works offline, and it asks the server about once an hour whether the model
still exists, waiting at most about a second. `usage` reads 0 and `id` starts with `local_`.

The Node packages take about 320 MB on disk (Linux x64). A loaded model took about 300 MB of
memory after one short text, and a long text at 28 labels added about 1.3 GB (Node on the
same laptop).

Local runs the same model file as hosted, the file the score describes. On 3 real models and
185 test texts each, every score was within 0.000001 of our reference script and no answer
changed. That covers Node 20 and 22 on Windows, Node 20 on Linux, Python 3.9, 3.10, 3.12 on
Windows and Python 3.12 on Linux, all on one AVX2 laptop, and our server on 2 AMD AVX2 machines.
On an Intel CPU with AVX-512 VNNI, scores moved by up to 0.1 from that laptop's, in our server
and our reference script alike, and 14 of 185 answers changed on a 28-label model, 1 on a
10-label model and none on a 2-label model. So a model can answer a few texts differently from
one machine to another, most often with many labels. macOS and arm64 have not been tested. `decideLocal` runs only on Node.js, and `onnxruntime-node` has
no build for Alpine or Intel Macs, so use `decide` there. Python needs `onnxruntime` at 1.26.0
or older, because newer versions change the scores. `tokz[local]` keeps it there.

## Download

On the model's page, **Download model** gives you a `.tar` of every model file: about 47 MB
on the 3 models we measured, and more for a model trained on long text. Any member of the
organization can download it.

```bash theme={null}
tar -xf support-triage.tar
cd support-triage
pip install onnxruntime==1.26.0 tokenizers numpy
python predict.py "I was charged twice."
```

`predict.py` runs `hosted.onnx`. Add `download` after the text to run `download.onnx`, which
is smaller and is not the file we scored: the two answered differently on 3, 31 and 100 of
185 mixed test texts across 3 models, and at 77 labels the small one scored 61 against 69 on
the same rows, in one run.

## Delete

On the model's page, choose **Delete**. Owners and admins can delete. A model stays until
you do.

* Hosted calls stop right away, and the files leave our servers.
* `decideLocal` and `decide_local` stop, and remove their cached copy, the next time they
  check with us, about once an hour, once the machine can reach us. A machine that never
  reaches us keeps running its copy.
* A `.tar` you downloaded keeps working.
* We erase the question, labels and description. We keep the name, the scores and any
  charge, which stays in your spend. You can use the name again.

## Prices

| | Price |
| - | - |
| Training, with a file | \$2.00, charged only if it beats OpenDecision-Small on rows labeled by you |
| Training, no file | No charge |
| Hosted calls | \$0.025 per 1M billed tokens, at least 256 tokens a request, the same as `opendecision-small` |
| Local calls and downloads | Free |

A hosted call whose client disconnects first is billed for the texts that already ran, with the
same minimum. Texts that had not run are not billed.

Starting any fine tune needs $2.00 in your balance. A job with a file holds $2.00 while it
runs. The hold counts as spent, so a low balance can get a `402` on `/decide` until the hold
ends. See [Billing](https://docs.tokz.si/billing.md#fine-tunes).

A hosted call bills the text's tokens plus the question's, once per 16 labels, plus the label
tokens. Above 16 labels all of that bills at 3x, so at 77 labels a text bills about 15 times
its tokens.

## Limits

| Per organization | |
| - | - |
| Running at once | 1 |
| Started per UTC day | 5 |
| Models stored, counting running jobs | 5 |
| Jobs with no charge, per 30 days | 10 |
| Without a paid top-up | 1 fine tune with a file and 1 without, ever |
| Downloads | 2 GB per UTC day, 2 at once, counting SDK downloads (about 33 MB per model) |

| Per job | |
| - | - |
| Upload | 20 MB in total, 5 files |
| Typed examples | 20 |
| Labels | 2 to 96, each up to 100 characters |

| Per hosted request | |
| - | - |
| Texts | 16, each up to 20,000 characters |
| Texts in progress, per organization | 16 |
| Requests | 120 a minute per key |

The dashboard says which limit you hit and what to do next. There is no API to list models:
`GET /v1/decide/models` shows only the catalog, and `GET /v1/decide/usage` shows a hosted fine
tune under an id starting `ft_`.

## Errors

| Status | `error` | Meaning |
| - | - | - |
| 400 | `unknown_model` | This key's organization has no ready model with that name |
| 400 | `decisions_not_allowed` | Remove `decisions` |
| 400 | `too_many_inputs` | More than 16 texts |
| 402 | `insufficient_credits` | No credit left for this call |
| 413 | `input_too_large` | The text needs more memory than hosted allows. `message` gives the limit |
| 413 | `decide_too_long` | These texts together would take too long to run at once. Send fewer at a time |
| 429 | `decide_org_busy` | Your organization has 16 texts in progress. Wait `Retry-After` |
| 429 | `fine_tune_download_limit` | The day's downloads are used |
| 429 | `fine_tune_download_busy` | 2 downloads are running. Wait `Retry-After`. Both SDKs do this for you, for up to 5 minutes |
| 503 | `decide_busy` | Hosted fine tunes are busy. Wait `Retry-After` |
| 503 | `model_unavailable` | The model is loading (`Retry-After`) or cannot load right now |
| 503 | `fine_tune_off` | Fine Tune is not available right now |

Nothing is billed for any of these. Local mode's errors are on the
[Troubleshooting](https://docs.tokz.si/troubleshooting.md#decidelocal-or-decide_local-fails) page.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.