Chat

Enter to send · Shift+Enter for new line

What should we build?

Enter to send · Shift+Enter for a new line

Test

Ask a model six fixed questions and compare speed and answers side by side.

Hosted-provider tests only use the network, so they can run alongside a local test or training. Local tests run one at a time.

Speed

Tokens/sec is how fast a model writes (a token is about a word fragment; 20 or more feels fluid). First token is the wait before text appears. Errors include empty answers.

Experiments

Prompt calibration rounds. Every input, output and judge score is here, read-only.

Leaderboard

Average judge score per config, with the five sub-scores as small bars. Click a row to look at one config.

Configs

What each config changed, compared with the first one. Changed sentences are highlighted.

Runs

Models

Models stored on this Mac. Smaller ones run faster and leave room for everything else.

Download a model

Type an Ollama model name. Downloads over 10 GB ask for confirmation.

Try

Installed

The bar shows size against a 10 GB comfort limit for this 16 GB Mac.

Providers

Hosted services that run big models very fast. Many have free tiers. A key is stored in your Mac's Keychain.

Train

Teach a small model your own examples. LoRA trains a small add-on called an adapter instead of the whole model, so it fits in memory.

1

Data

A folder under data/ with training and validation examples.

2

Base model

The Hugging Face id of the model to start from. Pick a small one first.

3

Settings

Iterations are practice steps. More takes longer and learns more.

Adapter name Iterations

Training needs a lot of memory and runs one at a time.

Export to Ollama Experimental

Merge a trained adapter into its base model and add it to Ollama so you can run it by name. This can fail for some model types.

Jobs

Downloads, tests, training and exports, including ones started from the command line.