A terminal tool that right-sizes LLM models to your system's RAM, CPU, and GPU. Detects your hardware, compares it against a database of 33 popular models, and tells you which ones will actually run on your machine.

Ships with an interactive TUI (default) and a classic CLI mode.

curl -fsSL https://llmfit.axjns.dev/install.sh | sh

Downloads the latest release binary from GitHub and installs it to /usr/local/bin (or ~/.local/bin).

Example of a medium performance home laptop

git clone https://github.com/AlexsJones/llmfit.git
cd llmfit
cargo build --release
# binary is at target/release/llmfit

llmfit

Launches the interactive terminal UI. Your system specs are shown at the top. Models are listed in a scrollable table sorted by compatibility.

Use --cli or any subcommand to get classic table output:

# Table of all models ranked by fit
llmfit --cli

# Only perfectly fitting models, top 5
llmfit fit --perfect -n 5

# Show detected system specs
llmfit system

# List all models in the database
llmfit list

# Search by name, provider, or size
llmfit search "llama 8b"

# Detailed view of a single model
llmfit info "Mistral-7B"

1. Hardware detection -- Reads total/available RAM via sysinfo, counts CPU cores, and probes for NVIDIA (nvidia-smi) or AMD (rocm-smi) GPUs.

2. Model database -- 33 models sourced from the HuggingFace API, stored in data/hf_models.json and embedded at compile time. Memory requirements are computed from parameter counts using Q4_K_M quantization (0.5 bytes/param). VRAM is the primary constraint for GPU inference; system RAM is the fallback for CPU-only execution.

3. Fit analysis -- Each model is scored against available memory with awareness of GPU vs CPU execution:

   Run modes:

   • GPU -- Model fits in VRAM. Fast inference.
   • CPU+GPU -- VRAM insufficient, model spills to system RAM with partial GPU offload.
   • CPU -- No GPU detected. Model loaded entirely into system RAM. Slow.

   Fit levels:

   • Perfect -- Recommended memory met on GPU (VRAM). Requires GPU acceleration.
   • Good -- Fits with headroom. Best achievable for CPU+GPU offload.
   • Marginal -- Tight fit, or CPU-only (CPU-only always caps here).
   • Too Tight -- Not enough VRAM or system RAM anywhere.

The model list is generated by scripts/scrape_hf_models.py, a standalone Python script (stdlib only, no pip dependencies) that queries the HuggingFace REST API. Models include families from Meta Llama, Mistral, Qwen, Google Gemma, Microsoft Phi, DeepSeek, Cohere, BigCode, and Nomic.

See MODELS.md for the full list of all 33 included models with parameters, quantization, context length, and use case.

To refresh:

python3 scripts/scrape_hf_models.py
cargo build

The scraper writes data/hf_models.json, which is baked into the binary via include_str!.

src/
  main.rs         -- CLI argument parsing, entrypoint, TUI launch
  hardware.rs     -- System RAM/CPU/GPU detection
  models.rs       -- Model database loaded from embedded JSON
  fit.rs          -- Compatibility analysis (FitLevel scoring)
  display.rs      -- Classic CLI table rendering (tabled crate)
  tui_app.rs      -- TUI application state, filters, navigation
  tui_ui.rs       -- TUI rendering (ratatui)
  tui_events.rs   -- TUI keyboard event handling (crossterm)
data/
  hf_models.json  -- Model database (33 models)
scripts/
  scrape_hf_models.py  -- HuggingFace API scraper

The Cargo.toml already includes the required metadata (description, license, repository). To publish:

# Dry run first to catch issues
cargo publish --dry-run

# Publish for real (requires a crates.io API token)
cargo login
cargo publish

Before publishing, make sure:

• The version in Cargo.toml is correct (bump with each release).
• A LICENSE file exists in the repo root. Create one if missing:

# For MIT license:
curl -sL https://opensource.org/license/MIT -o LICENSE
# Or write your own. The Cargo.toml declares license = "MIT".

• data/hf_models.json is committed. It is embedded at compile time and must be present in the published crate.
• The exclude list in Cargo.toml keeps target/, scripts/, and demo.gif out of the published crate to keep the download small.

To publish updates:

# Bump version
# Edit Cargo.toml: version = "0.2.0"
cargo publish

• Linux -- Full support. GPU detection via nvidia-smi (NVIDIA) and rocm-smi (AMD).
• macOS (Apple Silicon) -- Full support. Detects unified memory via system_profiler. VRAM = system RAM (shared pool). Models run via Metal GPU acceleration.
• macOS (Intel) -- RAM and CPU detection works. Discrete GPU detection if nvidia-smi available.
• Windows -- RAM and CPU detection works. NVIDIA GPU detection via nvidia-smi if installed.

Contributions are welcome, especially new models.

1. Add the model's HuggingFace repo ID (e.g., meta-llama/Llama-3.1-8B) to the TARGET_MODELS list in scripts/scrape_hf_models.py.
2. If the model is gated (requires HuggingFace authentication to access metadata), add a fallback entry to the FALLBACK dict in the same script with the parameter count and context length.
3. Run python3 scripts/scrape_hf_models.py to regenerate data/hf_models.json.
4. Run cargo build to verify compilation.
5. Open a pull request.

See MODELS.md for the current list and AGENTS.md for architecture details.

MIT