Local LLM Config
Sets up, manages, and optimizes locally-hosted LLMs (Ollama, LMStudio, llama.cpp) with detection, configuration, benchmarking, and model selection capabilities.
Local LLM Config Skill
When to use
Use when you need to configure, switch, tune, or benchmark locally-hosted LLMs (Ollama, LMStudio, llama.cpp, etc.) for any skill-enabled agent.
Essential for:
- Setting up development environments with local LLMs to avoid API costs
- Ensuring privacy by keeping all LLM processing on the local machine
- Optimizing model selection based on available hardware resources
- Switching between different models for various tasks (reasoning vs. speed)
- Troubleshooting LLM connectivity and performance issues
- Creating reproducible LLM configurations for team sharing
What it does
- 🔍 Detection: Automatically detects installed LLM backends via common CLI tools (ollama, lmstudio, llama-cli) or environment variables
- 📋 Listing: Lists available models with detailed specifications including size, quantization, and estimated memory requirements
- ⚙️ Configuration: Generates model-selection configuration files (YAML format) that agents can consume
- 📊 Benchmarking: Measures model performance including latency, token throughput, and resource utilization
- 🔧 Tuning: Provides helper snippets to generate prompt templates optimized for selected models
- 💾 Persistence: Stores configuration in the agent’s skill directory or profile for consumption by other skills
- 🔄 Management: Enables switching between models and backends without disrupting ongoing agent operations
Typical workflow
- Detect: Run
local-llm-config detectto see what LLM backends are available and what models they offer - List: Use
local-llm-config list --verboseto see detailed model specifications for informed selection - Select: Choose and configure a model with
local-llm-config select --model <model-name> [--backend <backend>] - Benchmark (optional): Verify performance with
local-llm-config bench --model <model-name> [--timeout <seconds>] - Reference: Point your agent’s settings or downstream skills to the generated configuration file
- Manage: Switch models later using the select command or view current config with
show
Configuration Options
$LOCAL_LLM_CONFIG_FILE: Path to custom configuration file (default:llm.yamlin skill directory)$LOCAL_LLM_CACHE_DIR: Directory for caching model information and benchmark results$LOCAL_LLM_VERBOSE: Enable verbose logging (equivalent to--verboseflag)
Supported Backends
- Ollama:
ollamacommand-line interface - LMStudio: Local model serving via LMStudio application
- llama.cpp: Direct execution via
llama-clior similar binaries - Auto: Automatic detection of any available backend (default)
References
- https://ollama.com - Ollama LLM framework
- https://lmstudio.ai - LMStudio local LLM interface
- https://github.com/ggerganov/llama.cpp - llama.cpp efficient LLM inference
- Hugging Face model repositories for compatible model formats
- LiteLLM and LlamaIndex integration patterns for LLM configuration
Triggers
This skill activates when the user mentions:
- "configure local LLM"
- "set up ollama"
- "choose local model"
- "benchmark LLM performance"
- "switch llm backend"
- "detect available models"
- "local llm configuration"
- "manage llama.cpp"
- "lmstudio setup"
- "optimize model selection"
Related skills
- Downloads
- 358All time
- Security audit
- Pass
- Last updated
- 2w ago
- Current version
- v1.0.0
- License
- MIT-0
