Hosted API or local inference?
A hosted API handles GPUs, model loading and the inference server. Local inference gives you control over hardware, runtime, capacity planning, updates and access control.
Model setup and configuration
For hosted inference, choose the API ID, protocol and token budget. For local inference, choose the checkpoint, licence, quantization and runtime, then record the release hash. Evaluate each configuration with the same prompts and settings.
Understand the privacy boundary
Local inference keeps prompts on hardware you control when the application is configured for local processing. A hosted API sends request data to the service and selected inference infrastructure. Review retention terms and application logging.
RouterHub retains account usage and security/audit records, and web chat can save conversation history. Read Privacy for data handling and retention, and keep secrets out of prompts.
Compare total cost
Hosted costs depend on input, output, cache usage and all measured attempts. Local costs include GPUs or rental, memory, electricity, operations, utilization and your time. Compare models with similar task quality and context requirements.
Choose your next step
- Start with a hosted APIConnect with Python or cURL.
- Compare hosted uncensored modelsBrowse uncensored models and compare prices.
- Community model library and open-weight referencesRead the source, examples and project notes on GitHub.
- Evaluate both optionsCompare task quality and cost with repeatable tests.