koboldcpp

Overview KoboldCpp is a portable local inference application focused on GGUF models, story generation and compatible APIs. Output quality, hardware needs and data handling depend on the selected deployment. Best for Users evaluating local or self-hosted AI with greater control over models and...