15 lines
632 B
Markdown
15 lines
632 B
Markdown
# Go OpenAI-Compatible Client for llama.cpp
|
|
|
|
|
|
This gem provides a Go implementation of an OpenAI-compatible API client that interfaces with llama.cpp models.
|
|
It enables seamless integration with LLMs deployed via llama.cpp, supporting common APIs like chat, embeddings, and image generation.
|
|
Features include:
|
|
- REST API compatibility with OpenAI's model
|
|
- Support for various model formats (GGUF, GGML)
|
|
- Configuration via environment variables
|
|
- Request/response streaming support
|
|
- Token counting and usage metrics
|
|
- Batch processing capabilities
|
|
- Flexible prompt templating
|
|
- Error handling with retry logic
|
|
``` |