Initial commit

2026-05-10 11:40:53 +02:00
commit 140d736fc7
16 changed files with 1917 additions and 0 deletions
@@ -0,0 +1,191 @@
+# Go OpenAPI Compatibility Layer
+
+A Go library and CLI tool that provides compatibility with OpenAI API, enabling interactive sessions with local LLMs via llama.cpp.
+
+## Overview
+
+This project bridges the gap between OpenAI's API and local LLM inference engines (llama.cpp). It provides:
+
+- **OpenAPI-compatible client** - Full support for ChatCompletion API
+- **Streaming responses** - Real-time token streaming like OpenAI's API
+- **Interactive CLI** - Session-based interaction with system prompts
+- **Tool support** - Custom tools for reasoning (abstraction analysis, instruction updates)
+- **Standard tools** - Built-in shell and file system tools
+- **Extensible registry** - Easy addition of custom tools
+
+## Features
+
+### Core Functionality
+
+- **Local LLM Integration**: Works with GGUF model files loaded via llama.cpp
+- **System Prompts**: Configurable context for consistent responses
+- **Streaming Output**: Chunked responses for real-time interaction
+- **Reasoning Support**: Built-in reasoning content field for CoT models
+
+### CLI Tools
+
+| Tool | Description |
+|------|-------------|
+| `analyze_abstraction` | Analyzes code patterns (Decorator/Strategy) |
+| `update_instructions` | Updates the system prompt for the session |
+| `run_command` | Execute a shell command |
+| `read_file` | Read the contents of a file |
+| `write_file` | Write content to a file |
+| `list_dir` | List contents of a directory |
+| `replace_content` | Replace a block of text in a file |
+
+### Architecture
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    CLI Interface                             │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│                    Session Manager                           │
+│  - System prompts                                           │
+│  - Session state                                           │
+│  - Hook system (BeforeToolCall, OnReasoning, etc.)         │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│                 Registry System                              │
+│  - MapRegistry (custom tools)                              │
+│  - ShellRegistry (command execution)                       │
+│  - FileRegistry (filesystem operations)                    │
+│  - CompositeRegistry (tool composition)                    │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│                  OpenAPI Client                             │
+│  - ChatCompletion API implementation                       │
+│  - Request/Response handling                               │
+│  - Streaming support                                       │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│           llama.cpp (llama-cpp-go)                         │
+│  - GGUF model loading                                     │
+│  - LLM inference                                          │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Configuration
+
+Environment variables:
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `OPENAI_BASE_URL` | API endpoint | `http://10.10.11.33:8082/v1/` |
+| `OPENAI_API_KEY` | API key for authentication | (empty) |
+
+Model configuration (default):
+```
+Jackrong/Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-GGUF
+```
+
+## Usage
+
+### Quick Start
+
+```bash
+# Run the CLI (uses default settings)
+./main.go
+
+# Run with custom model
+OPENAI_BASE_URL=http://localhost:8080/v1/ \
+OPENAI_API_KEY=your-api-key \
+./main.go
+```
+
+### CLI Interaction
+
+```bash
+$ ./main.go
+Interactive Architect Session (BaseURL: http://10.10.11.33:8082/v1/)
+Type your message and press Enter. Type 'exit' to quit.
+
+User > What is the purpose of this project?
+Assistant > This is an OpenAPI compatibility layer that enables...
+
+User > Analyze my current package structure
+Assistant > [Tool] Executing: analyze_abstraction({"name":"...", ...})...
+[Tool] Returning results...
+
+User > exit
+```
+
+### Custom Tool Registration
+
+```go
+registry := openapi.NewMapRegistry()
+
+// Register a custom tool
+registry.Register("my_custom_tool", "Describe the given function",
+    map[string]any{
+        "type": "object",
+        "properties": map[string]any{
+            "code": map[string]any{"type": "string"},
+        },
+    },
+    func(ctx context.Context, s *openapi.Session, args json.RawMessage) (any, error) {
+        var params struct{ Code string }
+        json.Unmarshal(args, &params)
+        // Your tool logic here
+        return "Analysis complete.", nil
+    })
+```
+
+## Architecture Details
+
+### Session Hooks
+
+The session supports a powerful hook system for intercepting and modifying various events:
+
+```go
+session.SetHooks(openapi.SessionHooks{
+    BeforeToolCall: func(ctx context.Context, s *openapi.Session, tc *openapi.ToolCall) error {
+        // Called before each tool is invoked
+        return nil
+    },
+    OnReasoning: func(ctx context.Context, s *openapi.Session, reasoning string) error {
+        // Called for each reasoning step
+        return nil
+    },
+    OnDone: func(ctx context.Context, s *openapi.Session) error {
+        // Called when the session completes
+        return nil
+    },
+    OnError: func(ctx context.Context, s *openapi.Session, err error) {
+        // Called on errors
+        return nil
+    },
+})
+```
+
+### Registry Pattern
+
+The composite registry allows combining multiple tool sources:
+
+```go
+composite := &openapi.CompositeRegistry{}
+
+// Base registry for custom tools
+baseRegistry := openapi.NewMapRegistry()
+composite.Add(baseRegistry)
+
+// Add standard tools
+composite.Add(openapi.NewShellRegistry())
+composite.Add(openapi.NewFileRegistry("."))
+
+// Apply to session
+session.SetRegistry(composite)
+```
+
+## License
+
+MIT