# Go OpenAPI Compatibility Layer

A Go library and CLI tool that provides compatibility with OpenAI API, enabling interactive sessions with local LLMs via llama.cpp.

## Overview

This project bridges the gap between OpenAI's API and local LLM inference engines (llama.cpp). It provides:

- **OpenAPI-compatible client** - Full support for ChatCompletion API
- **Streaming responses** - Real-time token streaming like OpenAI's API
- **Interactive CLI** - Session-based interaction with system prompts
- **Tool support** - Custom tools for reasoning (abstraction analysis, instruction updates)
- **Standard tools** - Built-in shell and file system tools
- **Extensible registry** - Easy addition of custom tools

## Features

### Core Functionality

- **Local LLM Integration**: Works with GGUF model files loaded via llama.cpp
- **System Prompts**: Configurable context for consistent responses
- **Streaming Output**: Chunked responses for real-time interaction
- **Reasoning Support**: Built-in reasoning content field for CoT models

### CLI Tools

| Tool | Description |
|------|-------------|
| `analyze_abstraction` | Analyzes code patterns (Decorator/Strategy) |
| `update_instructions` | Updates the system prompt for the session |
| `run_command` | Execute a shell command |
| `read_file` | Read the contents of a file |
| `write_file` | Write content to a file |
| `list_dir` | List contents of a directory |
| `replace_content` | Replace a block of text in a file |

### Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    CLI Interface                             │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                    Session Manager                           │
│  - System prompts                                           │
│  - Session state                                           │
│  - Hook system (BeforeToolCall, OnReasoning, etc.)         │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                 Registry System                              │
│  - MapRegistry (custom tools)                              │
│  - ShellRegistry (command execution)                       │
│  - FileRegistry (filesystem operations)                    │
│  - CompositeRegistry (tool composition)                    │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                  OpenAPI Client                             │
│  - ChatCompletion API implementation                       │
│  - Request/Response handling                               │
│  - Streaming support                                       │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│           llama.cpp (llama-cpp-go)                         │
│  - GGUF model loading                                     │
│  - LLM inference                                          │
└─────────────────────────────────────────────────────────────┘
```

### Configuration

Environment variables:

| Variable | Description | Default |
|----------|-------------|---------|
| `OPENAI_BASE_URL` | API endpoint | `http://10.10.11.33:8082/v1/` |
| `OPENAI_API_KEY` | API key for authentication | (empty) |

Model configuration (default):
```
Jackrong/Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-GGUF
```

## Usage

### Quick Start

```bash
# Run the CLI (uses default settings)
./main.go

# Run with custom model
OPENAI_BASE_URL=http://localhost:8080/v1/ \
OPENAI_API_KEY=your-api-key \
./main.go
```

### CLI Interaction

```bash
$ ./main.go
Interactive Architect Session (BaseURL: http://10.10.11.33:8082/v1/)
Type your message and press Enter. Type 'exit' to quit.

User > What is the purpose of this project?
Assistant > This is an OpenAPI compatibility layer that enables...

User > Analyze my current package structure
Assistant > [Tool] Executing: analyze_abstraction({"name":"...", ...})...
[Tool] Returning results...

User > exit
```

### Custom Tool Registration

```go
registry := openapi.NewMapRegistry()

// Register a custom tool
registry.Register("my_custom_tool", "Describe the given function",
    map[string]any{
        "type": "object",
        "properties": map[string]any{
            "code": map[string]any{"type": "string"},
        },
    },
    func(ctx context.Context, s *openapi.Session, args json.RawMessage) (any, error) {
        var params struct{ Code string }
        json.Unmarshal(args, &params)
        // Your tool logic here
        return "Analysis complete.", nil
    })
```

## Architecture Details

### Session Hooks

The session supports a powerful hook system for intercepting and modifying various events:

```go
session.SetHooks(openapi.SessionHooks{
    BeforeToolCall: func(ctx context.Context, s *openapi.Session, tc *openapi.ToolCall) error {
        // Called before each tool is invoked
        return nil
    },
    OnReasoning: func(ctx context.Context, s *openapi.Session, reasoning string) error {
        // Called for each reasoning step
        return nil
    },
    OnDone: func(ctx context.Context, s *openapi.Session) error {
        // Called when the session completes
        return nil
    },
    OnError: func(ctx context.Context, s *openapi.Session, err error) {
        // Called on errors
        return nil
    },
})
```

### Registry Pattern

The composite registry allows combining multiple tool sources:

```go
composite := &openapi.CompositeRegistry{}

// Base registry for custom tools
baseRegistry := openapi.NewMapRegistry()
composite.Add(baseRegistry)

// Add standard tools
composite.Add(openapi.NewShellRegistry())
composite.Add(openapi.NewFileRegistry("."))

// Apply to session
session.SetRegistry(composite)
```

## License

MIT