docs: add AGENTS.md for AI coding assistants

This commit is contained in:
Ishita Gupta 2025-11-24 15:15:44 +00:00
parent 47e272c35a
commit 350da7610d
1 changed files with 938 additions and 0 deletions

938
AGENT.md Normal file
View File

@ -0,0 +1,938 @@
# AGENTS.md
> **Purpose**: This document provides structured information for AI coding assistants (like Claude, GitHub Copilot, etc.) to understand the Ollama repository structure, conventions, and development workflows.
## Repository Overview
**Project Name**: Ollama
**Repository**: https://github.com/ollama/ollama
**Description**: Ollama enables users to run large language models (LLMs) locally. It supports models like Llama, Mistral, Gemma, DeepSeek, and many others across macOS, Linux, and Windows platforms.
**Primary Language**: Go
**License**: MIT License
**Current Go Version Required**: 1.24.1 (1.22+ minimum supported)
## Project Structure
```
ollama/
├── api/ # API definitions and client implementations
├── app/ # Application-specific code (desktop app)
├── cmd/ # Command-line interface implementations
├── convert/ # Model format conversion utilities
├── discover/ # Hardware and runtime discovery
├── docs/ # Documentation files
├── format/ # Model format handlers
├── fs/ # File system utilities
├── harmony/ # Additional functionality modules
├── integration/ # Integration tests
├── kvcache/ # Key-value cache implementation
├── llm/ # LLM inference engine
├── ml/ # Machine learning backend infrastructure
│ └── backend/ # GGML backend for GPU acceleration
├── model/ # Model definitions and configurations
├── parser/ # Modelfile parser
├── readline/ # Terminal readline implementation
├── runner/ # Model runner implementations
├── scripts/ # Build and utility scripts
├── server/ # HTTP server implementation
├── template/ # Prompt template system
├── thinking/ # Thinking/reasoning functionality
├── types/ # Type definitions and utilities
└── version/ # Version information
```
## Tech Stack
- **Language**: Go 1.24.1 (1.22+ compatible)
- **GPU Acceleration**:
- CUDA (NVIDIA)
- ROCm (AMD)
- Metal (Apple Silicon)
- GGML Backend (`ml/backend/ggml/`)
- **HTTP Framework**: `github.com/gin-gonic/gin` + standard `net/http`
- **CLI Framework**: `github.com/spf13/cobra`
- **Testing**: Go's standard testing package
- **Linting**: golangci-lint
- **Package Management**: Go Modules
## Development Environment Setup
### Prerequisites
1. **Install Go**: Version 1.24.1 or higher (1.22+ minimum)
```bash
# macOS
brew install go
# Linux
wget https://go.dev/dl/go1.24.1.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.24.1.linux-amd64.tar.gz
```
2. **Clone the Repository**:
```bash
git clone https://github.com/ollama/ollama.git
cd ollama
```
3. **Install Dependencies**:
```bash
go mod download
```
### Platform-Specific Requirements
**macOS**:
- Xcode Command Line Tools
- CMake (for building llama.cpp)
**Linux**:
- GCC or Clang
- CMake
- CUDA Toolkit (for NVIDIA GPU support)
- ROCm (for AMD GPU support)
**Windows**:
- Visual Studio 2019 or later with C++ tools
- CMake
- CUDA Toolkit (for NVIDIA GPU support)
## Build Commands
### Standard Build
```bash
# Build the main binary
go generate ./...
go build .
# Build for specific OS/architecture
GOOS=linux GOARCH=amd64 go build .
GOOS=darwin GOARCH=arm64 go build .
GOOS=windows GOARCH=amd64 go build .
```
### Development Build
```bash
# Build with debug symbols
go build -gcflags="all=-N -l" .
```
### Clean Build
```bash
go clean -cache
go build .
```
## Test Commands
### Run All Tests
```bash
go test ./...
```
### Run Tests with Verbose Output
```bash
go test -v ./...
```
### Run Tests with Coverage
```bash
go test -cover ./...
# Generate HTML coverage report
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out -o coverage.html
```
### Run Specific Package Tests
```bash
go test ./server/...
go test ./api/...
```
### Run Integration Tests
```bash
go test ./integration/...
```
### Run Tests with Race Detector
```bash
go test -race ./...
```
### Benchmarks
```bash
go test -bench=. ./...
```
## Linting and Code Quality
### Run Linter
```bash
# Install golangci-lint if not already installed
go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
# Run linter
golangci-lint run
# Run linter with auto-fix
golangci-lint run --fix
```
### Linter Configuration
The project uses `.golangci.yaml` for configuration. Key linters enabled:
- `gofmt` - Code formatting
- `goimports` - Import organization
- `govet` - Static analysis
- `errcheck` - Unchecked error handling
- `ineffassign` - Unused variable assignments
- `staticcheck` - Advanced static analysis
- `misspell` - Common misspellings
- `gosec` - Security issues
### Common Linter Errors and Fixes
```bash
# Error: "unused variable"
# Fix: Remove unused variable or use _ for intentional ignoring
x := someFunc() // linter error
_ = someFunc() // correct
# Error: "missing error check"
# Fix: Always check error returns
file, err := os.Open("test.txt")
if err != nil {
return err
}
# Error: "package not imported"
# Fix: Run goimports to auto-fix
goimports -w .
```
### Running Pre-commit Checks
```bash
# Run all quality checks before committing
go fmt ./...
go vet ./...
golangci-lint run
go test ./...
```
### Format Code
```bash
# Format all Go files
go fmt ./...
# Using goimports (recommended)
go install golang.org/x/tools/cmd/goimports@latest
goimports -w .
```
### Vet Code
```bash
go vet ./...
```
## Coding Standards
### Go Style Guide
- Follow the official [Effective Go](https://golang.org/doc/effective_go.html) guidelines
- Use [Uber's Go Style Guide](https://github.com/uber-go/guide/blob/master/style.md) for additional best practices
### Naming Conventions
- **Packages**: Short, lowercase, single-word names (e.g., `server`, `api`, `model`)
- **Files**: Lowercase with underscores (e.g., `model_file.go`, `http_server.go`)
- **Functions**: CamelCase, exported functions start with uppercase (e.g., `CreateModel`, `parseModelfile`)
- **Variables**: CamelCase, descriptive names (e.g., `modelName`, `ctx`, `err`)
- **Constants**: CamelCase or ALL_CAPS for exported constants
### Error Handling Patterns
```go
// Pattern 1: Simple error check
if err != nil {
return fmt.Errorf("operation failed: %w", err)
}
// Pattern 2: Error with context
if err := loadModel(ctx, name); err != nil {
return fmt.Errorf("failed to load model %q: %w", name, err)
}
// Pattern 3: Error wrapping with additional data
if err := save(file); err != nil {
slog.Error("failed to save file", "path", file, "error", err)
return fmt.Errorf("save failed: %w", err)
}
// Pattern 4: Defer-based cleanup and error handling
func process(path string) error {
file, err := os.Open(path)
if err != nil {
return fmt.Errorf("cannot open file: %w", err)
}
defer file.Close() // Always runs, even on error
// Process file...
return nil
}
```
### Context Usage
```go
// Always pass context.Context as the first parameter
func ProcessModel(ctx context.Context, modelName string) error {
// Check for cancellation
select {
case <-ctx.Done():
return ctx.Err()
default:
}
// ... rest of function
}
```
### Logging
```go
// Use structured logging with slog for consistency
import "log/slog"
slog.Info("loading model", "name", modelName, "size", modelSize)
slog.Error("failed to load model", "error", err, "name", modelName)
// Note: Some legacy code may use standard log package, but slog is preferred for new code
```
### Comments
```go
// Package-level comments
// Package server implements the HTTP server for Ollama.
package server
// Function comments for exported functions
// CreateModel creates a new model from the given Modelfile.
// It returns an error if the Modelfile is invalid or the model already exists.
func CreateModel(ctx context.Context, name string, modelfile string) error {
// Implementation
}
```
### Comment Conventions
- **Package Comments**: Required for all packages; start with "Package <name>"
```go
// Package server provides HTTP server implementation for Ollama.
package server
```
- **Function/Method Comments**: Required for all exported functions; explain purpose, parameters, and return values
```go
// LoadModel loads a model from disk and prepares it for inference.
// Returns an error if the model doesn't exist or is corrupted.
func LoadModel(ctx context.Context, name string) (*Model, error) {}
```
- **Type Comments**: Required for exported types
```go
// ModelConfig holds configuration for a model instance.
type ModelConfig struct {
Name string
Layers int
}
```
- **Inline Comments**: Use `//` for single-line, `/* */` for multi-line (sparingly)
```go
// Bad: Comments state the obvious
i++ // increment i
// Good: Explain why, not what
i++ // skip hidden layers that don't require gradient updates
```
- **Deprecated Items**: Mark with `// Deprecated:` comment
```go
// Deprecated: Use LoadModelV2 instead.
func LoadModel(ctx context.Context, name string) error {}
```
- **TODO Comments**: Include context and responsible person if known
```go
// TODO(john): Optimize memory usage for large batch sizes (issue #1234)
```
## Common Patterns
### API Request/Response Pattern
```go
type CreateRequest struct {
Name string `json:"name"`
Modelfile string `json:"modelfile"`
Stream bool `json:"stream"`
}
type CreateResponse struct {
Status string `json:"status"`
}
func (s *Server) CreateHandler(w http.ResponseWriter, r *http.Request) {
var req CreateRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
// Process request...
}
```
### Model Loading Pattern
```go
func LoadModel(ctx context.Context, name string) (*Model, error) {
// Check if model exists
manifest, err := GetManifest(name)
if err != nil {
return nil, fmt.Errorf("model not found: %w", err)
}
// Load model layers
model := &Model{Name: name}
for _, layer := range manifest.Layers {
if err := model.LoadLayer(ctx, layer); err != nil {
return nil, fmt.Errorf("failed to load layer: %w", err)
}
}
return model, nil
}
```
### Streaming Response Pattern
```go
func (s *Server) GenerateHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/x-ndjson")
w.Header().Set("Transfer-Encoding", "chunked")
encoder := json.NewEncoder(w)
for token := range generateTokens() {
if err := encoder.Encode(token); err != nil {
return
}
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
}
}
```
## Documentation Standards
### Code Documentation
- All exported functions, types, and constants must have GoDoc comments
- Comments should explain "why" not just "what"
- Include examples for complex functions
### README Updates
- Update README.md for user-facing features
- Include usage examples
- Document new CLI commands or API endpoints
### API Documentation
- Document all API endpoints in `docs/api.md`
- Include request/response examples
- Specify error codes and meanings
## Testing Standards
### Unit Tests
```go
func TestCreateModel(t *testing.T) {
tests := []struct {
name string
modelName string
modelfile string
wantErr bool
}{
{
name: "valid model",
modelName: "test-model",
modelfile: "FROM llama2",
wantErr: false,
},
{
name: "invalid modelfile",
modelName: "test-model",
modelfile: "INVALID",
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := CreateModel(context.Background(), tt.modelName, tt.modelfile)
if (err != nil) != tt.wantErr {
t.Errorf("CreateModel() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
```
### Integration Tests
- Place in `integration/` directory
- Use `-tags=integration` for build constraint
- Test end-to-end workflows
### Test Coverage Goals
- Aim for >70% overall coverage
- Critical paths should have >90% coverage
- All exported functions should have tests
## Git Workflow
### Branch Naming
- `feature/description` - New features
- `fix/description` - Bug fixes
- `docs/description` - Documentation updates
- `refactor/description` - Code refactoring
### Commit Messages
Follow [Conventional Commits](https://www.conventionalcommits.org/):
```
type(scope): subject
body (optional)
footer (optional)
```
Examples:
```
feat(api): add streaming support for chat endpoint
fix(server): resolve memory leak in model loading
docs(readme): update installation instructions
test(api): add tests for pull command
refactor(llm): optimize token generation loop
```
Types:
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation changes
- `test`: Adding or updating tests
- `refactor`: Code refactoring
- `perf`: Performance improvements
- `chore`: Build process or auxiliary tool changes
### Pull Request Process
1. **Before Creating PR**:
- Run tests: `go test ./...`
- Run linter: `golangci-lint run`
- Format code: `go fmt ./...`
- Update documentation if needed
2. **PR Title**: Follow commit message format
```
feat(api): add new endpoint for model comparison
```
3. **PR Description Template**:
```markdown
## Description
Brief description of changes
## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Testing
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] Manual testing performed
## Checklist
- [ ] Code follows style guidelines
- [ ] Self-review completed
- [ ] Comments added for complex code
- [ ] Documentation updated
- [ ] No new warnings generated
- [ ] Tests pass locally
```
4. **Review Process**:
- At least one approval required from maintainers
- Address all review comments
- Keep PR scope focused and reasonable size
## Dependencies Management
### Adding Dependencies
```bash
# Add a new dependency
go get github.com/example/package
# Add specific version
go get github.com/example/package@v1.2.3
# Update go.mod and go.sum
go mod tidy
```
### Updating Dependencies
```bash
# Update all dependencies
go get -u ./...
# Update specific dependency
go get -u github.com/example/package
# Clean up
go mod tidy
```
### Vendoring (if used)
```bash
go mod vendor
```
### Build Flags and Environment Variables
```bash
# Common build flags
go build -ldflags="-s -w" # Strip symbols for smaller binary
go build -ldflags="-X main.Version=v1.0.0" # Embed version at build time
# Common environment variables for building
GO111MODULE=on go build . # Ensure module mode (default in 1.24)
CGO_ENABLED=1 go build . # Enable C bindings (needed for GPU)
GOOS=linux GOARCH=amd64 go build . # Cross-compile
GODEBUG=gocachehash=1 go build . # Debug cache misses
```
### Checking Build Environment
```bash
# View all Go environment variables
go env
# Check specific variables
go env GOVERSION
go env GOPATH
go env GOPROXY
go env GOFLAGS
# Verify build toolchain
go version
go list -m all # List all dependencies
```
## Performance Considerations
### CPU Optimization
- Use goroutines for concurrent operations
- Implement worker pools for parallel processing
- Profile with `pprof` to identify bottlenecks
### Memory Management
- Use sync.Pool for frequently allocated objects
- Avoid memory leaks by closing resources
- Profile memory usage with `go test -memprofile`
### GPU Utilization
- Batch requests when possible
- Keep model loaded in GPU memory
- Use appropriate batch sizes for inference
## Security Best Practices
### Input Validation
```go
// Validate and sanitize all inputs
func ValidateModelName(name string) error {
if name == "" {
return errors.New("model name cannot be empty")
}
if matched, _ := regexp.MatchString(`^[a-zA-Z0-9._-]+$`, name); !matched {
return errors.New("invalid model name format")
}
return nil
}
```
### Secure File Operations
```go
// Use secure file permissions
os.WriteFile(path, data, 0600)
// Validate file paths to prevent directory traversal
filepath.Clean(userPath)
```
### API Security
- Validate all API inputs
- Implement rate limiting
- Use HTTPS in production
- Sanitize error messages (don't leak sensitive info)
## Release Process
### Version Numbering
- Follow Semantic Versioning (SemVer)
- Format: `vMAJOR.MINOR.PATCH`
- Example: `v0.1.25`
### Building Releases
```bash
# Build for all platforms
./scripts/build.sh
# Build specific platform
GOOS=linux GOARCH=amd64 go build -o ollama-linux-amd64
```
## Useful Resources
### Official Documentation
- [Ollama Documentation](https://github.com/ollama/ollama/tree/main/docs)
- [API Documentation](https://github.com/ollama/ollama/blob/main/docs/api.md)
- [Modelfile Documentation](https://github.com/ollama/ollama/blob/main/docs/modelfile.md)
### Go Resources
- [Effective Go](https://golang.org/doc/effective_go.html)
- [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
- [Uber Go Style Guide](https://github.com/uber-go/guide/blob/master/style.md)
### Community
- [GitHub Issues](https://github.com/ollama/ollama/issues)
- [GitHub Discussions](https://github.com/ollama/ollama/discussions)
- [Discord Server](https://discord.gg/ollama)
## Common Issues and Solutions
### Build Issues
```bash
# Clean cache and rebuild
go clean -cache -modcache
go mod download
go generate ./...
go build .
```
## Common Issues and Solutions
### Build Issues
```bash
# Clean cache and rebuild
go clean -cache -modcache
go mod download
go generate ./...
go build .
```
### Build Failure Debugging
#### Issue: "undefined reference to..." or linker errors
```bash
# Check if go generate was run
go generate ./...
# Rebuild everything from scratch
go clean -cache
go build .
# For GPU-related errors, ensure CUDA/ROCm is installed
echo $LD_LIBRARY_PATH # Check library path on Linux
```
#### Issue: "import cycle detected"
```bash
# Visualize import dependencies
go mod graph | grep <package-name>
# Solution strategies:
# 1. Move shared types to a new package (e.g., types/)
# 2. Use interfaces to decouple dependencies
# 3. Refactor to remove circular dependency
```
#### Issue: "module not found" or dependency errors
```bash
# Update go.mod and go.sum
go mod tidy
# Check for replaced modules in go.mod
cat go.mod | grep -i replace
# Verify specific dependency
go mod why github.com/example/package
```
#### Issue: CGO build failures (for GPU/ML code)
```bash
# Disable CGO for testing (may limit functionality)
CGO_ENABLED=0 go build .
# Force rebuild with CGO (requires C compiler)
CGO_ENABLED=1 go build .
# Check CGO-related environment
go env CGO_ENABLED
go env CC
go env CXX
```
#### Issue: "permission denied" on build scripts
```bash
# Make scripts executable
chmod +x ./scripts/*.sh
# Or run with bash
bash ./scripts/build.sh
```
### Debugging Build Output
```bash
# Verbose build output
go build -v
# Very verbose with all executed commands
go build -x
# Build with tags
go build -tags=integration .
# Check what will be built
go list -m all
```
### Test Failures
```bash
# Run tests with verbose output
go test -v -run TestName ./...
# Run specific test
go test -v -run TestCreateModel ./server/
# Debug with race detector
go test -race ./...
# Get detailed error output
go test -v -count=1 ./... # disable test caching
```
### Debugging with Delve
```bash
# Install delve
go install github.com/go-delve/delve/cmd/dlv@latest
# Debug a test
dlv test ./server -- -test.v
# Debug the main application
dlv debug . -- serve
# Delve interactive commands
(dlv) break main.main # Set breakpoint
(dlv) break file.go:10 # Break at line
(dlv) condition 1 err != nil # Conditional breakpoint
(dlv) continue # Resume execution
(dlv) next # Next line
(dlv) step # Step into function
(dlv) print variableName # Print variable value
(dlv) backtrace # Show stack trace
(dlv) goroutines # List goroutines
```
### Runtime Debugging
```bash
# Enable verbose/debug logging
OLLAMA_DEBUG=1 ./ollama serve
# Set log level
OLLAMA_DEBUG=1 OLLAMA_LOG_LEVEL=debug ./ollama serve
# Log to file
OLLAMA_DEBUG=1 ./ollama serve > debug.log 2>&1
# Use structured logging to filter output
OLLAMA_DEBUG=1 ./ollama serve 2>&1 | grep "component:server"
```
### Profiling for Performance Issues
```bash
# CPU profiling
go test -cpuprofile=cpu.prof ./...
go tool pprof cpu.prof
# Memory profiling
go test -memprofile=mem.prof ./...
go tool pprof mem.prof
# Interactive pprof commands
(pprof) top # Show top consumers
(pprof) list function # Show function source with stats
(pprof) web # Generate graph (requires graphviz)
```
### Import Cycle Errors
- Refactor code to break circular dependencies
- Move shared types to a common package
- Use interfaces to decouple packages
## AI Agent Instructions
When generating code for this repository:
1. **Always use Go 1.24+ features** when appropriate
2. **Include error handling** for all operations that can fail
3. **Add context.Context** as the first parameter for functions that may block
4. **Write tests** for new functions using table-driven test pattern
5. **Document exported symbols** with GoDoc comments
6. **Follow existing patterns** in the codebase for consistency
7. **Use structured logging** with `log/slog` package (preferred) or `log` package (legacy)
8. **Validate inputs** before processing
9. **Return wrapped errors** using `fmt.Errorf` with `%w`
10. **Keep functions focused** - single responsibility principle
### Example Code Generation Template
```go
// Package description
package packagename
import (
"context"
"fmt"
"log/slog"
)
// FunctionName performs a specific task.
// It returns an error if the operation fails.
func FunctionName(ctx context.Context, param1 string) error {
// Validate inputs
if param1 == "" {
return fmt.Errorf("param1 cannot be empty")
}
// Check context cancellation
select {
case <-ctx.Done():
return ctx.Err()
default:
}
// Perform operation with structured logging
slog.Info("performing operation", "param", param1)
// Handle errors with context wrapping
if err := someOperation(); err != nil {
return fmt.Errorf("failed to perform operation: %w", err)
}
return nil
}
```
---
**Maintainers**: Ollama Team
**Contributing**: Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.