Type-safe, provider-neutral agent runtime for Go.

Build streaming, tool-using agents across OpenAI, Anthropic, Gemini, and Mistral without hiding provider-native capabilities.

GAI is a composable Go runtime for model calls, tools, streaming, context, history, and observability. Shared APIs keep application code provider-neutral, while built-in adapters preserve native tools, structured output, reasoning controls, and message history where supported.

Project status: GAI is pre-v1. It is already usable, but public APIs may still change before the first stable release.

The included order-support example uses a typed local tool and streams the final answer:

git clone https://github.com/lace-ai/gai.git
cd gai
export OPENAI_API_KEY="..."
go run ./examples/order-support

Typical output:

User: Where is order LACE-1042, and when should it arrive?
Assistant: Order LACE-1042 is in transit and has left the Vienna logistics center. Austrian Post currently estimates delivery in 3 days.

The wording is generated by the model; the shipping facts come from the local lookup_order tool. See examples/order-support for the complete program and offline tests.

GAI is aimed at Go teams that want a small runtime they can compose into an application instead of a framework that owns the application.

• Typed application boundaries. Models, messages, tools, structured outputs, workflow inputs, and results use Go types.
• Provider-neutral without flattening everything. Built-in adapters preserve native tools, native messages, structured output, usage, and reasoning where available.
• Causally ordered streaming. Tokens, attempts, retries, completed iterations, errors, cancellation, and completion arrive through one stream.
• Composable runtime primitives. Token budgets, persisted history, optional summarization, middleware stages, debug events, and OpenTelemetry are available without forcing a hosting model.

• Go 1.26.1 or newer
• Credentials for the model provider you use

Install the module in an existing application:

go get github.com/lace-ai/gai

This complete program streams a response from an OpenAI-backed agent:

package main

import (
  "context"
  "fmt"
  "log"
  "os"

  "github.com/lace-ai/gai/agent"
  "github.com/lace-ai/gai/ai/openai"
  gaictx "github.com/lace-ai/gai/context"
  "github.com/lace-ai/gai/loop"
  "github.com/lace-ai/gai/ai"
)

func main() {
  if err := run(context.Background()); err != nil {
    log.Fatal(err)
  }
}

func run(ctx context.Context) error {
  provider := openai.New(os.Getenv("OPENAI_API_KEY"), nil)

  model, err := provider.Model("gpt-4.1-mini")
  if err != nil {
    return err
  }
  defer func() {
    if err := model.Close(); err != nil {
      log.Printf("close model: %v", err)
    }
  }()

  assistant := agent.New(agent.Definition{
    Name:  "assistant",
    Model: model,
    Prompt: func(context.Context, agent.RunInput) (gaictx.PromptBuilder, error) {
      return gaictx.New(gaictx.Definition{
        SystemInstructions: []gaictx.Part{
          gaictx.NewTextPart("You are a concise, helpful assistant."),
        },
      }), nil
    },
  })

  workflow, err := assistant.NewRun(ctx, agent.RunInput{
    Prompt: gaictx.PromptInput{
      User: gaictx.NewTextContent("What is the capital of France?"),
    },
  })
  if err != nil {
    log.Fatal(err)
  }

  var runErr error
  for event := range workflow.RunEvents(ctx) {
    switch event.Type {
    case loop.EventToken:
      if event.Token != nil && event.Token.Type == ai.TokenTypeText {
        if event.Token.Text != "" {
          fmt.Print(event.Token.Text)
        } else {
          fmt.Print(event.Token.String())
        }
      }

    case loop.EventError, loop.EventCanceled:
      runErr = event.Err
    }
  }

  fmt.Println()
  return runErr
}

Workflow.RunEvents is the preferred API for a primary agent when event order matters. Each workflow is single-use; create a new workflow from the reusable agent definition for each request.

GAI currently includes these adapters:

Each provider exposes the shared ai.Provider interface:

type Provider interface {
  Name() string
  Model(name string) (Model, error)
  ListModels() ([]string, error)
  Validate() error
}

Built-in providers discover compatible models dynamically and use a bundled fallback catalog when discovery is unavailable. Use ai.ModelRepository to register multiple providers and resolve models centrally.

A tool has a typed schema and a Go function:

type Tool interface {
  Name() string
  Description() string
  Params() ai.ToolParameters
  Function(ctx context.Context, req *ai.ToolCall) *ToolResponse
}

Tool parameters are converted to JSON Schema for provider-native function calling. Models that do not support native tools can use GAI's text-protocol compatibility path.

func (t *LookupOrderTool) Params() ai.ToolParameters {
  return ai.ToolParameters{
    Strict: true,
    Properties: []ai.ToolParameter{
      {
        Name:        "order_id",
        Type:        ai.ToolParameterString,
        Description: "The order ID to look up.",
        Required:    true,
      },
    },
  }
}

func (t *LookupOrderTool) Function(
  ctx context.Context,
  req *ai.ToolCall,
) *loop.ToolResponse {
  var args struct {
    OrderID string `json:"order_id"`
  }
  if err := loop.DecodeToolArgs(req, &args); err != nil {
    return loop.NewToolError(err)
  }
  return loop.NewToolSuccess(`{"status":"in_transit"}`)
}

Attach tools to an agent definition:

support := agent.New(agent.Definition{
  Name:  "support",
  Model: model,
  Tools: []loop.Tool{lookupOrderTool},
  Prompt: supportPrompt,
})

The loop sends definitions to the model, executes requested calls, appends tool results to the conversation, and continues until the model commits a normal response or the iteration limit is reached.

RunEvents forwards the loop's event stream without splitting it into unrelated channels:

for event := range workflow.RunEvents(ctx) {
  switch event.Type {
  case loop.EventAttemptStart:
    // A generation attempt began.
  case loop.EventToken:
    // Stream visible text or inspect other token types.
  case loop.EventRetry:
    // Roll back output associated with event.AttemptID.
  case loop.EventIterationDone:
    // One model/tool iteration completed.
  case loop.EventDone:
    // The loop completed successfully.
  case loop.EventError, loop.EventCanceled:
    // Handle terminal failure or cancellation.
  }
}

Workflow.Run remains available for workflows that use post-processing middleware. It exposes compatibility token, status, and error channels; consumers must drain all three concurrently.

The context package builds prompts from separate, typed inputs:

• system instructions;
• dynamic context sources;
• named machine context;
• genuine user content;
• assistant and tool conversation messages;
• token budgets and an output reserve.

Import it with an alias to avoid colliding with the standard library package:

import gaictx "github.com/lace-ai/gai/context"

builder := gaictx.New(gaictx.Definition{
  Renderer: &gaictx.XMLRenderer{},
  SystemInstructions: []gaictx.Part{
    gaictx.NewTextPart("Follow the application policy."),
  },
  ContextSources: []gaictx.ContextSource{
    history.NewHistory(sessionID, historyStore),
  },
  TokenBudget:        128000,
  OutputTokenReserve: 4096,
})

BuildContext allocates budget to context sources. BuildPrompt then renders the current user input and accumulated conversation for each loop iteration.

context/history provides a ContextSource backed by a HistoryStore. It loads persisted state, selects recent turns that fit the available budget, and reuses cached per-turn token counts.

Use history.NewHistory(sessionID, store) for budgeted history selection. Use history.New(sessionID, store, summarizerDefinition) when older turns should be summarized under token pressure. The built-in agent/summary package can supply the summarizer agent.

Call a model directly when no agent loop is needed or when you want provider-native request controls:

response, err := model.Generate(ctx, ai.AIRequest{
  Prompt:    "Return one JSON object describing Paris.",
  MaxTokens: 200,
  ResponseFormat: ai.ResponseFormat{
    Type: ai.ResponseFormatJSONObject,
  },
  Reasoning: ai.ReasoningConfig{
    Enabled: true,
    Effort:  ai.ReasoningEffortLow,
  },
})
if err != nil {
  return err
}
fmt.Println(response.Text)

AIRequest.Messages can carry provider-neutral native user, assistant, and tool-result history. When it is empty, Prompt remains the rendered compatibility fallback.

Middleware stages run after an upstream agent completes and are suited to tasks such as memory extraction, auditing, formatting, and evaluation.

agent.NewAgentMiddleware adapts another agent with one of three output policies:

• PreserveOutput keeps the upstream output and records the stage result.
• AppendOutput emits the stage output after the upstream output.
• ReplaceOutput replaces the visible output after a successful stage.

Use AgentMiddlewareConfig.MapInput to map a typed upstream WorkflowResult into the next agent's RunInput. Use MiddlewareFunc for transformations that do not need another model call.

Middleware is post-processing, not a supervisor or handoff runtime. Tool authorization and approval belong at the tool-execution boundary rather than in workflow middleware.

GAI creates OpenTelemetry spans across agent runs, workflows, model attempts, and tools. Safe metadata is recorded by default; prompts, completions, reasoning, tool arguments, tool results, and metadata values are not exported unless the application deliberately enables them.

Set Definition.DebugSink for structured lifecycle events. Install any OpenTelemetry-compatible exporter before creating agents.

The optional observability/langfuse package provides a batched OTLP tracer provider for Langfuse:

provider, err := langfuse.NewTracerProvider(ctx, langfuse.Config{
  Endpoint:    "https://cloud.langfuse.com/api/public/otel",
  PublicKey:   os.Getenv("LANGFUSE_PUBLIC_KEY"),
  SecretKey:   os.Getenv("LANGFUSE_SECRET_KEY"),
  ServiceName: "my-agent-service",
})
if err != nil {
  return err
}
defer func() {
  if err := provider.Shutdown(context.Background()); err != nil {
    log.Printf("flush Langfuse traces: %v", err)
  }
}()
otel.SetTracerProvider(provider)

The caller owns provider shutdown so the final batch can be flushed and any export error can be observed.

agent/                   Reusable agent definitions, workflows, middleware, and summary agents
ai/                      Provider-neutral requests, responses, tools, capabilities, and providers
context/                 Prompt construction, rendering, structured input, and messages
context/history/         Persisted history selection and optional summarization
loop/                    Ordered model/tool execution and tool helpers
observability/langfuse/  Optional Langfuse OpenTelemetry exporter setup
testutil/                Mocks and helpers used by tests

Run the full test suite:

go test ./...

Useful focused commands:

go test ./ai/...
go test ./agent/...
go test ./context/...
go test ./loop/...
go test ./examples/order-support

Issues and pull requests are welcome. For a new provider or tool, include tests and document its constructor, supported behavior, and required environment variables.

Before making a large API change, open an issue describing the use case and compatibility impact. GAI is still pre-v1, but migrations should remain deliberate and documented.

GAI is available under the MIT License.

Copyright (c) 2026 Samuel Konrad.