High-Performance Structured Logging for Common Lisp

LLOG is a modern, high-performance structured logging framework for Common Lisp with rich features for both development and production use.

• Dual API: Ergonomic sugared API and zero-allocation typed API
• Structured Logging: First-class support for key-value fields with type preservation
• Multiple Encoders: JSON, S-expressions, colored console output, and pattern layouts
• Thread-Safe: Concurrent logging with bordeaux-threads locks
• Contextual Logging: Attach fields that flow through call chains
• Leveled Logging: TRACE, DEBUG, INFO, WARN, ERROR, FATAL, PANIC
• Multiple Outputs: Stream and file outputs with per-output configuration
• Async Logging: Background worker threads for non-blocking I/O
• Buffer Pool: Thread-local caching with 92% allocation reduction (typed API vs sugared API)
• File Buffering: Configurable buffering strategies (:none, :line, :block)
• Condition System Integration: Rich error logging with backtrace capture, restarts, and condition chains
• Hook System: Extensible hooks for filtering, metrics, and notifications
• Sampling and Rate Limiting: Control log volume in high-throughput scenarios
• REPL Integration: Recent log buffer, grep search, log capture for testing
• Hierarchical Loggers: Named logger hierarchy with inheritance
• Tamper-Evident Audit Logs (optional): Cryptographic hash chaining with Ed25519 signatures for compliance

(asdf:load-system :llog)

;; Create a logger
(defvar *logger* (llog:make-logger))

;; Simple logging
(llog:info "Application started")
(llog:warn "Configuration missing" :key "database.url")

;; Structured logging with fields
(llog:info "User logged in"
  :user-id 12345
  :username "alice"
  :ip-address "192.168.1.1")

;; Typed API for performance-critical paths (zero-allocation)
(llog:info-typed "Order processed"
  (llog:int "order-id" order-id)
  (llog:string "status" "completed")
  (llog:float "amount" 99.99))

;; Error logging with automatic backtrace capture
(handler-case
    (risky-operation)
  (error (e)
    (llog:error "Operation failed"
                :error (llog:error-field-detailed "error" e :backtrace t))))

;; Multiple outputs: console + async JSON file
(let* ((console (llog:make-stream-output *standard-output*
                                         :encoder (llog:make-console-encoder :colors t)))
       (file (llog:make-file-output "app.log"
                                    :encoder (llog:make-json-encoder)
                                    :buffer-mode :block))
       (async (llog:make-async-output file :queue-size 4096))
       (logger (llog:make-logger :outputs (list console async))))
  (llog:info logger "Logs to both console and file"))

ocicl install llog

git clone https://github.com/atgreen/cl-llog.git
cd llog

In your Lisp REPL:

(asdf:load-system :llog)

Log structured data with typed fields for easy parsing and analysis:

(llog:info-typed "Payment processed"
  (llog:int "user-id" 12345)
  (llog:float "amount" 99.99)
  (llog:duration-ms "processing-time" 45)
  (llog:timestamp "completed-at"))

See the Fields API for all field types.

Multiple outputs with different encoders and buffering strategies:

;; High-throughput: block buffering for performance
(llog:make-file-output "app.log"
  :encoder (llog:make-json-encoder)
  :buffer-mode :block
  :buffer-size 32768)

;; Critical logs: no buffering for durability
(llog:make-file-output "audit.log"
  :encoder (llog:make-json-encoder)
  :buffer-mode :none)

;; Background processing: async output
(llog:make-async-output
  (llog:make-file-output "background.log")
  :queue-size 4096)

See the Outputs API for details on buffering modes and performance.

Rich error logging with automatic backtrace capture:

(handler-case
    (process-payment order)
  (payment-error (err)
    (llog:error "Payment failed"
                :order-id (order-id order)
                :error (llog:error-field-detailed "error" err
                         :backtrace t
                         :restarts t
                         :chain t))))

Output includes: error type, message, stack frames, available restarts, and condition chains.

See the Condition System API for full details.

Extend logger behavior with hooks for filtering, metrics, and notifications:

;; Filter sensitive data
(llog:add-hook *logger* :pre-log
  (lambda (logger entry)
    (redact-passwords entry)))

;; Track error metrics
(llog:add-hook *logger* :post-log
  (lambda (logger entry)
    (when (>= (llog:log-entry-level entry) llog:+error+)
      (incf *error-count*))))

See the Hooks API and examples/hooks.lisp for 10 complete examples.

Control log volume in high-throughput applications:

;; Sample 1% of DEBUG logs
(llog:set-sampling *logger* :debug 0.01)

;; Rate limit errors to 100/second
(llog:set-rate-limit *logger* :error 100 :per-second)

See the Sampling & Rate Limiting API for strategies and examples.

Interactive features for development:

;; Keep recent logs in memory
(llog:enable-recent-logs *logger*)
(llog:show-recent :pattern "error" :level :warn)

;; Search with regex
(llog:grep-logs "User \\d+")

;; Capture logs in tests
(multiple-value-bind (result logs)
    (llog:with-captured-logs ()
      (my-function))
  (assert (= 3 (length logs))))

See the REPL Integration API for full details.

Organize loggers by module with inheritance:

(defvar *app* (llog:get-logger "myapp" :level :info))
(defvar *db* (llog:get-logger "myapp.db" :level :debug))
(defvar *cache* (llog:get-logger "myapp.cache"))  ; Inherits :info from parent

See the Hierarchical Loggers API for details.

Optional extension for compliance requirements (SOC 2, ISO 27001, SOX, HIPAA, PCI DSS):

(asdf:load-system :llog/audit)

;; Create audit output with hash chaining and digital signatures
(llog:add-output *logger*
  (llog/audit:make-audit-output "audit.log"
    :signing-key "private.key"
    :checkpoint-interval 1000))

;; Verify integrity
(llog/audit:verify-audit-file "audit.log" :public-key "public.key")

See src/audit/README.md for full documentation and compliance use cases.

New to LLOG? Start with the User Guide - a comprehensive tutorial covering:

• Getting started with structured logging
• Output configuration and best practices
• Production patterns (high-throughput, buffering, sampling, rate limiting)
• Error logging with backtraces and condition integration
• Advanced features (hooks, custom encoders, hierarchical loggers)
• Testing strategies with log capture
• Deployment guide and production checklist

Complete API documentation is available in docs/api/:

Core APIs:

• Core - Loggers, log levels, logging functions, contextual logging
• Fields - Typed field constructors (string, int, float, bool, timestamp, duration, error)
• Outputs - Stream, file (with buffering modes), and async outputs
• Encoders - Console, JSON, S-expression, and pattern layouts

Advanced Features:

• Hooks - Pre-log, post-log, and error hooks
• Sampling & Rate Limiting - Volume control strategies
• Hierarchical Loggers - Named hierarchy with inheritance
• Condition System - Rich error logging with backtraces
• REPL Integration - Interactive development features

• Typed API: 92% allocation reduction vs sugared API
• Async Logging: Non-blocking I/O with bounded queues
• Block Buffering: 1M+ logs/second for high-throughput scenarios
• Buffer Pool: Thread-local caching eliminates allocation overhead

See the User Guide for performance tuning strategies.

LLOG was written by Anthony Green, and is distributed under the terms of the MIT license.

LLOG combines the best of both worlds:

Performance-oriented design inspired by Go logging libraries:

• uber-go/zap - Dual API design, zero-allocation goals
• rs/zerolog - Performance focus
• sirupsen/logrus - Structured logging

REPL-friendly features inspired by Common Lisp traditions:

• log4cl - Hierarchical logging, configuration management

• Issues: GitHub Issues