There should be a way to track agentic steps in production for easier inspection and debugging.

We use Weights and Biases Weave for evaluation (see example of evaluation and traces). We should explore how suitable it is for production workflows.

Also, we need dashboards that show API request rates, processed queue messages, error counts, AI costs, and other useful metrics for the Hackbot services. Ideally, those should be customizable (think Grafana way). Maybe GCP dashboards can be sufficient.