Global Field CTO | Book Author | Blogger | International Speaker | Enterprise Architecture · Data Integration · Process Intelligence · Trusted Agentic AI
Metropolregion München
41.250 Follower:innen
500+ Kontakte
Info
I work across four connected areas: enterprise architecture, data integration, process intelligence and workflow orchestration, and trusted agentic AI. My focus is where they meet, and how enterprises combine them into architectures that are governed, real-time, and ready for AI rather than stitched together after the fact. My analysis and advice stay independent of any vendor.
I am Global Field CTO at Kestra, the open-source unified orchestration platform for infrastructure automation, data pipelines, applications, and business processes. In this role, I work with customers, partners, and analysts on exactly that shift. Alongside this, I run an independent advisory practice through my own company, working with a small number of technology vendors and enterprises on architecture and market positioning. I recommend the right architecture regardless of vendor, including when it is not Kestra. Details at kai-waehner.de.
Before Kestra I spent nine years at Confluent, from the Apache Kafka startup in Silicon Valley through the IBM acquisition in 2026, with earlier roles at Talend and TIBCO, and independent consulting before that.
I publish annual landscape reports, industry guides, and books. I speak at conferences, enterprise events, and executive briefings worldwide across AMER, EMEA, and APAC.
Artikel von Kai Waehner
Aktivitäten
41.250 Follower:innen
-
Kai Waehner hat dies geteiltThe model became the easy part to change. That is exactly why your next lock-in problem will not be the model. An AI agent harness is everything between the model API and the business outcome: tool connections, context and memory, the agent loop, guardrails, and the runtime. Almost everything written about harnesses so far is aimed at the engineers building them. The architecture decision behind it gets much less attention. Model-layer lock-in is dead as a vendor strategy. Models commoditize in months and every platform now advertises model choice. The vendors did not become generous. They moved the lock-in one level higher, into the layer that does not commoditize. Your models change every release cycle. Your harness accumulates agent definitions, tool integrations, context about how your business runs, and hard-won knowledge about what fails. After eighteen months, the model is a config setting and the harness is your architecture. New blog post: "The AI Agent Harness: Where Vendor Lock-in Went After the Model Became Swappable" https://lnkd.in/eVRPsjza #AgenticAI #EnterpriseArchitecture #AgentHarness
-
Kai Waehner hat dies geteiltMCP is just another interface. Use it the right way. The Model Context Protocol gives #AIAgents a standard way to call tools and pull context. It solves a real problem. It does not move your data and it does not keep it consistent. That job belongs to the event backbone. In an #EventDrivenArchitecture, #ApacheKafka stores order events from SAP, customer events from Salesforce and machine data from IoT sensors in one append-only log. Every consumer reads the same events at its own pace. A Flink application reacts in real time. Iceberg tables serve batch analytics. A REST layer handles request-response. Operational and analytical systems share one source of truth instead of dozens of point-to-point pipelines. #MCP is the next consumer on that list. The agent asks for context and the MCP server answers from data that is already governed, versioned and current. No new pipeline, no stale copy. The failure mode is treating MCP as integration middleware. It has no replay, no backpressure, no schema contract, no durable storage. Get the event-driven architecture right first. Then MCP is the easy part.
-
Kai Waehner hat dies geteiltThe latest MCP specification update is a serious rewrite. The core protocol is stateless now, and long running tasks are handled through an official extension instead of an experiment. I still get the same question in almost every architecture discussion: can we just use #MCP for everything? No. MCP is a tool interface for agents. It works well when an agent needs to look up a document, open a ticket in ServiceNow, or start a job and come back later for the result. Operational data is a different problem. Payment flows, CDC from an SAP system, telemetry from a factory floor. These workloads need ordering, replay and delivery guarantees, and they often run at volumes where a request-response protocol was never the answer. #ApacheKafka has covered this for years, and nothing in the new MCP spec changes it. The #AgenticAI hype makes people skip the distinction. The pilot works fine against ten documents, then falls over the first time it touches real transaction data. Both belong in the architecture. The attached table compares MCP, REST and Kafka on transport, delivery, data freshness and governance.
-
Kai Waehner hat dies geteiltA pallet waits at a handover station because the AMR is stuck behind a forklift. Two buildings away, the nightly export job for the quality system failed at 2 a.m. and nobody noticed. Neither is a robot problem or a database problem. Both are coordination problems. Factories do not lack automation. They lack coordination between automations. PLCs, SCADA, MES, ERP, fleet managers, Python DAGs: each layer works, and the failures hide in the handoffs where no single system is in charge. That is the case for #UnifiedOrchestration in #manufacturing and #logistics: one control plane above the Unified Namespace (#MQTT, #OPCUA), the Python orchestrators, the legacy schedulers on shop floor Windows PCs, and the #VDA5050 fleet manager, deployed across edge, air-gapped, and cloud, with #AgenticAI governed on top. The boundary matters more than any feature: orchestration coordinates the fleet manager, the MES, and the pipelines. It never talks to a vehicle, and it should not want to. ISA-95 levels 0 to 2 stay untouched. Read the full article: https://lnkd.in/e9DpkczP
-
Kai Waehner hat dies geteiltThe hard part of data streaming is no longer running Kafka. It is keeping visibility and control as the platform landscape around it changes. On September 24 I am joining it-novum GmbH for a free 45-minute webinar on modern streaming architectures. My part is the neutral market view: ▪️ Where the #DataStreaming landscape stands in 2026 ▪️ How the platform layer around #ApacheKafka has shifted ▪️ What self-managed, BYOC and SaaS mean for digital sovereignty ▪️ Why governance runs through all of it: schema, catalog and lineage across system boundaries Kafka migrations come up regularly now, driven by cost, compliance and vendor changes. Philipp Heck shows a reference stack with migration paths, #DataGovernance and controlled delivery of streaming data to #AI applications. Take it as a blueprint you can map to your own requirements. Free, online, 10:00 to 10:45 CEST. Everyone who registers gets the recording. One note before you click: the session is in German. https://lnkd.in/ezM-tchz
-
Kai Waehner hat dies geteiltThomson Reuters spent about $40 million training its own legal model. Its flagship product, CoCounsel Legal, still runs on Anthropic's Claude Agent SDK. That is not a contradiction. It is a per layer decision. Most of that money went into further training on decades of Westlaw and Practical Law content rather than pre-training a foundation model from scratch. They have used under 10 percent of the content available to them so far. The moat is the data, not the compute. Retrieval stayed in place too. The model is trained to flag uncertainty instead of producing a confident sounding answer, and responses stay anchored to sources a lawyer can check. Trained judgment plus grounding is what #TrustedAgenticAI looks like in a regulated profession. The lesson is not that every enterprise should train a model. Build versus rent gets decided per capability in #AgenticAI, not once for the whole platform. Routing between the owned model and the rented one is #WorkflowOrchestration work before it is model selection.
-
Kai Waehner hat dies geteiltKnowing when NOT to use #StreamProcessing is harder than knowing when to use it, and almost nobody asks the question. Short answer: it is the wrong choice when a caller is waiting for a response, when state has to be queried and corrected directly, or when the work needs locks rather than transformation. After joining Kestra, one of the first things I learned was that Kestra 2.0 removes #KafkaStreams from the core and rebuilds the messaging layer around plain queueing. That triggered an old memory from my Confluent years. Control Center ran its metrics pipeline on Kafka Streams, with startup times of 15 to 50 minutes, until Confluent rearchitected it around Prometheus and got that to about one minute. Two companies that know event streaming as well as anyone both simplified an internal core. Neither concluded that stream processing is bad. Both concluded it was the wrong architecture for that particular job. My new post covers where stream processing earns its place, six signals that point away from it, and why this does not mean using #ApacheKafka less: https://lnkd.in/epWJ-YA5
-
Kai Waehner hat dies geteilt"Kafka vs. MQTT vs. AMQP" is the wrong debate. These are different messaging models, not rival products. The failure mode I see most often in the #EnterpriseArchitecture: forcing one tool to do everything: - #ApacheKafka misused as a queue for transactional request-reply - #MQTT stretched into a central data backbone - A #JMS broker pushed into analytics that need event replay Each works badly, and the workarounds cost more than running the right tool next to it. What works in practice is combination. MQTT connects thousands of devices over unreliable networks at the edge in #IoT use cases. Kafka acts as the central nervous system that retains and distributes events to many independent consumers. Queue-based brokers handle transactional integration with existing applications, in Java landscapes usually via JMS. One nuance worth knowing: AMQP is a wire protocol, JMS is a Java API. They often show up together in the same broker, but they live at different layers. When you evaluate messaging technology, start with the access pattern: consume once, replay many times, or constrained devices. The product decision follows from that.
-
Kai Waehner hat dies geteiltNew blog post: "OceanBase vs. Databricks vs. Snowflake: Which AI Database for Transactional Workloads, Analytics, and AI Agents?" Every database vendor now sells an AI database. The architecture underneath is not identical, and neither are the trade-offs. The deep dive covers: - What an AI database is: one term, three different product types - OceanBase's three stages: from mission-critical transactions at Alipay scale to real-time analytics to multimodal AI - How Databricks and Snowflake converge on the same goal from the analytics side, and where the architectures differ - Shift Left, one layer down: guarantees at write time vs. guarantees across a synchronization boundary - Deployment control as the overlooked axis: cloud-only services vs. running the database in your own environment - When to choose OceanBase, and when NOT to A dashboard never notices a freshness window of a few seconds. An agent approving a payment does. Payment proof points included: Alipay, GCash, DANA, and TNG Digital. Full deep dive: https://lnkd.in/ezm5AgJ9 #OceanBase #AIDatabase #DataArchitecture
-
Kai Waehner gefällt dasKai Waehner gefällt dasIn 2011, we published the Process Mining Manifesto. Looking at it again 15 years later, one part feels surprisingly current: the five maturity levels for event data. At the highest level, event data should not merely be complete and trustworthy. Events and their attributes should have clear semantics, explicitly linked to ontologies. This was 2011, long before today’s excitement around semantic layers, knowledge graphs, context engineering, and enterprise context models. And the Manifesto went further. It explicitly stated that events happen in a context (workload, time, circumstances, etc.) and that meaningful analysis requires merging event data with this contextual information. There is even a remarkably object-centric observation: event data are often “object-centric” rather than “process-centric”, referring to products, pallets, containers, and other interacting entities. Also note that Semantically annotated MXML (SA-MXML) was created in the context of the SUPER FP6 project and implemented within early versions of the ProM framework over twenty years ago! SA-MXML adds semantics to event data, a hot topic today. However, these early activities surrounding SA-MXML seem to have been forgotten. Several ProM plugins exploited this information. So perhaps some of what is currently presented as a new insight is not that new after all. Semantics are essential. Context is essential. But neither is enough. The real challenge is to connect semantics and context to behavior: how objects and events interact over time, where they deviate, where they wait, what caused the problem, and what should happen next. This is what the context model of Celonis aims to do. This is essential for enterprise-level AI. That is precisely why object-centric process mining matters. Ontologies tell us what things mean. Context tells us what surrounds them. Process mining tells us what actually happens. The hype may change. The underlying problem has been remarkably stable. Sometimes it is useful to reread old manifestos. #ProcessMining #ProcessIntelligence #BPM #OCPM #ObjectCentricProcessMining
Berufserfahrung
-
Confluent
-
TIBCO Software Inc.
-
IT Consultant
MaibornWolff et al GmbH
-
Junior IT Consultant
essendi it GmbH
-
Working Student
Client Vela GmbH
Ausbildung
-
University of Bamberg
Diploma Commercial Information Technology, Business Studies
–
-
Ehrenbürg-Gymnasium-Forchheim
Higher School Certificate Science-oriented
–
Veröffentlichungen
-
Apache Kafka vs. Enterprise Service Bus (ESB)—Friends, Enemies, or Frenemies?
Confluent Blog
Veröffentlichung anzeigenThis blog post shows why so many enterprises leverage the open source ecosystem of Apache Kafka for successful integration of different legacy and modern applications, and how this differs but also complements existing integration solutions like ESB or ETL tools.
-
How to Avoid the Anti-Pattern in Analytics: Three Keys for Machine Learning
RTInsights
Veröffentlichung anzeigenWhen a different analytic model is used in training versus deployment, results can be disastrous. Here’s how to avoid the anti-pattern.
-
Using Visual Analytics for Better Decisions: an Online Guide
RTInsights
Veröffentlichung anzeigenHow visual analytics helps businesses make better decisions, and what to look for when evaluating different tools.
-
How to Apply Machine Learning to Event Processing
RTInsights
Veröffentlichung anzeigenHow do you combine historical Big Data with machine learning for real-time analytics? An approach is outlined with different software vendors, business use cases, and best practices.
-
Do Good Microservices Architectures Spell the Death of the Enterprise Service Bus?
Voxxed
Veröffentlichung anzeigenThese days, it seems like everybody is talking about microservices. You can read a lot about it in hundreds of articles and blog posts. This article is about the challenges, requirements and best practices for creating a good microservices architecture, and what role an Enterprise Service Bus (ESB) plays in this game.
-
Real-Time Stream Processing as Game Changer in a Big Data World with Hadoop and Data Warehouse
InfoQ
Veröffentlichung anzeigenThe demand for stream processing is increasing a lot these days. The reason is that often processing big volumes of data is not enough. Data has to be processed fast, so that a firm can react to changing business conditions in real time. A “too late architecture” cannot realize these use cases. This article discusses what stream processing is, how it fits into a big data architecture with Hadoop and a data warehouse (DWH), when stream processing makes sense, and what technologies and products…
The demand for stream processing is increasing a lot these days. The reason is that often processing big volumes of data is not enough. Data has to be processed fast, so that a firm can react to changing business conditions in real time. A “too late architecture” cannot realize these use cases. This article discusses what stream processing is, how it fits into a big data architecture with Hadoop and a data warehouse (DWH), when stream processing makes sense, and what technologies and products you can choose from.
-
Spoilt for Choice – How to choose the right Big Data / Hadoop Platform?
InfoQ
Veröffentlichung anzeigenBig data becomes a relevant topic in many companies this year. Although there is no standard definition of the term „big data“, Hadoop is the de facto standard for processing big data. Almost all big software vendors such as IBM, Oracle, SAP, or even Microsoft use it. However, when you have decided to use Hadoop, the first question is how to start and which product to choose for your big data processes. Several alternatives exist for installing a version of Hadoop and realizing big data…
Big data becomes a relevant topic in many companies this year. Although there is no standard definition of the term „big data“, Hadoop is the de facto standard for processing big data. Almost all big software vendors such as IBM, Oracle, SAP, or even Microsoft use it. However, when you have decided to use Hadoop, the first question is how to start and which product to choose for your big data processes. Several alternatives exist for installing a version of Hadoop and realizing big data processes. This article discusses different alternatives and recommends when to use which one.
-
Choosing the Right ESB for Your Integration Needs
InfoQ
Veröffentlichung anzeigenDifferent applications within companies and between different companies need to communicate with each other. The Enterprise Service Bus (ESB) has been established as a tool to support application integration. But what is an ESB? When is it better to use an integration suite? And which product is best suited for the next project? This article explains why there is no silver bullet and why an ESB can also be the wrong choice. Selecting the right product is essential for project success.
-
Free integration frameworks on the Java platform
The H Developer
Veröffentlichung anzeigenIn addition to the increased data traffic between and within companies and organisations, the number of applications to be integrated has also been rising steadily. Despite the multitude of technologies, protocols and data formats, the integration of these applications should ideally allow standardised modelling, efficient implementation and automated testing. Spring Integration, Mule and Apache Camel are three open source integration frameworks that provide this functionality in the Java world.
-
Lessons Learned: Best Practices for a Successful Introduction of Business Process Management (BPM)
Service Technology Magazine
Veröffentlichung anzeigenBusiness Process Management (BPM) is complex, expensive, and often fails! If you agree (in the year of 2012+), then you should read the following rules to do BPM correctly in your next project.
Weitere ähnliche Profile
Weitere Beiträge entdecken
-
Stephen Abbott Pugh
Tomorrow MariaDB (one of the most popular open source relational databases) is hosting a FOSDEM side event in Brussels with a talk entitled "Building the European cloud with MariaDB" being delivered by Nextcloud https://lnkd.in/eF5G2_Ke #MariaDB has a commercial arm (https://mariadb.com) and a charitable foundation (https://mariadb.org). So I thought I would take a look to find out who its commercial beneficial owners are... 🕵️🔍 MariaDB PLC is registered as a public limited company in Ireland (company number 606330) https://lnkd.in/eAr3gKyX But until 2024, it was listed on the New York Stock Exchange with its corporate documents available on sec.gov. So we can see that in mid 2024, its beneficial owners included Shihuang “Simon” Xie, Lionyet International Ltd (controlled by Mr. Xie, registered in British Virgin Islands), Intel and Chinese tech firm Alibaba (via entities in the Cayman Islands and British Virgin Islands https://lnkd.in/eBezbY-3 Following a privatisation buyout in 2024, the beneficial ownership of the commercial arm was handed over to K1 Investment Management - a US company registered in California - which brought 88.7% of shares in MariaDB PLC https://lnkd.in/ejvP8imD https://lnkd.in/eQBnPVeY K1 Investment Management is an investor in enterprise software companies, headquartered in Manhattan Beach, California. Here is its announcement about acquiring MariaDB https://lnkd.in/eWnH9uvT So MariaDB PLC is registered as a public limited company in Ireland (which is obviously an European Union member state) https://lnkd.in/eAr3gKyX but its main beneficial owner since 2024 is a US private equity firm registered in California (unless it has been sold since the sales docs are not public) #FOSDEM26 #FOSDEM #beneficialownership #opensource
1 Kommentar -
Arvind Sundara Rajan
Semantic Enrichment is rapidly becoming a critical differentiator in LLM performance. It moves beyond simply *feeding* data to models and focuses on augmenting that data with detailed, contextually relevant metadata – essentially, teaching the model *how* to think about the information. This allows for more nuanced reasoning, improved accuracy in complex tasks, and unlocks the potential for truly intelligent automation. Think enhanced decision-making copilots, more reliable risk assessments, and scalable knowledge management systems. 🚀 By investing in semantic enrichment pipelines, organizations can dramatically improve the ROI of their LLM initiatives. 💡📈 #LLMs #AIStrategy #SemanticAI #DataIntelligence #KnowledgeManagement #AIAutomation #Reasoning #EnterpriseAI #GenAI #ToolUse
-
Reiner Weisssieker
SQL Server und AX 2012 Monitor https://lnkd.in/ehCf8mHX Proud to share a new project: rweisssieker-xp/ax_sqlserver — a lightweight, scriptable monitoring and administration tool for Microsoft SQL Server, purpose-built for production use in enterprise and ERP contexts. 🔗 https://lnkd.in/ehCf8mHX 🔍 What it is SQL Server Monitor is an open-source solution for monitoring, diagnostics, and performance analysis of SQL Server instances (SQL Server 2012+). It collects and visualizes performance metrics via Dynamic Management Views (DMVs), detects blocking and deadlocks, and monitors job activity and session load. Built as a modular PowerShell application with a WPF frontend and an optional web dashboard for multi-instance visibility. Released under the MIT license — fully free to use, adapt, and extend. ⚙️ Key Features Open Source & License-Free – no cost, no vendor lock-in. Lightweight Footprint – minimal CPU and memory usage even on busy systems. Fully Scriptable – extend and automate everything via PowerShell. Multi-Instance Monitoring – manage several SQL Servers from one dashboard. Historical Performance Tracking – SQLite backend for long-term trend analysis. Security by Design – least-privilege access (VIEW SERVER STATE, SQLAgentUserRole) and DPAPI-based password encryption. 🧠 Use Cases Quick deployment: clone → configure → connect — ready in minutes. REST API included for integration with other monitoring or ITSM tools. Extendable with your own DMV queries, alerting rules, and dashboards. Ideal for Dynamics AX / D365 FO environments where SQL Server load and transaction latency are key performance factors. If you’re running Microsoft Dynamics AX, D365, or other ERP systems on SQL Server and need a fast, transparent, and customizable performance monitor — this project is worth a look. ⭐ Check it out here: https://lnkd.in/ehCf8mHX 💬 Contributions, feedback, and ideas are welcome!
-
Israel Salgado
If you’ve ever tried to manage Dynatrace configs at scale and thought “this should be a CLI (and Git) problem”… this is for you. 🌟 In this Community Tips & Tricks session, Christoph Neumüller and Andi Grabner demo dtctl, an open source Dynatrace CLI that lets humans and AI agents (e.g., Copilot) browse, edit, query, and automate Dynatrace resources end-to-end. They show how to: - Install/auth and browse + edit objects (dashboards, settings, workflows, notebooks) - Run DQL queries from the CLI - Automate workflow lifecycle: update → execute → debug → closed-loop tests How are you managing Dynatrace configuration today—ClickOps, Terraform, API scripts, or something else? #opensource #cli #observability #devops