About
Engineering leader and architect with 27 years in software, including 17 years focused on…
Articles by Sanjay
-
Jumbune 1.4 Released
Jumbune 1.4 Released
Jumbune is "An open source MapReduce profiling, MapReduce flow debugging, HDFS data quality validation and Hadoop…
Activity
5K followers
-
Sanjay Sharma reposted thisSanjay Sharma reposted this✨ Excited to introduce the 2nd and the final blog in the series showcasing solutions built on Lakebase by Databricks partners The line between 'insight' and 'action' is disappearing - and our partners are building on it. With Lakebase, Databricks' native operational database, transactional operations, analytics, and AI governance finally live on one platform. That means agentic AI that doesn't just analyze what happened, it acts in the moment, with sub-10ms latency and zero-copy branching. What excites me most is what our partners are doing with it. Across 60+ industry-specific solutions, they're turning the platform into real outcomes: 🏦 Financial Services: Claims adjudication from days to minutes, real-time fraud detection, AI-powered wealth operations 🏭 Manufacturing & Energy: Predictive turbine maintenance cutting downtime 35–50%, grid efficiency gains, meter data at billions-of-readings scale 🛒 Retail, CPG & Travel: Dynamic pricing driving double-digit revenue growth, stockout detection in under 90 seconds 🏥 Healthcare & Life Sciences: Real-time prior authorization, behavioral health intelligence 📡 Communications & Media: churn prediction and local advertising optimization 🏛️ Public Sector: AI-powered emergency response reporting This is what the partner ecosystem does best: taking product innovation and shaping it into solutions that move the needle for customers. 👉 https://lnkd.in/gSR3HbTb
-
Sanjay Sharma reposted thisSanjay Sharma reposted thisThe difference between an agent that just answers a question and one that actually reasons consistently over time. The first treats every prompt like a brand-new case with no connection to anything before it, and will cheerfully contradict yesterday's conclusion. The second checks its own archive first if new evidences have cropped in, any contradicts to old reasoning and manage to supersede. Let's explore how agents use their own reasoning graphs to stay consistent, current and relevant in their reasoningPart 4 - "Consulting the Case Files": How Agents Use Their Own Reasoning GraphsPart 4 - "Consulting the Case Files": How Agents Use Their Own Reasoning Graphs
-
Sanjay Sharma reposted thisSanjay Sharma reposted thisIf everyone has access to the same AI models, where does competitive advantage come from? Its what your enterprise brings to it. Your data. Your context. Your institutional knowledge. Your workflows. And the way you put them to work. That’s what makes AI uniquely valuable to your business. Because AI ROI isn’t about having access to the latest model. It’s about turning what is uniquely yours into intelligence AI can act on. That’s the difference between having AI and making AI work for your enterprise. Listen to the podcast to explore where your AI differentiator really comes from. Watch it on Spotify: https://lnkd.in/gEsB2gw9 Watch it on YouTube: https://lnkd.in/gtAbba3Y Nachiket Deshpande #EnterpriseAI #AgenticAI #ContextEngineering #GroundTruth
-
Sanjay Sharma reposted thisSanjay Sharma reposted thisAcross AI takes NYC 🗽 Great executive dinner with Impetus CEO Nachiket Deshpande and AI leaders from JPMorganChase, Transamerica, Citi, Barclays, Wells Fargo and others. We spent the evening talking about what happens after the AI demo works and you actually try to put agents into production. A few of the more interesting debates: Are we about to repeat the mistakes of RPA? Nachiket made a great point that many RPA deployments looked fantastic at first. Then exceptions piled up and companies built large teams just to manage what the automation couldn't handle. Eventually some of the savings disappeared. Agents could follow the same path. A manually built harness might work today, but the business changes, new exceptions show up, and six months later you're maintaining the automation instead of benefiting from it. That led to a big part of our conversation around self-improving systems: agents that can adapt as the world changes, with evals checking they haven't broken something they used to know how to do. Giving an agent your data isn't the same as giving it context. A lot of what an agent needs isn't in a database. It's in SOPs, emails, past decisions, permissions, exceptions and often in the head of someone who has been doing the job for 20 years. One comment that stuck with me: if you deploy AI against a bad process, you're just getting to the bad outcome faster. We may also be thinking about AI cost the wrong way. The interesting metric isn't necessarily cost per token. It's how much useful intelligence you're getting per token. Not every step needs the biggest model. Some can be deterministic code, some can use smaller models, and some genuinely need a frontier model. And maybe agents shouldn't have perfect memory. We got into a long discussion about “purposeful forgetting.” If an agent remembers everything forever, some of what it remembers will eventually become wrong. A good memory system needs to know when something is stale, when it should be archived, and when new information changes what it already knows. This somehow led to an example involving a person who used to work at McDonald's who had a friend called Joe 😂 I suspect Joe will now survive in our collective memory forever. We ended on an even bigger question: as more execution gets automated, what do humans become responsible for? My answer was ambiguity. Defining what we're actually trying to accomplish, deciding what “good” means, understanding exceptions, simplifying messy problems, and building trust. If you can turn a problem into a clear objective and measurable outcome, agents can increasingly do the rest. That one generated some debate :) Really fun group and exactly the kind of candid conversation we wanted these dinners to create. Thank you Nachiket and everyone who joined us. If you'd like to join the next one in Mountain View or NYC, message Mark Azad.
-
Sanjay Sharma reposted thisSanjay Sharma reposted thisImpetus has earned the official Databricks Brickbuilder Specialization status for 𝐑𝐞𝐭𝐚𝐢𝐥, 𝐂𝐨𝐧𝐬𝐮𝐦𝐞𝐫 𝐆𝐨𝐨𝐝𝐬 & 𝐓𝐫𝐚𝐯𝐞𝐥. This recognition validates Impetus’ deep domain expertise, technical capabilities and demonstrated customer success across Retail, CPG and Travel on the Databricks Data Intelligence Platform, as well as our ability to help enterprises navigate the end-to-end journey of making AI work. From modernizing complex legacy data estates and building trusted data foundations, to engineering enterprise context, deploying AI and agentic workflows, and governing and optimizing them in production, Impetus helps organizations turn their Databricks investments into AI that works within the enterprise. This latest recognition adds to six existing Brickbuilder Specializations across products and industries and builds on our position as a Databricks Gold Partner, which further strengthens the depth and breadth of our Databricks expertise. Thank you to Jason McIntyre, Jennifer Zavacky, Skip Stenger and the larger Databricks team for the partnership. Let’s continue to make AI work for enterprises, together. Databricks for Consumer Industries | Databricks Partners Impetus Team: Michael Steinberg, Deepak Khosla, Barry Tuthill, Mike Patnode, David Thomas, Shilpi Saxena, Jeff Veis, Deeksha Gautam #ImpetusXDatabricks #DatabricksPartner #Retail #CPG #Travel #EnterpriseAI
-
Sanjay Sharma reposted thisSanjay Sharma reposted thisAIM Research has launched the Top Databricks Services Partners PeMa Quadrant 2026. Enterprise demand for consolidation, lakehouse modernization, and agentic AI on the Databricks Lakehouse Platform is soaring. To help buyers evaluate leading services partners, AIM Research has released its annual benchmark assessing 16 Consulting & SI partners across Market Penetration (Pe) and Technology Maturity (Ma). The study draws on RFI submissions and discussions, evaluating providers on delivery scale, partnership standing, client base and growth, joint go-to-market execution, validated IP and accelerators, certification depth, production deployment scale, and platform operations. Explore the full report: https://lnkd.in/gBdDrh_h Featured vendors (alphabetical): Aays, Aspire Systems, C5i, Chetu, Inc., Cognitio Analytics Inc, Exponentia.ai, Fractal, Ganit Inc., Hoonartek, Impetus, LatentView Analytics, Logic Pursuits, Polestar Analytics, Qubika, TO THE NEW, Tredence Inc. #Databricks #DataAndAI #Lakehouse #DataEngineering #PeMaQuadrant #AIMResearch
-
Sanjay Sharma reposted thisSanjay Sharma reposted thisFraud is evolving faster than the systems built to stop it. Global fraud is projected to cost financial institutions $58.3 billion by 2030, a 153% increase from 2025. But the challenge isn't just that fraud is increasing. It's becoming harder to see. Fraud signals are spread across transactions, devices, identities, behaviours and external threat intelligence. When these signals remain fragmented across systems, risk teams can struggle to connect the dots in time, leading to missed threats, false declines and slower investigations. This changes what effective fraud mitigation needs to look like. It's not about detecting more signals. It's about connecting them and acting on them in real time. Our latest Point of View explores how financial institutions can build a unified data foundation for real-time fraud intelligence and use agentic AI to strengthen fraud control, improve decisioning, enhance explainability and support audit-ready compliance. The POV covers: • A scalable reference architecture for fraud and risk mitigation • How agentic AI can enhance fraud and compliance outcomes • Customer success stories from Fortune 500 financial services enterprises Get your fraud and risk systems ready for the future. Read the POV now https://lnkd.in/d8wtW9dA #FraudPrevention #FraudDetection #FinancialServices #AI #RiskManagement #Impetus
-
Sanjay Sharma reposted thisSanjay Sharma reposted thisInformation Technology has evolved in three waves so far. The first wave changed how fast we worked. The second changed how we were organized. The third may change who or what gets to decide. The first wave was the computer revolution. Typewriters became word processors, letters became email, filing cabinets became databases, ledgers became spreadsheets. Work got faster and more efficient but remained fundamentally the same. The second wave was digital transformation. It was never really about computers or phones. It was about redesigning the business around what those technologies made possible: customer experience and business models. Successful enterprises weren't the ones who deployed the most technology; they were the ones willing to rebuild the enterprise around it. Now comes AI. Today's focus: copilots, agents, generative AI, increasingly autonomous workflows, is real and already creating value. But much of it may be the equivalent of that first wave: making existing work faster and cheaper, not fundamentally different. The bigger question is what comes next. The first two waves redesigned the work and then the business. This one reaches somewhere new, into judgment itself. As intelligence becomes abundant and systems grow more capable, the constraint stops being can the system decide? and becomes who answers for the decision when it's wrong? We've automated tasks for two waves. We have never automated accountability, and we shouldn't pretend a machine can carry it. A system can make a call but can't stand behind one. This gap needs us to rethink how the whole system comes together. History offers a clue: technology moves fast, organizations move slowly. The real constraints are rarely technological. They are data, skills, incentives, trust and ability to keep pace with accelerating change. A few considerations for leaders thinking about this change: 1. Decide who owns the decision before you decide what to automate. Be clear about which decisions AI will assist, influence, or make and name the person/role accountable. 2. Redesign the work, not just the task. Speeding up one step rarely changes the outcome. Question the whole process it sits inside. 3. Make trust the engine of adoption. People use what they understand and can question. Showing how a decision was reached matters more than how clever the model is. One enduring lesson from past waves: the winners were rarely the ones who adopted the most, or the earliest. They were the ones willing to redesign work rather than just adopt new technology. Computers taught us to digitize work. Digital transformation taught us to redesign the business. AI may teach us to redesign decision-making itself and to decide, deliberately, where human accountability still has to sit. The technology will keep getting more capable. What makes it durable is knowing who stands behind the call. That may be the real work of the third wave, and the most human thing Information Technology has ever asked of us.
Sanjay Sharma commented on a post
1mo
Congratulations! 🎉
-
Sanjay Sharma liked thisSanjay Sharma liked thisFor most of my career, working with data meant sending it somewhere else. A database on one side, your application on the other, a wire in between. That's no longer the case with engines like DuckDB, and today I am happy to share that DuckLabs, the team behind DuckDB, is joining AWS. Andrew Warfield tells the full story. It starts in a pub, moves through the changing economics of compute, and lands on why an embedded analytics engine that runs as a library in your own process is a fundamentally different way to build with data. Read the full post here: https://lnkd.in/dgs8r894 Now, go build!
-
Sanjay Sharma liked thisSanjay Sharma liked thisThe last few weeks at SpaceXAI have made one thing increasingly clear to me: The next frontier in enterprise AI isn’t simply a better model. It’s what happens when frontier intelligence, massive compute, agents, and the right application layer come together. We’re already seeing an early glimpse with Grok 4.6 — particularly its ability to stay with complex, multi-step work for longer periods of time. For Life Sciences, that gets especially interesting. Think beyond writing code: - Researchers moving from a question to analysis, simulation, and insight - Data science teams orchestrating complex workflows across models and tools - Engineering teams using agents across development, testing, review, and modernization - Clinical and commercial teams turning knowledge-intensive processes into repeatable agentic workflows And importantly, enterprises shouldn’t have to bet everything on one model. The ability to pair the right model with the right context, tools, governance, and workflow will matter just as much. Cursor joining SpaceXAI has dramatically expanded what we can build here. And what’s coming next has me even more excited. A fascinating time to be working at the intersection of AI and Life Sciences.
-
Sanjay Sharma liked thisSanjay Sharma liked thisThis Women’s Equality Day, we’re celebrating what equality makes possible. A voice that’s heard. The trust to take on what you haven’t done before. The courage to raise your hand. The freedom to choose what comes next. And, eventually, the chance to open the door for someone else. From there, every journey unfolds differently. Meet the women of Impetus—and hear, in their own words, what those moments have meant to them.—and the journeys that brought them here. Here’s to moving forward—and making the way forward. Swipe to read >> #WomensEqualityDay #WomenOfImpetus #WomenInTech Sheba Fernando Shikha Kandoi Rajani Nair Carly Sinkin Shilpi Saxena
-
Sanjay Sharma liked thisSanjay Sharma liked thisHappy to share that I’ve earned my Claude Certified Architect – Foundations credential, the first step in the AI Architect track. This credential validates my skills in designing, building, and delivering production-grade AI solutions with Claude. Excited to continue building and exploring #ClaudeCertified #AI #GenerativeAI #ClaudeClaude Certified Architect - Foundations was issued by Anthropic to Lalit Rathore.Claude Certified Architect - Foundations was issued by Anthropic to Lalit Rathore.
-
Sanjay Sharma liked thisSanjay Sharma liked thisAcross AI takes NYC 🗽 Great executive dinner with Impetus CEO Nachiket Deshpande and AI leaders from JPMorganChase, Transamerica, Citi, Barclays, Wells Fargo and others. We spent the evening talking about what happens after the AI demo works and you actually try to put agents into production. A few of the more interesting debates: Are we about to repeat the mistakes of RPA? Nachiket made a great point that many RPA deployments looked fantastic at first. Then exceptions piled up and companies built large teams just to manage what the automation couldn't handle. Eventually some of the savings disappeared. Agents could follow the same path. A manually built harness might work today, but the business changes, new exceptions show up, and six months later you're maintaining the automation instead of benefiting from it. That led to a big part of our conversation around self-improving systems: agents that can adapt as the world changes, with evals checking they haven't broken something they used to know how to do. Giving an agent your data isn't the same as giving it context. A lot of what an agent needs isn't in a database. It's in SOPs, emails, past decisions, permissions, exceptions and often in the head of someone who has been doing the job for 20 years. One comment that stuck with me: if you deploy AI against a bad process, you're just getting to the bad outcome faster. We may also be thinking about AI cost the wrong way. The interesting metric isn't necessarily cost per token. It's how much useful intelligence you're getting per token. Not every step needs the biggest model. Some can be deterministic code, some can use smaller models, and some genuinely need a frontier model. And maybe agents shouldn't have perfect memory. We got into a long discussion about “purposeful forgetting.” If an agent remembers everything forever, some of what it remembers will eventually become wrong. A good memory system needs to know when something is stale, when it should be archived, and when new information changes what it already knows. This somehow led to an example involving a person who used to work at McDonald's who had a friend called Joe 😂 I suspect Joe will now survive in our collective memory forever. We ended on an even bigger question: as more execution gets automated, what do humans become responsible for? My answer was ambiguity. Defining what we're actually trying to accomplish, deciding what “good” means, understanding exceptions, simplifying messy problems, and building trust. If you can turn a problem into a clear objective and measurable outcome, agents can increasingly do the rest. That one generated some debate :) Really fun group and exactly the kind of candid conversation we wanted these dinners to create. Thank you Nachiket and everyone who joined us. If you'd like to join the next one in Mountain View or NYC, message Mark Azad.
-
Sanjay Sharma liked thisSanjay Sharma liked thisAt the #CIO100 gala this week, a man walked up to my table, put his arm around me, and said something that instantly brought the lesson that I just learned that day. His name is Michael Lewis. Not the author — though I had the privilege of interviewing that Michael Lewis on our mainstage at #DX2026 earlier this year. This Michael is the CTO at Management Controls. He said, "I really enjoyed your panel today. And just so you know — I was part of your UC Berkeley Executive Education CTO presentation a few years back. It was insightful." I told him: thank you. I really enjoy presenting at the UC Berkeley CTO cohort as a way to pay it forward, but you rarely hear directly that it mattered. So hearing it from him meant a lot. And he said, "Well — now I try to do the same." I sat with that a while. Here's the part that matters to me. I'm not a teacher at Berkeley. I'm an alum — first cohort, 2019. I sat in those same seats once, trying to make the jump from technologist to CTO. I just keep going back, because someone did that for me. The panel ran under Chatham House Rule, so I won't put words in anyone's mouth. But I'll tell you about the room. Dan Roberts brought together 10 industry-recognized leaders for that masterclass, and I'll be honest: I learned more than I shared. I left with two takeaways. The first is for my team. We talked about the real difference between how startups move and how the rest of us do. They get to conviction faster — they test whether an idea is worth it before they fall in love with it. So here's what I'm bringing back: “speed to conviction matters more than speed to execution”. We're good at moving fast. The edge is getting to belief fast — and being willing to kill what we don't yet believe in. The second is personal. Michael reminded me of it without knowing he did. “Your success will be measured by the number of people you elevate”. Not your title, not your wins — the people. And you don't get to decide whether you elevated anyone. Other people decide that, on their own time, and they tell you when you least expect it — like in a ballroom, from someone whose cohort you once came back to share with. I’m grateful for all those who elevated me throughout my career. I still consider myself a student. I just happen to share a few chapters ahead of where I once was. To Dan Roberts for facilitating a masterclass I won't forget — and to the faculty I shared the stage with: I learned more than I shared. Grateful to have shared the stage with this industry thought leaders and faculty: Monica Caldas · Antonio Marin · Sumit Anand · Motti Finkelstein · Kathy Kay · @Dani Brown · Jennifer Hartsock · Deepa Soni · Mike Anderson Who elevated you recently — and have you told them? Foundry Events Allie Billington CIO Online
Experience & Education
Licenses & Certifications
Volunteer Experience
-
Organizer
Hadoop India Linkedin Group/ Meetup
Hadoop India linked in group.
Started first few Hadoop/Big data meet ups in India in year 2009-2010
Publications
-
Learning Apache Cassandra by Mat Brown
Packt Publishing
Excellent book on Learning Cassandra by Matt Brown. I was one of the reviewer for this book.
Other authors -
Cassandra Design Patterns
Packt Publishing
See publication-Learn how to identify real world use cases that Cassandra solves easily, in order to use it effectively
-Identify and apply usage and design patterns to solve specific business and technical problems including technologies that work in tandem with Cassandra
-A hands-on guide that will show you the strengths of the technology and help you apply Cassandra design patterns to data models -
The Cloud Is the Way to Go
Ulitzer.com
-
Cloud Computing for SMBs: A Level Playing Field
Ulitzer.com
See publicationCloud computing hits a particular sweet spot for SMBs. Cloud services offers the huge benefit of not having to deploy physical infrastructure. Cloud also ensures the ‘anywhere, anytime' availability of solutions. It means hassle-free collaboration between all the stakeholders of the business
Projects
-
Ankush
-
See projectOne of the cofounders and architect for a big data management tool. Ankush is a big data cluster management tool that creates and manages clusters of different technologies. Ankush is extensible and currently supports unified monitoring for all Hadoop vendor distributions (Hortonworks, Cloudera, MapR, Biginsights, Pivotal HD ) including Apache Hadoop, Cassandra, Storm, Kafka, Elastic Search.
Ankush helps you:
-Provision a Big Data cluster and its required ecosystem rapidly, easily, and…One of the cofounders and architect for a big data management tool. Ankush is a big data cluster management tool that creates and manages clusters of different technologies. Ankush is extensible and currently supports unified monitoring for all Hadoop vendor distributions (Hortonworks, Cloudera, MapR, Biginsights, Pivotal HD ) including Apache Hadoop, Cassandra, Storm, Kafka, Elastic Search.
Ankush helps you:
-Provision a Big Data cluster and its required ecosystem rapidly, easily, and productively
Manage multiple clusters and cluster activities for both local and cloud-based clusters
-Perform proactive monitoring and analytics
-Manage a polyglot Big Data infrastructure -
ETL/SQL Offloading Solution
-
Solution architect/cofounder for ETL/SQL Offloading Solution MPP/Columnar/RDBMS stores to Hadoop like big data solutions. The solution enables zero/minimal human intervention for offloading ETL and SQL processing and data from Teradata/Netezza/Greenplum/Vertica/Exadata etc into Hadoop platform
-
Jumbune - Hadoop Enterprise Profiler
-
See projectTechnical/Presales architect for Jumbune - an enterprise ready Hadoop profile that provides development & administrative insights of Hadoop based analytical solutions. It enables user to Debug, Profile, Monitor & Validate analytical solutions hosted on decoupled clusters.
-
kundera
-
See projectOne of the cofounders and architect for Kundera - a popular ORM for NoSQL stores.
The idea behind Kundera is to make working with NoSQL Databases drop-dead simple and fun. Kundera, which supports HBase, Cassandra, MongodDB, Redis, Neo4J, CouchDB, Oracle NoSQL, Elastic Search and all JDBC supporting RDBMS is being developed with following objectives:
-To make working with NoSQL as simple as working with SQL
-To serve as JPA Compliant mapping solution for NoSQL Datastores.
-To help…One of the cofounders and architect for Kundera - a popular ORM for NoSQL stores.
The idea behind Kundera is to make working with NoSQL Databases drop-dead simple and fun. Kundera, which supports HBase, Cassandra, MongodDB, Redis, Neo4J, CouchDB, Oracle NoSQL, Elastic Search and all JDBC supporting RDBMS is being developed with following objectives:
-To make working with NoSQL as simple as working with SQL
-To serve as JPA Compliant mapping solution for NoSQL Datastores.
-To help developers, forget the complexity of NoSQL stores and focus on Domain Model.
-To make switching across data-stores as easy as changing a configuration. -
StreamAnalytix
-
See projectTechnical/Presales architect for StreamAnalytix which is a software platform that enables enterprises to analyze and respond to events in real-time or near real-time at Big Data scale supporting millions of events per second. It is designed to rapidly build and deploy streaming analytics applications for any industry vertical, any data format, and any use-case.
Honors & Awards
-
HadoopHackDay Delhi Winners (Outside Prize)
http://barcamp.org/w/page/11385339/hadoophackdaydelhi
Hadoop for building decision making engine for Business Analysts
Languages
-
English
-
-
Russian
-
-
Hindi
-
Recommendations received
2 people have recommended Sanjay
Join now to viewOther similar profiles
Explore more posts
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content