Karan Mittal

Amazing blog by Eloelo - India’s leading live social media app on how they built a unified batch + streaming “lakehouse” architecture - powering over 100M+ events per day. At the core of their data platform is "Apache Hudi." Here are some highlights: ✅ Adopted Change Data Capture (CDC) to stream DB changes into Apache Kafka, enabling near real-time ingestion without burdening production databases. ✅ Built a Bronze–Silver–Gold architecture, where Apache Hudi powers the Silver layer - enabling record-level upserts, schema evolution, and incremental ETL. ✅ Reduced query latency by 90% and cut Athena + RDS costs by over $35K/month by compacting data in Hudi and enabling efficient file pruning. ✅ Enabled interactive analytics and machine learning by exposing Hudi tables across Athena, Trino, and Spark - with fresh, query-optimized snapshots. ✅ Actively working to enable real-time Flink-to-Hudi ingestion via DeltaStreamer to converge streaming and batch pipelines even further. Blog Link 👉 https://lnkd.in/d89Y5G_i #dataengineering #softwareengineering

180

3 Comments

🧠 𝐍𝐞𝐰 𝐑𝐞𝐬𝐞𝐚𝐫𝐜𝐡 𝐀𝐧𝐧𝐨𝐮𝐧𝐜𝐞𝐦𝐞𝐧𝐭 We’re proud to announce the publication of our latest research paper: 𝑰𝒏𝒕𝒆𝒓𝒑𝒓𝒆𝒕𝒂𝒃𝒊𝒍𝒊𝒕𝒚-𝑨𝒘𝒂𝒓𝒆 𝑷𝒓𝒖𝒏𝒊𝒏𝒈 𝒇𝒐𝒓 𝑬𝒇𝒇𝒊𝒄𝒊𝒆𝒏𝒕 𝑴𝒆𝒅𝒊𝒄𝒂𝒍 𝑰𝒎𝒂𝒈𝒆 𝑨𝒏𝒂𝒍𝒚𝒔𝒊𝒔 Developed by the AryaXAI Alignment Lab, Mumbai, this work introduces a novel framework that combines model compression with interpretability—two critical factors for deploying AI in healthcare. 𝐊𝐞𝐲 𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬: – Uses attribution methods like DL-Backtrace, LRP, and Integrated Gradients to guide pruning – Achieves 40–85% reduction in model size with minimal accuracy loss – Tested across real-world datasets: MURA, KVASIR, CPN, Fetal Planes – Preserves clinically meaningful features for trustworthy diagnostics 𝐋𝐢𝐧𝐤 𝐨𝐟 𝐭𝐡𝐞 𝐫𝐞𝐬𝐞𝐚𝐫𝐜𝐡 𝐩𝐚𝐩𝐞𝐫 𝐢𝐧 𝐭𝐡𝐞 𝐟𝐢𝐫𝐬𝐭 𝐜𝐨𝐦𝐦𝐞𝐧𝐭 𝐛𝐞𝐥𝐨𝐰 👇 Vinay Kumar Sankarapu | Pratinav Seth | Nikita Malik | Arya.ai - An Aurionpro Company #Arya.ai #Aurionpro #AryaXAI #AIReserach #MedicalAI #XAI #DeepLearning #HealthcareAI

45

2 Comments

How do you extract structured data from messy, unstructured clinical notes? This article by Parul Pandey on Google for Developers LangExtract explains a three-step workflow that uses LLMs to identify medications, dosages, and adverse reactions.

8

𝗔𝘁𝘁𝗲𝗻𝗱𝗶𝗻𝗴 𝘁𝗵𝗲 #testmuconf 𝗖𝗼𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 2025 LambdaTest 𝗛𝗲𝗿𝗲’𝘀 𝗪𝗵𝗮𝘁 𝗜 𝗟𝗲𝗮𝗿𝗻𝗲𝗱 𝗔𝗯𝗼𝘂𝘁 𝗔𝗜’𝘀 𝗘𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻 from Siddhant Wadhwani What a thrill to dive into the future of AI at the #testmuconf Conference 2025! The theme Generative to Agentic to Quantum – The Evolution of AI was a masterclass in how AI is reshaping industries—especially software testing and automation. Here are my top takeaways:  2025: 𝗧𝗵𝗲 𝗜𝗻𝗳𝗹𝗲𝗰𝘁𝗶𝗼𝗻 𝗣𝗼𝗶𝗻𝘁 𝗳𝗼𝗿 𝗔𝗜 2025 marks a pivotal shift where AI transitions from outputs (e.g., generating text/code) to outcomes (solving real-world problems). We’re moving from generative AI (helping humans) → agentic AI (doing tasks independently) → quantum AI (discovering new solutions). This means testing and automation will rely on AI that plans, executes, and adapts—not just reacts.    GPT-5 & Beyond: The New AI Frontier The launch of GPT-5 is a game-changer:  -Multi-step reasoning: Solves complex problems with fewer hallucinations.   -Unified model family: Handles text, images, code, and more in one system.  Enterprises are already piloting autonomous developer/QA agents that plan, execute, and validate work with minimal human input. Older models are being phased out—making today’s AI far more reliable for business-critical workflows.  LLMs, Agentic AI, & Quantum: What’s Next. -LLMs: Expect a wave of new releases in 2025 (GPT-4.5, Claude 4, Llama 5, Gemini 3, and more!). These models will be larger, more specialized, and better at domain-specific tasks.  - Agentic AI: Think AI agents that act autonomously setting goals, reasoning, and learning from feedback. The Agentic AI Loop (Set Goal → Plan → Observe → Learn) is revolutionizing how we build intelligent systems.  - Quantum AI: Solves legacy AI pain points (resource intensity, optimization bottlenecks, lack of transparency). Tools like Google’s Willow chip and Microsoft’s Majorana 1 are pushing boundaries—enabling AI to tackle problems classical systems can’t.  Gen AI in Testing & Automation: The PALI Framework For testers and automation engineers, Gen AI is a force multiplier:  - Predictive analytics optimize testing processes.   - AI analyzes test results to predict defects.   - leads to more efficient testing cycles.   - Intelligent insights improve decision-making.    𝗙𝗶𝗻𝗮𝗹 𝗧𝗵𝗼𝘂𝗴𝗵𝘁: The line between human work and AI work is blurring and that’s exciting! As AI evolves from helper to doer to discoverer, our roles in testing and automation will shift toward guiding and validating AI systems. The future isn’t just about what AI can do—it’s about how we collaborate with it.  What’s your take on AI’s evolution? Drop your thoughts below!  #AIevolution #GenerativeAI #AgenticAI #testmuconf #testmuconf2025 #TestMuConf #lambdatest

47

27 Comments

Came across this well-organized repository of machine learning projects that covers a broad range of practical applications — from medical imaging and human activity recognition to NLP-based chatbots and image processing. The collection projects in: ➜ Brain tumor detection from MRI scans ➜ Driver distraction and drowsiness detection ➜ Diabetes and heart disease prediction ➜ Human activity recognition with LSTM ➜ Emotion-based emoji generation ➜ Room booking and medical chatbots using NLP ➜ Image colorization, age/gender prediction, and more Projects use familiar tools like scikit-learn, TensorFlow, PyTorch, OpenCV, and are great for exploring hands-on ML implementations.

49

5 Comments

🚀 Unlock Advanced AI Capabilities with Snowflake Cortex! Snowflake Cortex brings a suite of AI services directly into your Data Cloud environment, enabling seamless integration of advanced AI functionalities. Let's explore its key offerings and their practical applications: 1. 𝗖𝗼𝗿𝘁𝗲𝘅 𝗟𝗟𝗠 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀 Access powerful Large Language Models (LLMs) within Snowflake to perform tasks like text completion, summarization, sentiment analysis, and translation—all without managing external infrastructure. 𝘜𝘴𝘦 𝘊𝘢𝘴𝘦𝘴: - 𝗦𝗨𝗠𝗠𝗔𝗥𝗜𝗭𝗘: Quickly distill lengthy documents into concise summaries for efficient consumption. - 𝗦𝗘𝗡𝗧𝗜𝗠𝗘𝗡𝗧: Analyze customer feedback to understand sentiment trends and inform strategic decision. - 𝗧𝗥𝗔𝗡𝗦𝗟𝗔𝗧𝗘: Facilitate multilingual communication by translating content across various language. 𝟮. 𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗴𝗲𝗻𝘁𝘀 Orchestrate tasks by integrating multiple tools and data source. Cortex Agents plan and execute workflows, utilizing services like Cortex Analyst and Cortex Search to deliver accurate, context-aware response. 𝘜𝘴𝘦 𝘊𝘢𝘴𝘦𝘴: - 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗖𝗵𝗮𝘁𝗯𝗼𝘁𝘀: Develop AI-driven conversational agents capable of answering complex business queries. - 𝗔𝘂𝘁��𝗺𝗮𝘁𝗲𝗱 𝗥𝗲𝗽𝗼𝗿𝘁𝗶𝗻𝗴: Generate comprehensive reports by aggregating data from structured and unstructured sources. 𝟯. 𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗻𝗮𝗹𝘆𝘀𝘁 Empower users to query structured data in Snowflake using natural language. Cortex Analyst translates business questions into SQL queries, providing accurate answers without requiring SQL expertise. 𝘜𝘴𝘦 𝘊𝘢𝘴𝘦𝘴: - 𝗕𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲: Enable business users to extract insights from data warehouses by posing questions in plain language. 𝟰. 𝗖𝗼𝗿𝘁𝗲𝘅 𝗦𝗲𝗮𝗿𝗰𝗵 Combine semantic and keyword-based methods to enhance document retrieval within Snowflake. Cortex Search supports retrieval-augmented generation (RAG), enabling LLMs to provide answers grounded in your proprietary data. 𝘜𝘴𝘦 𝘊𝘢𝘴𝘦𝘴: - 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗦𝗲𝗮𝗿𝗰𝗵: Implement robust search functionalities within applications, allowing users to efficiently find relevant documents and information. --- Please do share your experience working with Snowflake Cortex AI! #AI #MachineLearning #SnowflakeCortex #DataCloud #BusinessIntelligence #EnterpriseAI 

51

📊 Evaluating chunking strategies with Agno, Qdrant, Ragas, and LlamaIndex Chunking changes everything in RAG. This project by Kameshwara Pavan Kumar benchmarks 4 strategies: Fixed, Semantic, Agentic (LLM-guided), and Recursive, across real evaluation metrics: → Faithfulness → Context Relevance → Factual Correctness → Context Recall → Utilization It’s a full test bench for retrieval pipelines, with side-by-side comparisons of how chunking impacts performance. 🚀 👉 Read the full writeup: https://lnkd.in/dFghY2-x

37

2 Comments

𝐀𝐈 𝐀𝐥𝐨𝐧𝐞 𝐖𝐨𝐧’𝐭 𝐅𝐢𝐱 𝐇𝐞𝐚𝐥𝐭𝐡𝐜𝐚𝐫𝐞 — 𝐂𝐨𝐧𝐭𝐞𝐱𝐭 𝐖𝐢𝐥𝐥 “Technology does not replace people; it changes what people are able to do.” — Abhijeet Vaidya, Head of Data & AI Practice, Emids In this deep dive, Abhijeet Vaidya explains how healthcare is entering a new era where AI works alongside clinicians, enhances longitudinal medical records, removes system friction, and amplifies human capability. From rural diagnostics to ambient clinical documentation and Emids’ PACCA AI platform, the future of care will be shaped not by algorithms alone — but by the context in which they operate. Read more: https://lnkd.in/gDXYupqy #CIOandLeader #HealthcareAI #DigitalHealth #AIinHealthcare #HealthTech #Emids #DataAndAI #FutureOfCare Vikas Gupta | Sachin Mhashilkar | Vandana Chauhan | Giridhar Rajagopalan | Jatinder S. | Hafeez Shaikh | Rajiv Pathak | Shokeen Saifi | Ekta Srivastav | Sourabh Dixit | Subhadeep Sen | Jagrati Rakheja | Musharrat Shahin | Aanchal G.

138

2 Comments

In her latest blog, Anamika Krishanpal, PhD, explains why AI-driven discovery can stall during the data preprocessing stage, even before models are deployed. Manual curation consumes 60–80% of expert time and stretches harmonization to 6–8 weeks. An agentic AI approach changes the equation: compressing preprocessing to under 48 hours, improving data fidelity, and unlocking 15–30% R&D productivity gains with auditable, human-in-the-loop control. The outcome: AI-ready multi-omics datasets, faster discovery cycles, and scientists focused on insight, not cleanup. To learn more, read the full blog: https://lnkd.in/gdAMqtQW

32

1 Comment

🚀 Demystifying How Large Language Models (LLMs) Actually Work We often hear about pre-training, fine-tuning, and reinforcement learning when it comes to LLMs — but what do these stages really mean? Let’s break it down in simple terms: 🔹 Pre-training Think of this as the foundation. We have two files: A run file (small, with key instructions). A parameter file (huge, compressed, requiring GPUs to process). This stage teaches the LLM the structure of language by exposing it to massive datasets. 🔹 Fine-tuning Now we feed the LLM questions and answers. This allows it to specialize and learn from more focused examples. 🔹 Reinforcement Learning Here’s where feedback loops in: We ask the LLM questions. It generates answers. We rate those answers, and over time, the LLM gets better at giving responses that align with human expectations. 🧠 Why Transformers Matter At the core of all this is the transformer architecture — a neural network that makes calculations based on weights (numbers). Since text is messy, we break it into tokens (word pieces). Every prediction is basically the model calculating: “What’s the most likely next word?” But there’s a catch — once the token limit is reached, the LLM forgets older context. That’s why asking the right questions becomes so important which is called prompt engineering. ✨ Key Takeaway LLMs aren’t magic — they’re math, architecture, and feedback loops working together. The quality of questions we ask directly shapes the quality of answers we get. 👉 What do you think: is better data or better questions more important for improving LLMs? Pls share, like or comment to make sure the post reaches as many people as possible :-).

13

2 Comments

𝐈𝐧𝐝𝐢𝐚 𝐁𝐞𝐭𝐬 𝐨𝐧 𝐇𝐨𝐦𝐞𝐠𝐫𝐨𝐰𝐧 𝐀𝐈 𝐭𝐨 𝐏𝐨𝐰𝐞𝐫 𝐇𝐞𝐚𝐥𝐭𝐡𝐜𝐚𝐫𝐞 𝐁𝐫𝐞𝐚𝐤𝐭𝐡𝐫𝐨𝐮𝐠𝐡𝐬 Under the ₹10,371 crore IndiaAI Mission, eight foundational AI projects are underway, with two directly targeting medicine. IntelliHealth is building a 20B-parameter model for EEG analysis to spot neurological disorders early. Fractal Analytics is creating a 70B-parameter reasoning model to tackle complex medical problem-solving. Other projects like BharatGen, Genloop, and Avatar AI will also shape healthcare, from multilingual patient communication to advanced diagnostics. Read More: https://lnkd.in/gB6c-RdP #IndiaAI #HealthTech #DigitalHealth #ArtificialIntelligence #HealthcareInnovation

20

TIME & Statista recently released their “World’s Top HealthTech Companies of 2025” report. Here are my observations 🔍 𝗪𝗵𝗮𝘁’𝘀 𝗥𝗶𝘀𝗶𝗻𝗴 𝟭. 𝗔𝗜 & 𝗗𝗮𝘁𝗮 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 𝗮𝗿𝗲 𝘁𝗵𝗲 𝗳𝗿𝗼𝗻𝘁-𝗿𝘂𝗻𝗻𝗲𝗿𝘀. Firms leading in AI/data are not just getting noticed - they’re dominating. From screening to diagnostics, matching treatments, innovations here are what TIME marks as “most impactful.” 𝟮. 𝗗𝗶𝗮𝗴𝗻𝗼𝘀𝘁𝗶𝗰𝘀 𝘁𝗵𝗿𝗼𝘂𝗴𝗵 𝘂𝗻𝗰𝗼𝗻𝘃𝗲𝗻𝘁𝗶𝗼𝗻𝗮𝗹 𝗺𝗲𝗮𝗻𝘀. Companies like Fedo (selfies to assess vitals), Canary Speech (voice-based detection) are pushing boundaries.  𝟯. 𝗪𝗲𝗮𝗿𝗮𝗯𝗹𝗲𝘀 & 𝗧𝗲𝗹𝗲𝗵𝗲𝗮𝗹𝘁𝗵 𝗱𝗲𝗺𝗮𝗻𝗱 𝗶𝘀 𝗻𝗼𝗻-𝗻𝗲𝗴𝗼𝘁𝗶𝗮𝗯𝗹𝗲 We have been trained to be demanding consumers. Patients today expect transparency and instant access. These companies are delivering just that. ⚠ 𝗪𝗵𝗮𝘁’𝘀 𝗟𝗮𝗴𝗴𝗶𝗻𝗴 / 𝗥𝗶𝘀𝗸𝘆 𝟭. 𝗣𝗿𝗲𝘃𝗲𝗻𝘁𝗶𝗼𝗻 𝗶𝘀 𝘀𝘁𝗶𝗹𝗹 𝘂𝗻𝗱𝗲𝗿𝗽𝗿𝗶𝗼𝗿𝗶𝘁𝗶𝘇𝗲𝗱. Despite its obvious potential for cost-savings and improving population health, prevention is not getting equal attention. Innovations here aren’t scaling fast. Why? Maybe because systems (and incentives) tend to reward curative measures over preventive ones.  𝟮. 𝗔𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝘃𝘀. 𝗮𝗰𝗰𝗲𝘀𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 𝗶𝘀 𝘀𝘁𝗶𝗹𝗹 𝗮 𝘁𝗲𝗻𝘀𝗶𝗼𝗻. Diagnostic tools are powerful but often expensive or resource-intensive. Scaling them globally (especially in low-resource settings) remains challenging. Without bridging that gap, health inequities could worsen. 💡 𝗪𝗵𝗮𝘁 𝗟𝗲𝗮𝗱𝗲𝗿𝘀/𝗜𝗻𝗻𝗼𝘃𝗮𝘁𝗼𝗿𝘀 𝗦𝗵𝗼𝘂𝗹𝗱 𝗗𝗼 𝟭. 𝗜𝗻𝘃𝗲𝘀𝘁 𝗶𝗻 𝗮𝗻𝗱 𝗿𝗲𝘄𝗮𝗿𝗱 𝗽𝗿𝗲𝘃𝗲𝗻𝘁𝗶𝗼𝗻. Health systems, governments, insurers must fix their incentives. We must reward keeping people healthy, not just treating them when things go wrong. 𝟮. 𝗙𝗼𝗰𝘂𝘀 𝗼𝗻 𝗱𝗲𝗺𝗼𝗰𝗿𝗮𝘁𝗶𝘇𝗮𝘁𝗶𝗼𝗻. Tools shouldn’t just help those living in urban centers. The most insight (and impact) comes when diagnostics, wearables, telehealth are accessible everywhere. 🔮 𝗕𝗶𝗴 𝗣𝗶𝗰𝘁𝘂𝗿𝗲: 𝗪𝗵𝗮𝘁’𝘀 𝗖𝗼𝗺𝗶𝗻𝗴 1. More voice-based diagnostics (speech patterns, tone, and linguistic features) could flag health issues before symptoms are obvious. 2. Wearables predicting - not just tracking. Machine learning models integrated with wearables could anticipate changes in health status triggering early interventions. 3. Hybrid care models merging telehealth + local physical care + AI-driven diagnostics will become the norm. The pure in-person model is already being disrupted. ✅ 𝗜𝗻 𝗦𝘂𝗺 TIME’s list is more than a roster of winners; it’s a preview of where health tech is headed. Would love to hear your take: which of these trends do you think will have the biggest impact over the next 2-3 years. Where do you see the biggest risk of being left behind? #HealthTech #DigitalHealth #Innovation #AI #PreventiveCare #Diagnostics #Leadership

62

3 Comments

Sarvam-M: A 24B Parameter Multilingual AI Model is finetuned Mistral-Small with Indic language desi nuances covering 11 major languages from india 🇮🇳 Key Features: • Dual-mode interface: Quick "non-think" responses + detailed "think" mode for complex reasoning 🧠 • Strong performance on math (GSM-8K) and coding (SWE-Bench) benchmarks 📊 • Supports both native Indic scripts and romanized text ✍️ • Built for real-world multilingual conversational agents 💬 I think it's a good start , instead of training model from scratch Sarvam focused on building indic dataset and model like in video shows good translingual conversational capability . Try it out via the Colab link - https://lnkd.in/gzcW4hru Playground - https://lnkd.in/gXijZrkY

27

1 Comment

🧠 Fine-Tuning vs Instruction Tuning — What’s the Real Difference? As teams dive deeper into customizing LLMs, one question keeps coming up: “Should we fine-tune the model or instruction-tune it?” Here’s a quick visual to make the difference super clear 👇 📸 Save this if you’re working on GenAI systems in healthcare, enterprise, or research! 🔧 Fine-Tuning • Focused on a specific task (e.g., ICD code prediction) • Needs labeled data • Narrow, domain-specific scope • More compute, longer training • Great when high precision is needed in a niche domain 📘 Instruction Tuning • Teaches the model to follow general instructions across tasks • Uses instruction–response pairs • Boosts performance for unseen prompts • Broadens the model’s skillset (e.g., summarize, translate, classify) • Great for building general-purpose assistants and zero-shot use cases 🧠 Both update model weights, but for very different goals. 💡 Want to build smarter LLM apps? → Use fine-tuning when your task is specific and critical → Use instruction tuning to make the model more adaptable across tasks #GenAI #LLMs #FineTuning #InstructionTuning #AIProduct #PromptEngineering #MachineLearning #OpenSourceModels #AIInHealthcare #RAG #AIExplained

29

The success of value-based care isn’t just about collecting healthcare data, it’s about turning it into clinically aligned, AI-powered decisions that improve patient outcomes and cut costs. From unifying fragmented data to predicting patient risk and enabling real-time care interventions, the Tredence VBC Accelerator Suite delivers clarity and action where it matters most. A platform trusted to process 100M+ medical encounters, cover 7M+ lives, and drive up to 20% reduction in care costs, all while enabling smarter, faster, more impactful care delivery. Watch how Unify. Predict. Care. comes to life:  https://lnkd.in/gdee2fV6 Prashant Sareen | PradeepKumar Jain Vimalraj | Nick Kane | Kaustubh Kane #Tredence #ValueBasedCare #BeyondPossible

30

#BlogArticle - What Is The Role Of Data Science In Healthcare - https://lnkd.in/dx7bnrAq 10 Real Ways Data Science is Saving Lives ❗ 🚀 Key breakthroughs covered: ➝ Virtual assistants reducing clinical overload ➝ Genome analytics enabling personalized treatments ➝ Medical image AI detecting cancer earlier than radiologists ➝ How disease forecasting models inform national policies ➝ Use cases in real-time global response coordination Big Data Solutions now carries a huge impact and is not optional anymore. Why? Dig deeper into Victoria Yaskevich's article - https://lnkd.in/dx7bnrAq #PublicHealthTech #BigData #PersonalizedMedicine #Genomics #Elinext #DataDrivenCare #DigitalHealth #AIHealth #HealthTech #DataScience #HealthcareAnalytics

46