Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

πŸ“¬ Daily Gmail Digest

A lightweight, fully autonomous automation service that connects multiple Gmail accounts, fetches unread emails daily, filters out noise, and delivers a clean aggregated digest to Telegram and Discord β€” completely hands-free.

Deployed on Render with MongoDB Atlas for persistent cloud storage and cron-job.org for external scheduling.


✨ Features

  • Multi-Account Support β€” Connect multiple Gmail accounts via Google OAuth 2.0
  • Daily Automated Digest β€” Runs via external cron (production) or internal scheduler (development)
  • Deterministic Filtering β€” Strips out promotions, social updates, newsletters, and spam without AI/NLP
  • Dual Delivery β€” Sends digest to both Telegram and Discord simultaneously
  • Rich Discord Embeds β€” Each account gets a color-coded embed card for instant visual differentiation
  • Telegram HTML Formatting β€” Clean, structured digest with emoji and grouped sections
  • Failure Isolation β€” If one delivery channel fails, the other still works
  • Retry Logic β€” Automatic retry with 3-second delay on delivery failures
  • Protected API β€” /digest/run endpoint secured with Bearer token authentication
  • Cloud Persistence β€” MongoDB Atlas ensures data survives container restarts and redeploys
  • Run History β€” Track past digest executions via the API
  • Graceful Error Handling β€” Partial failures, expired tokens, and empty inboxes are all handled cleanly

πŸ—οΈ Architecture

The system operates as 5 autonomous agents coordinating through a pipeline:

# Agent Role Module
1 Orchestrator Timekeeper & process manager src/scheduler/cron.ts
2 Fetcher Gmail API data retriever src/gmail/fetcher.ts
3 Bouncer Deterministic noise filter src/gmail/filter.ts
4 Editor Digest formatter (Telegram + Discord) src/digest/formatter.ts
5 Courier Delivery to Telegram & Discord src/telegram/sender.ts, src/discord/sender.ts

Pipeline Flow

Cron Trigger (external or internal)
  β†’ Load Gmail accounts from MongoDB
  β†’ For each account: Authenticate β†’ Fetch unread emails β†’ Filter noise
  β†’ Format digest (Telegram HTML + Discord Embeds)
  β†’ Deliver to Discord
  β†’ Deliver to Telegram
  β†’ Log run result to MongoDB

Production Stack

Component Technology
Runtime Node.js + TypeScript
Database MongoDB Atlas (cloud)
Hosting Render (free tier)
Scheduling cron-job.org (external)
Delivery Telegram Bot API + Discord Webhooks

πŸ“ Project Structure

daily-gmail-digest/
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ auth/              # Google OAuth 2.0 flows
β”‚   β”‚   β”œβ”€β”€ oauth.ts       # OAuth client, auth URL, token exchange & refresh
β”‚   β”‚   └── routes.ts      # Express routes: /auth/google, /auth/google/callback
β”‚   β”œβ”€β”€ config/            # Environment variable validation
β”‚   β”‚   └── index.ts       # Typed config with startup validation
β”‚   β”œβ”€β”€ db/                # Database layer
β”‚   β”‚   β”œβ”€β”€ client.ts      # MongoDB connection via mongoose
β”‚   β”‚   └── schema.ts      # Mongoose models (GmailAccount, DigestRun)
β”‚   β”œβ”€β”€ digest/            # Digest formatting
β”‚   β”‚   └── formatter.ts   # Telegram (HTML) and Discord (embeds) formatters
β”‚   β”œβ”€β”€ discord/           # Discord delivery
β”‚   β”‚   └── sender.ts      # Webhook-based delivery with retry & embed batching
β”‚   β”œβ”€β”€ gmail/             # Gmail integration
β”‚   β”‚   β”œβ”€β”€ fetcher.ts     # Fetches unread emails via Gmail API
β”‚   β”‚   └── filter.ts      # Deterministic spam/promo filtering
β”‚   β”œβ”€β”€ scheduler/         # Cron orchestration
β”‚   β”‚   └── cron.ts        # Daily pipeline coordinator
β”‚   β”œβ”€β”€ telegram/          # Telegram delivery
β”‚   β”‚   └── sender.ts      # Bot API delivery with retry & message chunking
β”‚   β”œβ”€β”€ utils/             # Shared utilities
β”‚   β”‚   └── logger.ts      # Structured logging with agent tags
β”‚   └── server.ts          # Main entry point (Express + MongoDB + cron startup)
β”œβ”€β”€ docs/                  # Project planning documents
β”‚   β”œβ”€β”€ AGENTS.md
β”‚   β”œβ”€β”€ ARCHITECTURE.md
β”‚   β”œβ”€β”€ IDEA.md
β”‚   β”œβ”€β”€ IMPLEMENTATION.md
β”‚   └── PLAN.md
β”œβ”€β”€ .env.example           # Template for environment variables
β”œβ”€β”€ .gitignore
β”œβ”€β”€ LICENSE
β”œβ”€β”€ package.json
β”œβ”€β”€ package-lock.json
β”œβ”€β”€ tsconfig.json
└── README.md

πŸš€ Getting Started

Prerequisites

  • Node.js v18+ (required for native fetch support)
  • npm v8+
  • A Google Cloud Project with Gmail API enabled
  • A MongoDB Atlas cluster (free tier available)
  • A Telegram Bot (created via @BotFather)
  • A Discord Webhook URL (optional β€” created in Discord channel settings)

1. Clone the Repository

git clone https://github.com/DevDebpriyo/Daily-Digest.git
cd Daily-Digest

2. Install Dependencies

npm install

3. Configure Environment Variables

cp .env.example .env

Edit .env and fill in your credentials:

# Google OAuth 2.0 Credentials
GOOGLE_CLIENT_ID=your_client_id
GOOGLE_CLIENT_SECRET=your_client_secret
GOOGLE_REDIRECT_URI=http://localhost:3000/auth/google/callback

# Telegram Bot
TELEGRAM_BOT_TOKEN=your_telegram_bot_token
TELEGRAM_CHAT_ID=your_telegram_chat_id

# Discord Webhook (optional β€” skip to disable Discord delivery)
DISCORD_WEBHOOK_URL=your_discord_webhook_url

# MongoDB Atlas
MONGODB_URI=mongodb+srv://user:password@cluster.mongodb.net/daily-digest

# Scheduler (cron expression β€” default: every day at 6:00 AM)
CRON_SCHEDULE=0 6 * * *

# Server
PORT=3000

# Internal / External CRON
ENABLE_INTERNAL_CRON=true  # true -> local development, false -> production

# Secret token for protecting the /digest/run endpoint
CRON_SECRET=my_super_secret_key

4. Build & Start

# Build TypeScript β†’ JavaScript
npm run build

# Start the production server
npm start

Or run directly in development mode:

npm run dev

5. Connect Gmail Accounts

  1. Open your browser and navigate to: http://localhost:3000/auth/google
  2. Sign in with your Google account and grant Gmail read-only access
  3. You'll receive a confirmation with the connected email address
  4. Repeat for each Gmail account you want to monitor

πŸ”‘ Setup Guides

Google Cloud Setup

  1. Go to the Google Cloud Console
  2. Create a new project (or select an existing one)
  3. Enable the Gmail API under APIs & Services β†’ Library
  4. Go to APIs & Services β†’ Credentials
  5. Click Create Credentials β†’ OAuth 2.0 Client ID
  6. Set Application type to Web application
  7. Add http://localhost:3000/auth/google/callback as an Authorized redirect URI
  8. Copy the Client ID and Client Secret into your .env file

MongoDB Atlas Setup

  1. Go to MongoDB Atlas and create a free account
  2. Create a new Shared Cluster (M0 free tier works fine)
  3. Go to Database Access β†’ Add a database user with read/write permissions
  4. Go to Network Access β†’ Add 0.0.0.0/0 to allow connections from anywhere (required for Render)
  5. Go to Database β†’ Click Connect β†’ Choose Connect your application
  6. Copy the connection string and paste it into your .env as MONGODB_URI
  7. Replace <password> with your database user's password

Telegram Bot Setup

  1. Open Telegram and message @BotFather
  2. Send /newbot and follow the prompts to create your bot
  3. Copy the bot token into your .env as TELEGRAM_BOT_TOKEN
  4. To get your Chat ID:
    • Message your bot, then visit: https://api.telegram.org/bot<YOUR_TOKEN>/getUpdates
    • Find the chat.id value in the response
  5. Copy the chat ID into your .env as TELEGRAM_CHAT_ID

Discord Webhook Setup

  1. Open Discord and go to the channel where you want digests delivered
  2. Click the gear icon (Edit Channel) β†’ Integrations β†’ Webhooks
  3. Click New Webhook, name it (e.g., "Daily Digest"), and copy the Webhook URL
  4. Paste the URL into your .env as DISCORD_WEBHOOK_URL

Note: Discord delivery is optional. If DISCORD_WEBHOOK_URL is not set, the system will skip Discord and still deliver to Telegram.


πŸ“‘ API Endpoints

Method Endpoint Description
GET / Health check β€” returns service status
GET /auth/google Initiates Google OAuth flow (redirects to Google)
GET /auth/google/callback OAuth callback β€” exchanges code for tokens
GET /auth/accounts Lists all connected Gmail accounts
DELETE /auth/accounts/:id Removes a connected Gmail account
POST /digest/run Manually triggers the digest pipeline (requires Authorization: Bearer <CRON_SECRET> header)
GET /digest/history Returns the last 20 digest run records

Quick Manual Test

# Trigger a digest run manually (requires CRON_SECRET)
curl -X POST http://localhost:3000/digest/run \
  -H "Authorization: Bearer my_super_secret_key"

# Check run history
curl http://localhost:3000/digest/history

# List connected accounts
curl http://localhost:3000/auth/accounts

πŸ—ƒοΈ Database

MongoDB Atlas is used for cloud persistence with the following collections:

Collection Purpose
gmailaccounts Connected Gmail accounts with OAuth refresh tokens
digestruns Audit log of every pipeline execution (time, status, details)

Data persists permanently in the cloud β€” container restarts and redeploys on Render do not affect stored data.


πŸ”§ Configuration

Cron Scheduling

The application supports two scheduling modes:

Mode ENABLE_INTERNAL_CRON How It Works
Production false External service (e.g., cron-job.org) calls POST /digest/run with the CRON_SECRET
Development true Internal node-cron runs on the CRON_SCHEDULE expression

The CRON_SCHEDULE environment variable accepts standard cron expressions:

Expression Schedule
0 6 * * * Every day at 6:00 AM (default)
0 8 * * * Every day at 8:00 AM
0 */6 * * * Every 6 hours
*/30 * * * * Every 30 minutes (for testing)

Filtering Rules

The Bouncer agent filters emails using deterministic rules β€” no AI required:

  1. Gmail Categories β€” Excludes CATEGORY_PROMOTIONS, CATEGORY_SOCIAL, CATEGORY_FORUMS
  2. Label IDs β€” Excludes SPAM and TRASH
  3. Keyword Patterns β€” Filters senders/subjects containing: unsubscribe, no-reply, noreply, newsletter, notification@, updates@, marketing@, promo@, bulk@, mailer-daemon

⚠️ Error Handling

Scenario Behavior
Gmail auth failure for one account Skips the failed account, processes remaining, notes error in digest
All Gmail accounts fail Sends a failure notification to Telegram & Discord
Telegram delivery fails Retries once after 3 seconds; logs failure if retry also fails
Discord delivery fails Retries once after 3 seconds; logged only, never crashes pipeline
Empty inbox (no new emails) Sends a "No new emails today" confirmation digest
Missing Discord webhook Silently skips Discord delivery, Telegram still works
Long message exceeds limits Telegram: auto-splits at 4096 chars; Discord: batches at 10 embeds
MongoDB connection fails Application logs error and exits β€” will be restarted by Render

🚒 Deployment

Render (Recommended)

  1. Connect your GitHub repository to Render
  2. Create a new Web Service
  3. Set Build Command: npm install && npm run build
  4. Set Start Command: npm start
  5. Add all environment variables in the Render dashboard
  6. Set ENABLE_INTERNAL_CRON=false (use external cron)

External Cron Setup (cron-job.org)

  1. Go to cron-job.org and create a free account
  2. Create a new cron job:
    • URL: https://your-app.onrender.com/digest/run
    • Method: POST
    • Schedule: Your desired frequency (e.g., daily at 6:00 AM)
    • Headers: Add Authorization: Bearer <your_CRON_SECRET>
  3. Save and enable the job

Why external cron? Render free tier instances sleep after inactivity. External cron ensures the digest runs reliably even if the instance was sleeping.

Deploy Checklist

  1. βœ… MongoDB Atlas cluster created and configured
  2. βœ… All environment variables set on Render
  3. βœ… ENABLE_INTERNAL_CRON=false for production
  4. βœ… External cron job configured with correct URL and Bearer token
  5. βœ… Gmail accounts connected via /auth/google
  6. βœ… Verify health: curl https://your-app.onrender.com/

πŸ›‘οΈ Security

  • OAuth Tokens β€” Refresh tokens are stored in MongoDB Atlas; never exposed to any frontend
  • Secrets β€” All credentials live in .env (local) or platform env vars (production), excluded from version control via .gitignore
  • Gmail Scope β€” Only gmail.readonly is requested β€” the app cannot send, delete, or modify emails
  • Endpoint Protection β€” /digest/run is protected by Bearer token authentication
  • No External Dependencies for HTTP β€” Telegram and Discord delivery use native fetch β€” no third-party HTTP libraries

πŸ“œ License

This project is licensed under the MIT License.

About

A modular cron-based Gmail aggregation system using Google OAuth2, Gmail API, Telegram Bot API, Discord Webhooks, and MongoDB Atlas for fully automated daily inbox digests.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages