Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GitHub Issue Exporter (Python)

A secure, Python-based web application for exporting GitHub issues to Excel with customizable field mappings and parent relationship tracking.

Features

  • 🔐 Secure Authentication - OAuth-based social sign-in
  • 🔒 Encrypted Token Storage - GitHub tokens encrypted with AES-256-GCM
  • 📊 Excel Export - Download issues as formatted Excel files
  • 🎯 Custom Field Mappings - Choose which issue fields to export
  • 🔗 Parent Relationships - Automatically resolve parent issue references
  • 👥 Multi-User Support - Each user has isolated configurations and credentials
  • ⚡ Fast & Modern - Built with FastAPI + HTMX for optimal performance

Technology Stack

  • Backend: FastAPI (Python 3.11+)
  • Frontend: HTMX + Tailwind CSS + Alpine.js
  • Database: MySQL/MariaDB with SQLAlchemy ORM
  • Security: AES-256-GCM encryption, JWT authentication
  • Excel Generation: openpyxl
  • Storage: AWS S3 (optional)

Prerequisites

  • Python 3.11 or higher
  • MySQL/MariaDB database
  • GitHub Personal Access Token with repo scope
  • OAuth provider credentials (for authentication)
  • AWS S3 credentials (optional, for file storage)

Installation

1. Clone the Repository

git clone https://github.com/mmockd/Gexport-python.git
cd Gexport-python

2. Create Virtual Environment

python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install Dependencies

pip install -r requirements.txt

4. Configure Environment Variables

Copy the example environment file and configure it:

cp .env.example .env

Edit .env and set your values:

# Database
DATABASE_URL=mysql+pymysql://user:password@localhost:3306/github_exporter

# Security (generate a secure random key)
SECRET_KEY=your-secret-key-min-32-characters-long

# OAuth (configure your OAuth provider)
OAUTH_CLIENT_ID=your-oauth-client-id
OAUTH_CLIENT_SECRET=your-oauth-client-secret
OAUTH_AUTHORIZE_URL=https://oauth-provider.com/authorize
OAUTH_TOKEN_URL=https://oauth-provider.com/token
OAUTH_USER_INFO_URL=https://oauth-provider.com/userinfo
OAUTH_REDIRECT_URI=http://localhost:8000/auth/callback

# AWS S3 (optional)
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
S3_BUCKET=github-exporter-files

5. Initialize Database

# Run Alembic migrations
alembic upgrade head

6. Run the Application

# Development mode
python -m app.main

# Or with uvicorn
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

The application will be available at http://localhost:8000

Usage Guide

1. Sign In

Navigate to the application and click "Sign In" to authenticate via OAuth.

2. Add GitHub Token

  1. Go to GitHub Settings → Tokens
  2. Create a new token with the repo scope
  3. Copy the token
  4. Navigate to Credentials page in the app
  5. Paste and save your token

Your token is encrypted using AES-256-GCM before storage.

3. Configure Repository

  1. Go to Configuration page
  2. Enter the repository owner and name (e.g., facebook/react)
  3. Click "Add Repository"
  4. The repository will be validated and set as active

4. Customize Field Mappings

  1. Go to Field Mappings page
  2. Enable/disable fields you want in your export
  3. Drag to reorder fields
  4. Available fields include:
    • Issue number, title, state
    • Created, updated, closed dates
    • Author, assignees, labels
    • Milestone, comments, URL
    • Parent issue information

5. Export Issues

  1. Go to Export page
  2. Choose issue state (open/closed/all)
  3. Enable "Resolve Parent Relationships" if needed
  4. Click "Preview Issues" to see what will be exported
  5. Click "Export to Excel" to download

Project Structure

gexport-python/
├── app/
│   ├── api/                  # API endpoints
│   │   ├── auth.py          # Authentication
│   │   ├── config.py        # Repository configuration
│   │   ├── credentials.py   # Token management
│   │   ├── fields.py        # Field mappings
│   │   └── export_routes.py # Export functionality
│   ├── core/                # Core utilities
│   │   ├── auth.py          # Auth dependencies
│   │   ├── config.py        # Settings
│   │   ├── database.py      # Database connection
│   │   ├── excel.py         # Excel generation
│   │   ├── github.py        # GitHub API integration
│   │   ├── security.py      # Encryption & JWT
│   │   └── storage.py       # S3 storage
│   ├── models/              # Database models
│   │   └── models.py        # SQLAlchemy models
│   ├── templates/           # HTML templates
│   │   ├── base.html
│   │   ├── index.html
│   │   ├── config.html
│   │   ├── credentials.html
│   │   ├── fields.html
│   │   └── export.html
│   ├── static/              # Static files
│   └── main.py              # FastAPI application
├── alembic/                 # Database migrations
├── tests/                   # Tests
├── requirements.txt         # Python dependencies
├── .env.example            # Environment template
└── README.md               # This file

Security

Token Encryption

GitHub Personal Access Tokens are encrypted before storage using:

  • Algorithm: AES-256-GCM (Galois/Counter Mode)
  • Key Derivation: PBKDF2-SHA256 with 100,000 iterations
  • Random Components: Unique salt and nonce per encryption
  • Master Key: Derived from SECRET_KEY environment variable

The encryption process:

  1. Generate random 16-byte salt
  2. Derive 256-bit encryption key using PBKDF2
  3. Generate random 12-byte nonce
  4. Encrypt token using AES-GCM
  5. Store: base64(salt + nonce + ciphertext + auth_tag)

Token Display

Tokens are never displayed in full. Only a masked version is shown:

  • Format: ghp_****...****wxyz
  • Shows first 4 and last 4 characters only

User Isolation

  • Each user's data is completely isolated
  • Configurations, credentials, and exports are user-specific
  • Database queries are filtered by user ID
  • No cross-user data access possible

Best Practices

  1. Use strong SECRET_KEY: Minimum 32 characters, randomly generated
  2. Enable HTTPS: In production, always use HTTPS
  3. Rotate tokens: Periodically regenerate GitHub tokens
  4. Limit token scope: Only grant necessary permissions
  5. Monitor access: Review export history regularly

API Endpoints

Authentication

  • GET /auth/login - Redirect to OAuth provider
  • GET /auth/callback - OAuth callback
  • POST /auth/logout - Sign out
  • GET /auth/me - Get current user

Configuration

  • GET /api/config/ - List configurations
  • POST /api/config/ - Create configuration
  • PUT /api/config/{id}/activate - Activate configuration
  • DELETE /api/config/{id} - Delete configuration
  • GET /api/config/active - Get active configuration

Credentials

  • GET /api/credentials/ - Get credential status
  • POST /api/credentials/ - Save token
  • DELETE /api/credentials/ - Delete token

Field Mappings

  • GET /api/fields/ - List field mappings
  • GET /api/fields/available - List available fields
  • PUT /api/fields/{id} - Update field mapping
  • POST /api/fields/bulk-update - Bulk update
  • POST /api/fields/reset - Reset to defaults

Export

  • POST /api/export/preview - Preview issues
  • POST /api/export/excel - Export to Excel
  • GET /api/export/history - Get export history

Testing

Run the test suite:

pytest

Run with coverage:

pytest --cov=app tests/

Development

Database Migrations

Create a new migration:

alembic revision --autogenerate -m "Description"

Apply migrations:

alembic upgrade head

Rollback migration:

alembic downgrade -1

Code Formatting

# Format with black
black app/ tests/

# Sort imports
isort app/ tests/

Deployment

Production Configuration

  1. Set DEBUG=False in .env
  2. Use a production-grade database
  3. Enable HTTPS
  4. Use a reverse proxy (nginx/Apache)
  5. Set up proper logging
  6. Configure S3 for file storage

Docker Deployment

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Troubleshooting

Database Connection Issues

# Test database connection
python -c "from app.core.database import engine; print(engine.connect())"

Token Validation Fails

  • Ensure token has repo scope
  • Check if token is expired
  • Verify token hasn't been revoked

OAuth Errors

  • Verify OAUTH_REDIRECT_URI matches your OAuth app configuration
  • Check client ID and secret are correct
  • Ensure OAuth provider URLs are accessible

Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests
  5. Submit a pull request

License

MIT License - see LICENSE file for details

Support

For issues or questions:

  • Open an issue on GitHub
  • Contact: [your-email]

Roadmap

  • Advanced filtering (labels, milestones, date ranges)
  • Scheduled exports
  • CSV export option
  • Custom parent relationship patterns
  • Bulk repository operations
  • Export templates
  • API rate limit handling
  • Webhook integration

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages