Merge branch 'main' into copilot/fix-password-length-issue

This commit is contained in:
Christian Krakau-Louis
2026-03-23 13:21:20 +01:00
committed by GitHub
30 changed files with 2083 additions and 331 deletions
+22
View File
@@ -0,0 +1,22 @@
# EditorConfig — https://editorconfig.org
root = true
[*]
indent_style = space
indent_size = 4
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
[*.{js,jsx,ts,tsx,json,css,scss,yml,yaml}]
indent_size = 2
[*.md]
trim_trailing_whitespace = false
[Makefile]
indent_style = tab
[Dockerfile*]
indent_size = 4
+7
View File
@@ -24,6 +24,13 @@ cd frontend
npm run lint npm run lint
``` ```
## Documentation Requirements
When making changes, always update the following files:
- **`docs/TODO.md`**: Update task checkboxes, progress percentages, and status indicators to reflect completed work and any new items discovered.
- **`CHANGELOG.md`**: Add entries under the `[Unreleased]` section using the appropriate category (`Added`, `Changed`, `Fixed`, `Security`, `Removed`, `Deprecated`).
## Conventions ## Conventions
- **Python**: Follow PEP 8. Use `black` for formatting. All ruff and mypy errors must be resolved before committing. - **Python**: Follow PEP 8. Use `black` for formatting. All ruff and mypy errors must be resolved before committing.
+53
View File
@@ -0,0 +1,53 @@
version: 2
updates:
# Python dependencies (backend)
- package-ecosystem: "pip"
directory: "/backend"
schedule:
interval: "weekly"
day: "monday"
open-pull-requests-limit: 10
labels:
- "dependencies"
- "python"
commit-message:
prefix: "chore(deps):"
# GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
open-pull-requests-limit: 5
labels:
- "dependencies"
- "github-actions"
commit-message:
prefix: "chore(ci):"
# npm dependencies (frontend)
- package-ecosystem: "npm"
directory: "/frontend"
schedule:
interval: "weekly"
day: "monday"
open-pull-requests-limit: 10
labels:
- "dependencies"
- "javascript"
commit-message:
prefix: "chore(deps):"
# Docker dependencies
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
open-pull-requests-limit: 5
labels:
- "dependencies"
- "docker"
commit-message:
prefix: "chore(docker):"
+9 -1
View File
@@ -22,6 +22,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Rate limiting per user/tier - Rate limiting per user/tier
- Comprehensive test infrastructure setup - Comprehensive test infrastructure setup
- CI/CD pipeline for testing and security scanning - CI/CD pipeline for testing and security scanning
- Dependabot configuration for automated dependency updates (pip, npm, GitHub Actions, Docker)
- Copilot instructions requiring TODO.md and CHANGELOG.md updates
- Unit tests for security middleware (SecurityHeadersMiddleware, CSRFProtectionMiddleware)
- Unit tests for JWT token lifecycle (access tokens, refresh tokens, decode, edge cases)
- Unit tests for credential encryption edge cases (empty, long, unicode, special chars)
- Unit tests for FastAPI application factory and core endpoints (root, health, OpenAPI)
- Unit tests for Pydantic schema validation (users, mail accounts, notifications, subscriptions)
- Reached 57% test coverage (up from 54%)
### Changed ### Changed
- Reorganized documentation into `docs/` directory - Reorganized documentation into `docs/` directory
@@ -114,4 +122,4 @@ Use these standard categories:
--- ---
**Maintained by**: Development Team **Maintained by**: Development Team
**Last Updated**: 2026-02-06 **Last Updated**: 2026-03-23
+27 -22
View File
@@ -20,7 +20,7 @@ Be respectful and inclusive. We welcome contributions from everyone.
### Suggesting Features ### Suggesting Features
1. Check the [Roadmap](ROADMAP.md) to see if it's already planned 1. Check the [Roadmap](docs/ROADMAP.md) to see if it's already planned
2. Open an issue with the "enhancement" label 2. Open an issue with the "enhancement" label
3. Describe the feature and its use case 3. Describe the feature and its use case
4. Explain why it would be useful 4. Explain why it would be useful
@@ -40,14 +40,14 @@ Be respectful and inclusive. We welcome contributions from everyone.
4. **Test your changes** 4. **Test your changes**
```bash ```bash
# Test Python syntax # Run the test suite
python3 -m py_compile pop3_forwarder.py make test
# Or run linting + formatting + tests together
make quick-test
# Test Docker build # Test Docker build
docker build -t pop3-test . docker build -t pop3-test .
# Test with your configuration
docker-compose up
``` ```
5. **Commit your changes** 5. **Commit your changes**
@@ -80,19 +80,18 @@ Be respectful and inclusive. We welcome contributions from everyone.
git clone https://github.com/YOUR-USERNAME/pop_puller_to_gmail.git git clone https://github.com/YOUR-USERNAME/pop_puller_to_gmail.git
cd pop_puller_to_gmail cd pop_puller_to_gmail
# Create virtual environment # Install all development dependencies
python3 -m venv venv make install-dev
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Copy example config # Copy example config
cp .env.example .env cp .env.example .env
# Edit .env with test credentials # Edit .env with test credentials
# Run locally # Run the legacy forwarder script directly
python pop3_forwarder.py python pop3_forwarder.py
# Or start the SaaS backend in dev mode
make run-dev
``` ```
### Docker Development ### Docker Development
@@ -112,23 +111,29 @@ docker run --env-file .env pop3-dev
- Add docstrings to functions and classes - Add docstrings to functions and classes
- Keep functions focused and small - Keep functions focused and small
- Handle errors gracefully - Handle errors gracefully
- Run `make format` to auto-format with Black and Ruff
## Testing ## Testing
Before submitting a PR: Before submitting a PR:
1. **Syntax check** 1. **Run the test suite**
```bash ```bash
python3 -m py_compile pop3_forwarder.py make test
``` ```
2. **Docker build** 2. **Run linting**
```bash
make lint
```
3. **Docker build**
```bash ```bash
docker build -t pop3-test . docker build -t pop3-test .
``` ```
3. **Manual testing** 4. **Manual testing** (if applicable)
- Test with real POP3 account (or mock) - Test with a real POP3 account or mock
- Verify emails are forwarded correctly - Verify emails are forwarded correctly
- Check error handling - Check error handling
- Review logs - Review logs
@@ -143,9 +148,9 @@ Update documentation when:
Files to update: Files to update:
- `README.md` - Main documentation - `README.md` - Main documentation
- `QUICKSTART.md` - If setup changes - `docs/QUICKSTART.md` - If setup changes
- `MVP.md` - If MVP scope changes - `docs/MVP.md` - If MVP scope changes
- `ROADMAP.md` - If adding future plans - `docs/ROADMAP.md` - If adding future plans
## Security ## Security
@@ -158,7 +163,7 @@ Files to update:
**Do NOT open public issues for security vulnerabilities.** **Do NOT open public issues for security vulnerabilities.**
Email security concerns to the maintainers privately. Please see [SECURITY.md](SECURITY.md) for responsible disclosure instructions.
## Questions? ## Questions?
+103 -261
View File
@@ -1,135 +1,61 @@
# POP3 to Gmail Forwarder # POP3 to Gmail Forwarder
[![CI Tests](https://github.com/christianlouis/pop_puller_to_gmail/actions/workflows/test.yml/badge.svg)](https://github.com/christianlouis/pop_puller_to_gmail/actions/workflows/test.yml)
[![Lint](https://github.com/christianlouis/pop_puller_to_gmail/actions/workflows/lint.yml/badge.svg)](https://github.com/christianlouis/pop_puller_to_gmail/actions/workflows/lint.yml)
[![Security Scan](https://github.com/christianlouis/pop_puller_to_gmail/actions/workflows/security.yml/badge.svg)](https://github.com/christianlouis/pop_puller_to_gmail/actions/workflows/security.yml)
[![Docker Build](https://github.com/christianlouis/pop_puller_to_gmail/actions/workflows/docker-build.yml/badge.svg)](https://github.com/christianlouis/pop_puller_to_gmail/actions/workflows/docker-build.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![Docker](https://img.shields.io/badge/docker-ready-blue.svg)](https://www.docker.com/)
A Docker-based solution that automatically fetches emails from POP3 mailboxes and forwards them to Gmail, replacing Google's discontinued POP3 import feature. A Docker-based solution that automatically fetches emails from POP3 mailboxes and forwards them to Gmail, replacing Google's discontinued POP3 import feature.
## Features ## Features
- **Multiple POP3 Accounts**: Support for unlimited POP3 mailboxes via environment variables - **Multiple POP3 Accounts** — support for unlimited POP3 mailboxes via environment variables
- **Automatic Forwarding**: Sends emails to your Gmail account via SMTP - **Automatic Forwarding** — sends emails to your Gmail account via SMTP
- **Smart Throttling**: Rate limiting to avoid Gmail quotas (configurable emails per minute) - **Smart Throttling** — configurable rate limiting to stay within Gmail quotas
- **Error Reporting**: Email notifications via Postmarkapp when issues occur - **Error Reporting** — notifications via Postmarkapp when issues occur
- **Scheduled Polling**: Configurable check intervals (default: every 5 minutes) - **Scheduled Polling** — configurable check intervals (default: every 5 minutes)
- **Docker Ready**: Fully containerized with docker-compose support - **Docker Ready** — fully containerized with Docker Compose support
- **Secure**: Runs as non-root user, uses SSL/TLS for connections - **Secure** — runs as non-root user, SSL/TLS connections
-**Production Ready**: Comprehensive logging, error handling, and best practices
### SaaS Platform (in development)
The repository also includes a multi-tenant SaaS backend built with FastAPI, PostgreSQL, Redis, and Celery. It adds multi-user support, OAuth2 authentication, POP3/IMAP protocol support, encrypted credential storage, and background job processing. See the [SaaS README](docs/README_SAAS.md) for details.
## Quick Start ## Quick Start
### Prerequisites ### Using a Pre-built Docker Image (Recommended)
- Docker and Docker Compose installed
- A Gmail account with [App Password](https://support.google.com/accounts/answer/185833) enabled
- POP3 account credentials
- (Optional) Postmarkapp account for error notifications
### Option 1: Using Pre-built Docker Image (Recommended)
The Docker images are automatically built and published to GitHub Container Registry.
1. **Create configuration file**
```bash ```bash
# Download the docker-compose.yml and .env.example # Pull and configure
curl -O https://raw.githubusercontent.com/christianlouis/pop_puller_to_gmail/main/docker-compose.yml curl -O https://raw.githubusercontent.com/christianlouis/pop_puller_to_gmail/main/docker-compose.yml
curl -o .env https://raw.githubusercontent.com/christianlouis/pop_puller_to_gmail/main/.env.example curl -o .env https://raw.githubusercontent.com/christianlouis/pop_puller_to_gmail/main/.env.example
# Edit .env with your credentials # Edit .env with your credentials
nano .env nano .env
```
2. **Update docker-compose.yml to use the pre-built image** # Start
```yaml
version: '3.8'
services:
pop3-forwarder:
image: ghcr.io/christianlouis/pop_puller_to_gmail:latest
container_name: pop3-gmail-forwarder
restart: unless-stopped
env_file:
- .env
```
3. **Run the container**
```bash
docker-compose up -d docker-compose up -d
``` ```
### Option 2: Building from Source ### Building from Source
1. **Clone the repository**
```bash ```bash
git clone https://github.com/christianlouis/pop_puller_to_gmail.git git clone https://github.com/christianlouis/pop_puller_to_gmail.git
cd pop_puller_to_gmail cd pop_puller_to_gmail
``` cp .env.example .env # then edit .env
2. **Configure environment variables**
```bash
cp .env.example .env
# Edit .env with your credentials
nano .env
```
3. **Essential Configuration**
Edit `.env` and set:
```bash
# Your POP3 account(s)
POP3_ACCOUNT_1_HOST=pop.yourprovider.com
POP3_ACCOUNT_1_PORT=995
POP3_ACCOUNT_1_USER=your-email@provider.com
POP3_ACCOUNT_1_PASSWORD=your-password
# Your Gmail SMTP settings
SMTP_USER=your-gmail@gmail.com
SMTP_PASSWORD=your-app-password # Generate at myaccount.google.com/apppasswords
GMAIL_DESTINATION=your-gmail@gmail.com
# Optional: Postmarkapp for error notifications
POSTMARK_API_TOKEN=your-token
POSTMARK_FROM_EMAIL=errors@yourdomain.com
POSTMARK_TO_EMAIL=admin@yourdomain.com
```
4. **Run with Docker Compose**
```bash
docker-compose up -d docker-compose up -d
``` ```
5. **Check logs** See the [Quick Start Guide](docs/QUICKSTART.md) for detailed instructions.
```bash
docker-compose logs -f
```
## Using Pre-built Docker Images
Docker images are automatically built and published to GitHub Container Registry for every release and commit to the main branch.
### Available Image Tags
- `ghcr.io/christianlouis/pop_puller_to_gmail:latest` - Latest build from main branch
- `ghcr.io/christianlouis/pop_puller_to_gmail:v1.0.0` - Specific version tags
- `ghcr.io/christianlouis/pop_puller_to_gmail:main` - Main branch builds
### Pull and Run
```bash
# Pull the latest image
docker pull ghcr.io/christianlouis/pop_puller_to_gmail:latest
# Run directly with Docker
docker run -d \
--name pop3-forwarder \
--env-file .env \
--restart unless-stopped \
ghcr.io/christianlouis/pop_puller_to_gmail:latest
```
## Configuration ## Configuration
### POP3 Accounts ### POP3 Accounts
Add multiple POP3 accounts by incrementing the account number: Add multiple POP3 accounts by incrementing the account number in your `.env`:
```bash ```bash
POP3_ACCOUNT_1_HOST=pop.provider1.com POP3_ACCOUNT_1_HOST=pop.provider1.com
@@ -139,199 +65,115 @@ POP3_ACCOUNT_1_PASSWORD=password1
POP3_ACCOUNT_2_HOST=pop.provider2.com POP3_ACCOUNT_2_HOST=pop.provider2.com
POP3_ACCOUNT_2_USER=user2@provider2.com POP3_ACCOUNT_2_USER=user2@provider2.com
POP3_ACCOUNT_2_PASSWORD=password2 POP3_ACCOUNT_2_PASSWORD=password2
# ... add more as needed
``` ```
### Gmail App Password ### Gmail App Password
1. Go to your Google Account: https://myaccount.google.com/ 1. Go to your [Google Account Security](https://myaccount.google.com/security)
2. Select Security 2. Under "Signing in to Google," select **App Passwords**
3. Under "Signing in to Google," select App Passwords 3. Generate a new app password for "Mail"
4. Generate a new app password for "Mail" 4. Use this password as `SMTP_PASSWORD`
5. Use this password in `SMTP_PASSWORD`
### Environment Variables ### Environment Variables
| Variable | Required | Default | Description | | Variable | Required | Default | Description |
|----------|----------|---------|-------------| |----------|----------|---------|-------------|
| `POP3_ACCOUNT_N_HOST` | Yes | - | POP3 server hostname | | `POP3_ACCOUNT_N_HOST` | Yes | | POP3 server hostname |
| `POP3_ACCOUNT_N_PORT` | No | 995 | POP3 server port | | `POP3_ACCOUNT_N_PORT` | No | `995` | POP3 server port |
| `POP3_ACCOUNT_N_USER` | Yes | - | POP3 username | | `POP3_ACCOUNT_N_USER` | Yes | | POP3 username |
| `POP3_ACCOUNT_N_PASSWORD` | Yes | - | POP3 password | | `POP3_ACCOUNT_N_PASSWORD` | Yes | | POP3 password |
| `POP3_ACCOUNT_N_USE_SSL` | No | true | Use SSL/TLS | | `POP3_ACCOUNT_N_USE_SSL` | No | `true` | Use SSL/TLS |
| `SMTP_HOST` | No | smtp.gmail.com | SMTP server | | `SMTP_HOST` | No | `smtp.gmail.com` | SMTP server |
| `SMTP_PORT` | No | 587 | SMTP port | | `SMTP_PORT` | No | `587` | SMTP port |
| `SMTP_USER` | Yes | - | SMTP username | | `SMTP_USER` | Yes | | SMTP username |
| `SMTP_PASSWORD` | Yes | - | SMTP password (App Password) | | `SMTP_PASSWORD` | Yes | | SMTP password (App Password) |
| `SMTP_USE_TLS` | No | true | Use STARTTLS | | `SMTP_USE_TLS` | No | `true` | Use STARTTLS |
| `GMAIL_DESTINATION` | Yes | - | Destination Gmail address | | `GMAIL_DESTINATION` | Yes | | Destination Gmail address |
| `CHECK_INTERVAL_MINUTES` | No | 5 | How often to check for new mail | | `CHECK_INTERVAL_MINUTES` | No | `5` | Polling interval |
| `MAX_EMAILS_PER_RUN` | No | 50 | Max emails to process per account per run | | `MAX_EMAILS_PER_RUN` | No | `50` | Max emails per account per run |
| `THROTTLE_EMAILS_PER_MINUTE` | No | 10 | Rate limit for sending emails | | `THROTTLE_EMAILS_PER_MINUTE` | No | `10` | Rate limit |
| `POSTMARK_API_TOKEN` | No | - | Postmarkapp API token | | `POSTMARK_API_TOKEN` | No | | Postmarkapp API token |
| `POSTMARK_FROM_EMAIL` | No | - | Error notification sender | | `POSTMARK_FROM_EMAIL` | No | | Error notification sender |
| `POSTMARK_TO_EMAIL` | No | - | Error notification recipient | | `POSTMARK_TO_EMAIL` | No | | Error notification recipient |
| `LOG_LEVEL` | No | INFO | Logging level (DEBUG, INFO, WARNING, ERROR) | | `LOG_LEVEL` | No | `INFO` | Logging level |
## How It Works ## How It Works
1. **Polling**: The application checks configured POP3 mailboxes at regular intervals
2. **Fetching**: Retrieves new emails from each POP3 account
3. **Forwarding**: Sends emails to your Gmail account via SMTP with original metadata preserved
4. **Cleanup**: Deletes emails from POP3 server after successful forwarding
5. **Throttling**: Respects rate limits to avoid Gmail quota issues
6. **Error Handling**: Sends notifications via Postmarkapp if issues occur
## Email Format
Forwarded emails include:
- Original sender information in the subject line: `[Fwd from user@provider.com] Original Subject`
- Header section with original From, Date, Subject, and source account
- Original email body preserved
## Monitoring and Logs
### View logs
```bash
docker-compose logs -f pop3-forwarder
```
### Check container status
```bash
docker-compose ps
```
### Restart the service
```bash
docker-compose restart
```
## Troubleshooting
### Gmail Authentication Issues
**Problem**: "Username and Password not accepted"
**Solution**:
- Ensure 2FA is enabled on your Google account
- Generate an App Password (don't use your regular Gmail password)
- Use the 16-character app password without spaces
### POP3 Connection Issues
**Problem**: "Connection refused" or "SSL error"
**Solution**:
- Verify POP3 server hostname and port
- Check if POP3 is enabled in your email provider settings
- Try with `POP3_ACCOUNT_N_USE_SSL=false` for non-SSL connections (port 110)
### No Emails Being Forwarded
**Problem**: Container runs but no emails are forwarded
**Solution**:
- Check if there are emails in your POP3 mailbox
- Review logs for errors: `docker-compose logs -f`
- Verify `GMAIL_DESTINATION` is correct
- Check Gmail spam folder
### Rate Limiting
**Problem**: "Too many requests" or quota errors
**Solution**:
- Increase `CHECK_INTERVAL_MINUTES`
- Decrease `THROTTLE_EMAILS_PER_MINUTE`
- Reduce `MAX_EMAILS_PER_RUN`
## Security Best Practices
1. **Never commit `.env` file** - It contains sensitive credentials
2. **Use App Passwords** - Don't use your main Gmail password
3. **Rotate credentials regularly** - Update passwords periodically
4. **Enable 2FA** - On all email accounts
5. **Review logs** - Monitor for suspicious activity
6. **Use SSL/TLS** - Keep `USE_SSL` and `USE_TLS` enabled
7. **Limit network access** - Use firewall rules if needed
## Development
### Local Development (without Docker)
```bash
# Create virtual environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Copy and configure .env
cp .env.example .env
# Edit .env with your settings
# Run the application
python pop3_forwarder.py
```
### Building the Docker Image
```bash
docker build -t pop3-gmail-forwarder .
```
### Running Tests
```bash
# Run with verbose logging
LOG_LEVEL=DEBUG docker-compose up
```
## Architecture
``` ```
┌─────────────────┐ ┌─────────────────┐
│ POP3 Server 1 │ │ POP3 Server 1 │
└────────┬────────┘ └────────┬────────┘
│ (Fetch emails) │ (Fetch emails)
┌─────────────────┐ ┌──────────────┐ ┌─────────────┐ ┌─────────────────┐ ┌──────────────┐ ┌─────────────┐
│ POP3 Server 2 │─────▶│ Forwarder │─────▶│ Gmail │ │ POP3 Server 2 │─────▶│ Forwarder │─────▶│ Gmail │
└─────────────────┘ │ Container │ │ (SMTP) │ └─────────────────┘ │ Container │ │ (SMTP) │
│ └──────┬───────┘ └─────────────┘ │ └──────┬───────┘ └─────────────┘
│ │ ┌────────▼────────┐ │ (Error notifications)
┌────────▼────────┐ │ │ POP3 Server N │ ▼
│ POP3 Server N │ │ └─────────────────┘ ┌─────────────────┐
└─────────────────┘ │
│ (Error notifications)
┌─────────────────┐
│ Postmarkapp │ │ Postmarkapp │
└─────────────────┘ └─────────────────┘
``` ```
1. **Polling** — checks POP3 mailboxes at the configured interval
2. **Fetching** — retrieves new emails from each account
3. **Forwarding** — delivers to Gmail with original metadata preserved
4. **Cleanup** — deletes from POP3 after successful forwarding
5. **Throttling** — respects rate limits to avoid quota issues
6. **Error Handling** — sends notifications if something goes wrong
## Development
```bash
# Install dependencies
make install-dev
# Run linting & formatting
make lint
make format
# Run tests
make test
# Start backend in dev mode
make run-dev
```
See the [Testing Guide](docs/TESTING_GUIDE.md) for the full test workflow.
## Documentation
Detailed documentation lives in the [`docs/`](docs/) directory:
| Document | Description |
|----------|-------------|
| [Architecture](docs/ARCHITECTURE.md) | System design and component overview |
| [Quick Start](docs/QUICKSTART.md) | Step-by-step setup guide |
| [Migration Guide](docs/MIGRATION_GUIDE.md) | Upgrading from v1 to v2 |
| [Deployment Checklist](docs/DEPLOYMENT_CHECKLIST.md) | Production deployment guide |
| [Roadmap](docs/ROADMAP.md) | Planned features and milestones |
| [Testing Guide](docs/TESTING_GUIDE.md) | How to run and write tests |
| [Coding Patterns](docs/CODING_PATTERNS.md) | Code style and conventions |
| [SaaS README](docs/README_SAAS.md) | Multi-tenant SaaS platform details |
## Contributing ## Contributing
Contributions are welcome! Please: Contributions are welcome! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on:
1. Fork the repository - Reporting bugs and suggesting features
2. Create a feature branch - Development setup and code style
3. Make your changes - Pull request process
4. Submit a pull request
## Security
To report a vulnerability, please see [SECURITY.md](SECURITY.md). **Do not open public issues for security concerns.**
## License ## License
MIT License - See LICENSE file for details This project is licensed under the MIT License — see [LICENSE](LICENSE) for details.
## Support ## Support
- **Issues**: https://github.com/christianlouis/pop_puller_to_gmail/issues - [Issue Tracker](https://github.com/christianlouis/pop_puller_to_gmail/issues)
- **Discussions**: https://github.com/christianlouis/pop_puller_to_gmail/discussions - [Discussions](https://github.com/christianlouis/pop_puller_to_gmail/discussions)
## Acknowledgments
Built to replace Gmail's discontinued POP3 import feature. Uses industry-standard Python libraries for email handling and Docker for easy deployment.
+41
View File
@@ -0,0 +1,41 @@
# Security Policy
## Supported Versions
| Version | Supported |
|---------|--------------------|
| 2.x | ✅ Yes |
| 1.x | ❌ No |
| < 1.0 | ❌ No |
## Reporting a Vulnerability
**Please do NOT open public issues for security vulnerabilities.**
If you discover a security vulnerability, please report it responsibly:
1. **Email**: Send details to the repository maintainer via the email listed on the [GitHub profile](https://github.com/christianlouis).
2. **GitHub Private Vulnerability Reporting**: Use [GitHub's security advisory feature](https://github.com/christianlouis/pop_puller_to_gmail/security/advisories/new) to report privately.
### What to Include
- A description of the vulnerability
- Steps to reproduce the issue
- Potential impact
- Suggested fix (if any)
### Response Timeline
- **Acknowledgment**: Within 48 hours
- **Initial Assessment**: Within 1 week
- **Fix & Disclosure**: Coordinated with the reporter
## Security Best Practices for Users
- **Never commit `.env` files** containing credentials
- **Use App Passwords** for Gmail instead of your main password
- **Enable 2FA** on all email accounts
- **Rotate credentials** regularly
- **Use SSL/TLS** for all mail connections
- **Run containers as non-root** (default in provided Dockerfile)
- **Keep dependencies updated** — Dependabot is enabled on this repository
+108
View File
@@ -0,0 +1,108 @@
"""
Unit tests for the FastAPI application factory and core endpoints.
"""
import pytest
from httpx import AsyncClient, ASGITransport
from app.main import create_application
@pytest.fixture
def app():
"""Create a fresh application instance for testing."""
return create_application()
@pytest.mark.asyncio
class TestRootEndpoint:
"""Test root endpoint"""
async def test_root_returns_200(self, app):
"""Test that root endpoint returns 200"""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/")
assert response.status_code == 200
async def test_root_returns_api_info(self, app):
"""Test that root returns API information"""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/")
data = response.json()
assert "message" in data
assert "version" in data
assert "docs" in data
assert data["docs"] == "/api/docs"
@pytest.mark.asyncio
class TestHealthEndpoint:
"""Test health check endpoint"""
async def test_health_returns_200(self, app):
"""Test that health endpoint returns 200"""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/health")
assert response.status_code == 200
async def test_health_returns_healthy(self, app):
"""Test that health endpoint returns healthy status"""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/health")
data = response.json()
assert data["status"] == "healthy"
@pytest.mark.asyncio
class TestSecurityHeaders:
"""Test that security headers are present in responses"""
async def test_security_headers_on_root(self, app):
"""Test security headers on root endpoint"""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/")
assert response.headers["X-Frame-Options"] == "DENY"
assert response.headers["X-Content-Type-Options"] == "nosniff"
assert response.headers["X-XSS-Protection"] == "1; mode=block"
async def test_security_headers_on_health(self, app):
"""Test security headers on health endpoint"""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/health")
assert response.headers["X-Frame-Options"] == "DENY"
@pytest.mark.asyncio
class TestApplicationFactory:
"""Test the create_application factory"""
async def test_app_title(self, app):
"""Test that app has correct title"""
assert app.title == "POP3 Forwarder SaaS"
async def test_app_version(self, app):
"""Test that app has a version"""
assert app.version is not None
assert len(app.version) > 0
async def test_openapi_endpoint(self, app):
"""Test that OpenAPI schema is available"""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/api/openapi.json")
assert response.status_code == 200
schema = response.json()
assert "openapi" in schema
assert "info" in schema
+127
View File
@@ -0,0 +1,127 @@
"""
Unit tests for security middleware.
"""
from starlette.testclient import TestClient
from starlette.applications import Starlette
from starlette.responses import PlainTextResponse
from starlette.routing import Route
from app.core.middleware import SecurityHeadersMiddleware, CSRFProtectionMiddleware
def _make_app(middleware_classes):
"""Helper to build a Starlette app with given middleware."""
async def homepage(request):
return PlainTextResponse("OK")
app = Starlette(
routes=[
Route("/", homepage, methods=["GET", "HEAD", "POST", "OPTIONS"]),
Route("/api/v1/auth/login", homepage, methods=["GET", "POST"]),
]
)
for cls in middleware_classes:
app.add_middleware(cls)
return app
class TestSecurityHeadersMiddleware:
"""Test security headers added to all responses"""
def setup_method(self):
app = _make_app([SecurityHeadersMiddleware])
self.client = TestClient(app)
def test_x_frame_options_header(self):
"""Test X-Frame-Options is set to DENY"""
response = self.client.get("/")
assert response.headers["X-Frame-Options"] == "DENY"
def test_x_content_type_options_header(self):
"""Test X-Content-Type-Options is set to nosniff"""
response = self.client.get("/")
assert response.headers["X-Content-Type-Options"] == "nosniff"
def test_x_xss_protection_header(self):
"""Test X-XSS-Protection header is set"""
response = self.client.get("/")
assert response.headers["X-XSS-Protection"] == "1; mode=block"
def test_content_security_policy_header(self):
"""Test Content-Security-Policy header is present"""
response = self.client.get("/")
csp = response.headers["Content-Security-Policy"]
assert "default-src 'self'" in csp
assert "script-src" in csp
def test_referrer_policy_header(self):
"""Test Referrer-Policy header"""
response = self.client.get("/")
assert response.headers["Referrer-Policy"] == "strict-origin-when-cross-origin"
def test_permissions_policy_header(self):
"""Test Permissions-Policy header"""
response = self.client.get("/")
policy = response.headers["Permissions-Policy"]
assert "geolocation=()" in policy
assert "microphone=()" in policy
assert "camera=()" in policy
def test_no_hsts_for_localhost(self):
"""Test that HSTS header check depends on hostname"""
# The HSTS header is only skipped when hostname is localhost or 127.0.0.1.
# TestClient uses 'testserver' as hostname, which is not in the skip list,
# so HSTS will be set. Verify the logic works with a direct check.
response = self.client.get("/")
# TestClient hostname is 'testserver', not localhost, so HSTS IS set
assert "Strict-Transport-Security" in response.headers
class TestCSRFProtectionMiddleware:
"""Test CSRF protection middleware"""
def setup_method(self):
app = _make_app([CSRFProtectionMiddleware])
self.client = TestClient(app)
def test_get_requests_pass_through(self):
"""Test that GET requests are not blocked"""
response = self.client.get("/")
assert response.status_code == 200
def test_head_requests_pass_through(self):
"""Test that HEAD requests are not blocked"""
response = self.client.head("/")
assert response.status_code == 200
def test_options_requests_pass_through(self):
"""Test that OPTIONS requests are not blocked"""
response = self.client.options("/")
assert response.status_code == 200
def test_exempt_paths_pass_through(self):
"""Test that exempt paths are not CSRF-checked for POST"""
response = self.client.post("/api/v1/auth/login")
assert response.status_code == 200
def test_post_to_non_exempt_path_passes(self):
"""Test that POST to non-exempt path also passes (JWT provides CSRF protection)"""
response = self.client.post("/")
assert response.status_code == 200
def test_generate_csrf_token(self):
"""Test CSRF token generation produces valid token"""
token = CSRFProtectionMiddleware._generate_csrf_token()
assert isinstance(token, str)
assert len(token) == 43 # token_urlsafe(32) produces 43 chars
def test_validate_csrf_token_valid(self):
"""Test CSRF token validation with valid token"""
token = CSRFProtectionMiddleware._generate_csrf_token()
assert CSRFProtectionMiddleware._validate_csrf_token(token) is True
def test_validate_csrf_token_invalid(self):
"""Test CSRF token validation with invalid token"""
assert CSRFProtectionMiddleware._validate_csrf_token("short") is False
+277
View File
@@ -0,0 +1,277 @@
"""
Unit tests for Pydantic schema validation.
"""
import pytest
from pydantic import ValidationError
from app.models.schemas import (
UserCreate,
UserUpdate,
MailAccountCreate,
MailAccountUpdate,
MailAccountTestRequest,
MailAccountAutoDetectRequest,
MailProtocol,
DeliveryMethod,
SubscriptionTier,
AccountStatus,
NotificationChannel,
Token,
TokenPayload,
GoogleAuthRequest,
NotificationConfigCreate,
SubscriptionCheckoutRequest,
ProviderPreset,
)
class TestEnums:
"""Test enum values"""
def test_subscription_tiers(self):
"""Test all subscription tier values"""
assert SubscriptionTier.FREE == "free"
assert SubscriptionTier.BASIC == "basic"
assert SubscriptionTier.PRO == "pro"
assert SubscriptionTier.ENTERPRISE == "enterprise"
def test_mail_protocols(self):
"""Test all mail protocol values"""
assert MailProtocol.POP3 == "pop3"
assert MailProtocol.POP3_SSL == "pop3_ssl"
assert MailProtocol.IMAP == "imap"
assert MailProtocol.IMAP_SSL == "imap_ssl"
def test_account_status(self):
"""Test all account status values"""
assert AccountStatus.ACTIVE == "active"
assert AccountStatus.INACTIVE == "inactive"
assert AccountStatus.ERROR == "error"
assert AccountStatus.TESTING == "testing"
def test_delivery_method(self):
"""Test all delivery method values"""
assert DeliveryMethod.SMTP == "smtp"
assert DeliveryMethod.GMAIL_API == "gmail_api"
def test_notification_channels(self):
"""Test all notification channel values"""
assert NotificationChannel.EMAIL == "email"
assert NotificationChannel.TELEGRAM == "telegram"
assert NotificationChannel.WEBHOOK == "webhook"
assert NotificationChannel.SLACK == "slack"
assert NotificationChannel.DISCORD == "discord"
class TestUserSchemas:
"""Test user-related schemas"""
def test_user_create_with_email(self):
"""Test UserCreate with valid email"""
user = UserCreate(email="test@example.com", password="password123")
assert user.email == "test@example.com"
assert user.password == "password123"
def test_user_create_without_password(self):
"""Test UserCreate without password (OAuth users)"""
user = UserCreate(email="test@example.com")
assert user.password is None
def test_user_create_with_full_name(self):
"""Test UserCreate with full name"""
user = UserCreate(
email="test@example.com", full_name="Test User", password="pass"
)
assert user.full_name == "Test User"
def test_user_create_invalid_email(self):
"""Test UserCreate rejects invalid email"""
with pytest.raises(ValidationError):
UserCreate(email="not-an-email", password="pass")
def test_user_update_partial(self):
"""Test UserUpdate with partial data"""
update = UserUpdate(full_name="New Name")
assert update.full_name == "New Name"
assert update.email is None
class TestTokenSchemas:
"""Test token schemas"""
def test_token_schema(self):
"""Test Token schema"""
token = Token(
access_token="abc123", refresh_token="def456", token_type="bearer"
)
assert token.access_token == "abc123"
assert token.token_type == "bearer"
def test_token_payload_schema(self):
"""Test TokenPayload schema"""
payload = TokenPayload(sub=42, type="access")
assert payload.sub == 42
assert payload.type == "access"
def test_google_auth_request(self):
"""Test GoogleAuthRequest schema"""
req = GoogleAuthRequest(
code="auth-code-123", redirect_uri="http://localhost:3000/callback"
)
assert req.code == "auth-code-123"
class TestMailAccountSchemas:
"""Test mail account schemas"""
def test_mail_account_create_valid(self):
"""Test creating a valid mail account"""
account = MailAccountCreate(
name="Test Account",
email_address="user@example.com",
host="imap.example.com",
port=993,
username="user@example.com",
password="secret",
forward_to="me@gmail.com",
)
assert account.name == "Test Account"
assert account.protocol == MailProtocol.POP3_SSL # default
assert account.use_ssl is True
def test_mail_account_create_invalid_port(self):
"""Test that invalid port is rejected"""
with pytest.raises(ValidationError):
MailAccountCreate(
name="Test",
email_address="user@example.com",
host="imap.example.com",
port=0, # invalid
username="user@example.com",
password="secret",
forward_to="me@gmail.com",
)
def test_mail_account_create_port_too_high(self):
"""Test that port above 65535 is rejected"""
with pytest.raises(ValidationError):
MailAccountCreate(
name="Test",
email_address="user@example.com",
host="imap.example.com",
port=70000, # invalid
username="user@example.com",
password="secret",
forward_to="me@gmail.com",
)
def test_mail_account_update_partial(self):
"""Test partial mail account update"""
update = MailAccountUpdate(is_enabled=False)
assert update.is_enabled is False
assert update.name is None
assert update.password is None
def test_mail_account_test_request(self):
"""Test mail account test connection schema"""
req = MailAccountTestRequest(
host="imap.gmail.com",
port=993,
protocol=MailProtocol.IMAP_SSL,
username="user@gmail.com",
password="app-password",
)
assert req.host == "imap.gmail.com"
def test_auto_detect_request(self):
"""Test auto-detect request schema"""
req = MailAccountAutoDetectRequest(email_address="user@gmail.com")
assert req.email_address == "user@gmail.com"
def test_auto_detect_invalid_email(self):
"""Test auto-detect rejects invalid email"""
with pytest.raises(ValidationError):
MailAccountAutoDetectRequest(email_address="not-email")
class TestNotificationSchemas:
"""Test notification schemas"""
def test_notification_config_create(self):
"""Test creating notification config"""
config = NotificationConfigCreate(
channel=NotificationChannel.TELEGRAM,
config={"bot_token": "123:abc", "chat_id": "456"},
)
assert config.channel == NotificationChannel.TELEGRAM
assert config.notify_on_errors is True # default
assert config.notify_on_success is False # default
def test_notification_config_threshold_validation(self):
"""Test notification threshold validation"""
with pytest.raises(ValidationError):
NotificationConfigCreate(
channel=NotificationChannel.EMAIL,
config={},
notify_threshold=0, # must be > 0
)
class TestSubscriptionSchemas:
"""Test subscription schemas"""
def test_subscription_checkout_request_monthly(self):
"""Test subscription checkout with monthly billing"""
req = SubscriptionCheckoutRequest(
tier=SubscriptionTier.PRO,
billing_period="monthly",
success_url="https://example.com/success",
cancel_url="https://example.com/cancel",
)
assert req.tier == SubscriptionTier.PRO
assert req.billing_period == "monthly"
def test_subscription_checkout_request_yearly(self):
"""Test subscription checkout with yearly billing"""
req = SubscriptionCheckoutRequest(
tier=SubscriptionTier.BASIC,
billing_period="yearly",
success_url="https://example.com/success",
cancel_url="https://example.com/cancel",
)
assert req.billing_period == "yearly"
def test_subscription_checkout_invalid_period(self):
"""Test that invalid billing period is rejected"""
with pytest.raises(ValidationError):
SubscriptionCheckoutRequest(
tier=SubscriptionTier.BASIC,
billing_period="quarterly", # invalid
success_url="https://example.com/success",
cancel_url="https://example.com/cancel",
)
class TestProviderPresetSchema:
"""Test provider preset schema"""
def test_provider_preset_with_imap(self):
"""Test provider preset with IMAP config"""
preset = ProviderPreset(
id="gmail",
name="Gmail",
domains=["gmail.com", "googlemail.com"],
imap_ssl={"host": "imap.gmail.com", "port": 993},
)
assert preset.id == "gmail"
assert "gmail.com" in preset.domains
def test_provider_preset_without_pop3(self):
"""Test provider preset without POP3 (IMAP only)"""
preset = ProviderPreset(
id="posteo",
name="Posteo",
domains=["posteo.de"],
imap_ssl={"host": "posteo.de", "port": 993},
)
assert preset.pop3_ssl is None
@@ -0,0 +1,188 @@
"""
Unit tests for extended security module functionality.
"""
from datetime import timedelta
from app.core.security import (
create_access_token,
create_refresh_token,
decode_token,
generate_random_token,
encrypt_credential,
decrypt_credential,
CredentialEncryption,
)
class TestAccessToken:
"""Test JWT access token creation and decoding"""
def test_create_access_token_with_custom_expiry(self):
"""Test creating an access token with custom expiry"""
data = {"sub": "test@example.com"}
token = create_access_token(data, expires_delta=timedelta(hours=1))
assert isinstance(token, str)
assert token.count(".") == 2
def test_decode_valid_access_token(self):
"""Test decoding a valid access token"""
data = {"sub": "user123"}
token = create_access_token(data)
payload = decode_token(token)
assert payload is not None
assert payload["sub"] == "user123"
assert payload["type"] == "access"
def test_decode_invalid_token_returns_none(self):
"""Test that decoding an invalid token returns None"""
result = decode_token("invalid.token.string")
assert result is None
def test_decode_empty_token_returns_none(self):
"""Test that decoding an empty string returns None"""
result = decode_token("")
assert result is None
def test_access_token_contains_type(self):
"""Test that access token payload contains type 'access'"""
data = {"sub": "user@example.com"}
token = create_access_token(data)
payload = decode_token(token)
assert payload is not None
assert payload["type"] == "access"
def test_access_token_contains_expiry(self):
"""Test that access token payload contains expiry"""
data = {"sub": "user@example.com"}
token = create_access_token(data)
payload = decode_token(token)
assert payload is not None
assert "exp" in payload
class TestRefreshToken:
"""Test JWT refresh token creation and decoding"""
def test_create_refresh_token(self):
"""Test refresh token creation"""
data = {"sub": "test@example.com"}
token = create_refresh_token(data)
assert isinstance(token, str)
assert token.count(".") == 2
def test_decode_refresh_token(self):
"""Test decoding a valid refresh token"""
data = {"sub": "user456"}
token = create_refresh_token(data)
payload = decode_token(token)
assert payload is not None
assert payload["sub"] == "user456"
assert payload["type"] == "refresh"
def test_refresh_token_different_from_access(self):
"""Test that refresh and access tokens are different"""
data = {"sub": "test@example.com"}
access = create_access_token(data)
refresh = create_refresh_token(data)
assert access != refresh
class TestRandomToken:
"""Test random token generation"""
def test_generate_random_token_default_length(self):
"""Test generating a random token with default length"""
token = generate_random_token()
assert isinstance(token, str)
assert len(token) > 0
def test_generate_random_token_custom_length(self):
"""Test generating a random token with custom length"""
token = generate_random_token(64)
assert isinstance(token, str)
assert len(token) > 0
def test_random_tokens_are_unique(self):
"""Test that generated tokens are unique"""
tokens = {generate_random_token() for _ in range(10)}
assert len(tokens) == 10
class TestGlobalEncryptionFunctions:
"""Test global encryption convenience functions"""
def test_encrypt_credential_returns_string(self):
"""Test that encrypt_credential returns a non-empty string"""
encrypted = encrypt_credential("my-password")
assert isinstance(encrypted, str)
assert len(encrypted) > 0
def test_decrypt_credential_roundtrip(self):
"""Test encrypt/decrypt roundtrip with global functions"""
original = "super-secret-password-123"
encrypted = encrypt_credential(original)
decrypted = decrypt_credential(encrypted)
assert decrypted == original
def test_encrypt_credential_is_not_plaintext(self):
"""Test that encrypted credential differs from plaintext"""
password = "my-password"
encrypted = encrypt_credential(password)
assert encrypted != password
class TestCredentialEncryptionEdgeCases:
"""Test edge cases in credential encryption"""
def test_encrypt_empty_string(self):
"""Test encrypting an empty string"""
encryptor = CredentialEncryption(user_id=1)
encrypted = encryptor.encrypt("")
decrypted = encryptor.decrypt(encrypted)
assert decrypted == ""
def test_encrypt_long_string(self):
"""Test encrypting a very long string"""
long_password = "a" * 10000
encryptor = CredentialEncryption(user_id=1)
encrypted = encryptor.encrypt(long_password)
decrypted = encryptor.decrypt(encrypted)
assert decrypted == long_password
def test_encrypt_special_characters(self):
"""Test encrypting a string with special characters"""
special = "p@$$w0rd!#%^&*()_+-=[]{}|;':\",./<>?"
encryptor = CredentialEncryption(user_id=1)
encrypted = encryptor.encrypt(special)
decrypted = encryptor.decrypt(encrypted)
assert decrypted == special
def test_encrypt_unicode(self):
"""Test encrypting unicode characters"""
unicode_str = "密码テスト🔒"
encryptor = CredentialEncryption(user_id=1)
encrypted = encryptor.encrypt(unicode_str)
decrypted = encryptor.decrypt(encrypted)
assert decrypted == unicode_str
def test_custom_key(self):
"""Test encryption with a custom key"""
key = "custom-encryption-key-that-is-at-least-32-chars-long"
encryptor = CredentialEncryption(key=key, user_id=1)
encrypted = encryptor.encrypt("test-data")
decrypted = encryptor.decrypt(encrypted)
assert decrypted == "test-data"
def test_system_salt_without_user_id(self):
"""Test encryption with system salt (no user_id)"""
encryptor = CredentialEncryption()
encrypted = encryptor.encrypt("system-data")
decrypted = encryptor.decrypt(encrypted)
assert decrypted == "system-data"
+3 -3
View File
@@ -316,9 +316,9 @@ alembic downgrade -1
## 📚 Additional Documentation ## 📚 Additional Documentation
- [API Documentation](http://localhost:8000/api/docs) - Interactive API docs - [API Documentation](http://localhost:8000/api/docs) - Interactive API docs
- [CONTRIBUTING.md](CONTRIBUTING.md) - Contribution guidelines - [CONTRIBUTING.md](../CONTRIBUTING.md) - Contribution guidelines
- [ROADMAP.md](ROADMAP.md) - Future development plans - [ROADMAP.md](ROADMAP.md) - Future development plans
- [SECURITY.md](SECURITY.md) - Security policies - [SECURITY.md](../SECURITY.md) - Security policies
## 🤝 Contributing ## 🤝 Contributing
@@ -331,7 +331,7 @@ Contributions welcome! Please:
## 📄 License ## 📄 License
MIT License - See [LICENSE](LICENSE) file MIT License - See [LICENSE](../LICENSE) file
## 🆘 Support ## 🆘 Support
File diff suppressed because it is too large Load Diff
View File
+10 -13
View File
@@ -312,7 +312,7 @@ Helm charts and Kubernetes manifests will be provided for production deployment.
## 🤝 Contributing ## 🤝 Contributing
Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. Contributions are welcome! Please see [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
### Areas for Contribution ### Areas for Contribution
@@ -325,7 +325,7 @@ Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for gui
## 📜 License ## 📜 License
MIT License - See [LICENSE](LICENSE) file for details. MIT License - See [LICENSE](../LICENSE) file for details.
## 🆘 Support ## 🆘 Support
@@ -357,16 +357,19 @@ Built with these amazing open-source projects:
| Background Jobs | ✅ Complete | 100% | | Background Jobs | ✅ Complete | 100% |
| Documentation | ✅ Complete | 100% | | Documentation | ✅ Complete | 100% |
| Stripe Integration | 🚧 In Progress | 60% | | Stripe Integration | 🚧 In Progress | 60% |
| Frontend Dashboard | 📋 Planned | 0% | | Frontend Dashboard | 🚧 In Progress | 70% |
| Notification System | 📋 Planned | 40% | | Notification System | 🚧 In Progress | 40% |
| Testing Suite | 📋 Planned | 20% | | Testing Suite | 🚧 In Progress | 30% |
> **Note:** The frontend pages and components are implemented but the API client
> layer (`lib/api.ts`) is not yet wired up, so the dashboard does not function
> end-to-end yet.
## 🔮 Roadmap ## 🔮 Roadmap
See [ROADMAP.md](ROADMAP.md) for detailed future plans, including: See [ROADMAP.md](ROADMAP.md) for detailed future plans, including:
- Complete web dashboard - Complete web dashboard
- Mobile app (iOS/Android)
- Advanced email filtering - Advanced email filtering
- Email archiving - Email archiving
- Multi-destination forwarding - Multi-destination forwarding
@@ -374,12 +377,6 @@ See [ROADMAP.md](ROADMAP.md) for detailed future plans, including:
- Kubernetes deployment - Kubernetes deployment
- High availability setup - High availability setup
## ⭐ Star History
If you find this project useful, please consider giving it a star! ⭐
--- ---
**Version**: 2.0.0 | **Status**: Production Ready (Backend) | **Updated**: 2026-02-01 **Status**: In Development | **Backend**: Production-ready | **Frontend**: In Progress
Made with ❤️ for the community
+1 -1
View File
@@ -243,7 +243,7 @@ We welcome contributions! Areas where help is needed:
5. **Performance**: Optimize slow operations 5. **Performance**: Optimize slow operations
6. **Security**: Security audits and improvements 6. **Security**: Security audits and improvements
See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
--- ---
+52 -13
View File
@@ -43,8 +43,8 @@ Comprehensive task breakdown for repository improvements and production readines
- [x] Add `.secrets.baseline` for detect-secrets - [x] Add `.secrets.baseline` for detect-secrets
### In Progress 🔨 ### In Progress 🔨
- [x] Reorganize documentation into `docs/` directory
- [ ] Complete ADR documentation (add ADR-003 through ADR-010) - [ ] Complete ADR documentation (add ADR-003 through ADR-010)
- [ ] Reorganize documentation into `docs/` directory
- [ ] Create GitHub Projects board for task management - [ ] Create GitHub Projects board for task management
### Not Started 📋 ### Not Started 📋
@@ -65,6 +65,11 @@ Comprehensive task breakdown for repository improvements and production readines
- [x] Add `backend/pytest.ini` configuration - [x] Add `backend/pytest.ini` configuration
- [x] Create sample unit tests (test_security.py, test_config.py) - [x] Create sample unit tests (test_security.py, test_config.py)
- [x] Add user and mail account factory fixtures - [x] Add user and mail account factory fixtures
- [x] Write unit tests for security module (100% coverage)
- [x] Write unit tests for middleware (98% coverage)
- [x] Write unit tests for schemas and validation
- [x] Write unit tests for application factory and core endpoints
- [x] Reach 50%+ test coverage (currently 57%)
### In Progress 🔨 ### In Progress 🔨
- [ ] Write unit tests for authentication (target 80%+ coverage) - [ ] Write unit tests for authentication (target 80%+ coverage)
@@ -88,6 +93,7 @@ Comprehensive task breakdown for repository improvements and production readines
- [x] Create `.github/workflows/lint.yml` for code quality checks - [x] Create `.github/workflows/lint.yml` for code quality checks
- [x] Create `.github/workflows/security.yml` for security scanning - [x] Create `.github/workflows/security.yml` for security scanning
- [x] Existing `.github/workflows/docker-build.yml` for Docker images - [x] Existing `.github/workflows/docker-build.yml` for Docker images
- [x] Set up automatic dependency updates (Dependabot)
### In Progress 🔨 ### In Progress 🔨
- [ ] Configure branch protection rules - [ ] Configure branch protection rules
@@ -95,7 +101,6 @@ Comprehensive task breakdown for repository improvements and production readines
### Not Started 📋 ### Not Started 📋
- [ ] Add deployment workflow (staging/production) - [ ] Add deployment workflow (staging/production)
- [ ] Set up automatic dependency updates (Dependabot)
- [ ] Add release workflow with automated changelog - [ ] Add release workflow with automated changelog
- [ ] Configure status checks for PRs - [ ] Configure status checks for PRs
- [ ] Add performance regression detection - [ ] Add performance regression detection
@@ -169,12 +174,46 @@ Comprehensive task breakdown for repository improvements and production readines
- [ ] Add advanced email filtering - [ ] Add advanced email filtering
- [ ] Implement OAuth2 for Gmail (instead of App Passwords) - [ ] Implement OAuth2 for Gmail (instead of App Passwords)
- [ ] Add attachment handling improvements - [ ] Add attachment handling improvements
- [ ] Build frontend dashboard (React/Next.js)
- [ ] Add email archiving feature - [ ] Add email archiving feature
- [ ] Implement webhook support for external integrations - [ ] Implement webhook support for external integrations
--- ---
## 🖥️ High Priority - Frontend Completion
The Next.js frontend has pages and components implemented but is **not functional**
because the API client layer is missing.
### Critical Blockers 🔴
- [ ] Create `frontend/src/lib/api.ts` — API client using axios
- Must export: `authApi`, `mailAccountsApi`, `processingRunsApi`, `userApi`
- Must export types: `User`, `MailAccount`, `MailAccountCreate`
- 8 files import from `@/lib/api` and will fail to compile without it:
`AuthGuard.tsx`, `AddMailAccountModal.tsx`, `authStore.ts`,
`login/page.tsx`, `register/page.tsx`, `auth/callback/page.tsx`,
`dashboard/page.tsx`, `accounts/page.tsx`
### Existing Pages (UI done, need API wiring) 🔨
- [x] Landing page (`app/page.tsx`)
- [x] Login page with email/password + Google OAuth
- [x] Registration page
- [x] OAuth callback handler
- [x] Dashboard with stats cards and processing runs table
- [x] Mail accounts list with CRUD operations
- [x] Settings page
- [x] `AddMailAccountModal` component (auto-detect, test connection)
- [x] `DashboardLayout` with responsive sidebar
- [x] `AuthGuard` for protected routes
### Not Started 📋
- [ ] End-to-end testing of frontend against backend API
- [ ] Error boundary components
- [ ] Loading skeletons / proper loading states
- [ ] Notification preferences UI
- [ ] Subscription management / billing UI
---
## 📅 Milestone Timeline ## 📅 Milestone Timeline
### Milestone 1: Security & Infrastructure (Week 1-2) 🔴 ### Milestone 1: Security & Infrastructure (Week 1-2) 🔴
@@ -253,31 +292,31 @@ Comprehensive task breakdown for repository improvements and production readines
| Category | Progress | Status | | Category | Progress | Status |
|----------|----------|--------| |----------|----------|--------|
| Security | 60% | 🟡 In Progress | | Security | 60% | 🟡 In Progress |
| Agentic Infrastructure | 80% | 🟢 Near Complete | | Agentic Infrastructure | 95% | 🟢 Near Complete |
| Testing | 30% | 🔴 Needs Work | | Testing | 57% | 🟡 In Progress |
| CI/CD | 70% | 🟡 In Progress | | CI/CD | 80% | 🟢 Near Complete |
| Code Quality | 40% | 🔴 Needs Work | | Code Quality | 40% | 🔴 Needs Work |
| Production Ready | 20% | 🔴 Needs Work | | Production Ready | 20% | 🔴 Needs Work |
| Observability | 10% | 🔴 Needs Work | | Observability | 10% | 🔴 Needs Work |
| Features | 70% | 🟡 In Progress | | Backend Features | 80% | 🟢 Near Complete |
| Frontend | 30% | 🔴 Blocked (missing lib/api.ts) |
**Overall Repository Readiness**: 47% ⚠️ **Overall Repository Readiness**: 52% ⚠️
--- ---
## 🎯 Next Actions (Priority Order) ## 🎯 Next Actions (Priority Order)
1. **Immediate** (Today): 1. **Immediate** (Today):
- [ ] Create `frontend/src/lib/api.ts` (frontend is broken without it)
- [ ] Fix remaining security issues (bare excepts, datetime, redirect_uri) - [ ] Fix remaining security issues (bare excepts, datetime, redirect_uri)
- [ ] Write 10 more unit tests
- [ ] Test security validators work correctly
2. **This Week**: 2. **This Week**:
- [ ] Enable rate limiting - [ ] Enable rate limiting
- [ ] Add audit logging - [ ] Add audit logging
- [ ] Reach 50% test coverage - [ ] Write more unit tests (target 70% coverage)
- [ ] Complete ADR documentation - [ ] Complete ADR documentation
- [ ] Reorganize docs into docs/ directory - [ ] End-to-end test frontend against backend
3. **Next Week**: 3. **Next Week**:
- [ ] Kubernetes manifests - [ ] Kubernetes manifests
@@ -319,6 +358,6 @@ Must complete before production:
--- ---
**Last Updated**: 2026-02-06 **Last Updated**: 2026-03-23
**Maintained By**: Development Team **Maintained By**: Development Team
**Review Frequency**: Weekly **Review Frequency**: Weekly
+1 -1
View File
@@ -156,7 +156,7 @@ def get_or_create_user_salt(user_id: int) -> bytes:
## Related Decisions ## Related Decisions
- See ADR-006 for key management in production - See ADR-006 for key management in production
- See SECURITY_REPORT.md for security analysis - See ../SECURITY_REPORT.md for security analysis
## References ## References