🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add DocumentComment and DocumentAnnotation models to app/models.py
- Create migration 041_add_document_comments_and_annotations
- Add API endpoints for CRUD operations on comments and annotations
- Add threaded comment support with parent_id relationships
- Add @mention extraction from comment body text
- Add resolve/unresolve comment thread endpoint
- Add mentionable users endpoint (GET /api/users/mentionable)
- Add 43 unit tests covering all endpoints and edge cases
- Add 29 i18n translation keys to en.json
- Update API documentation in docs/API.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/3894af37-0f19-457b-8811-f1feb18b17ef
Resolve all merge conflicts between our automation feature branch and
current main (v0.163.0, 920 commits ahead).
Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
(classification_rules, qr_auth, sessions, system_reset)
- app/config.py: add main's new settings (dropbox_use_global_credentials,
factory_reset_on_startup, enable_factory_reset)
- app/models.py: add main's new models (ClassificationRuleModel, UserSession,
QRLoginChallenge, SharePoint integration type)
- app/utils/settings_service.py: merge automation_hooks_enabled with main's
new metadata entries
- docs/API.md: merge automation API docs with main's classification rules docs
- docs/ConfigurationGuide.md: add factory reset settings
- tests/conftest.py: import both AutomationHook and new main models
Migration renumbered:
- 037_add_automation_hooks → 040_add_automation_hooks
- down_revision: 039_add_classification_rules (was 036_add_document_translation_fields)
- Chain: 036 → 037 → 038 → 039 → 040 (automation hooks)
For all non-automation files with conflicts, main's version was taken since
our branch did not modify those files (conflicts were from a stale prior merge).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/cb62f012-3b69-4415-835e-3857ce3e9f45
Add REST hooks subscription endpoints, incoming action endpoints, and
Zapier-compatible flat payload format for automation platform integration.
- AutomationHook model for webhook subscriptions
- POST /api/automation/hooks/subscribe and DELETE /hooks/{id}
- GET /api/automation/triggers/sample/{event} for Zapier field mapping
- POST /api/automation/actions/upload for incoming document uploads
- Celery task with retry for async hook delivery
- Integration with existing webhook dispatch flow
- 30 passing tests covering all new functionality
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add SharedLink model with token, expiry, view limit, password hash
- Add migration 025_add_shared_links
- Add API endpoints: create, list, revoke (auth) + public info/download
- Add management UI at /shared-links with revoke controls
- Add public share landing page at /share/{token}
- Add Share button on file_view.html
- Add Shared Links to user dropdown in common.js
- Write 35 unit tests covering all scenarios
- Update UserGuide.md with sharing documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Pipeline and PipelineStep models with user-specific ownership
- Add pipeline_id FK column to FileRecord
- Migration 017_add_pipelines (batch mode for SQLite FK compat)
- Pipeline CRUD API at /api/pipelines with step management endpoints
- Reorder steps PUT endpoint placed before parameterised {step_id} routes
- POST /api/files/{id}/assign-pipeline for per-file pipeline assignment
- Admin-only POST /api/pipelines/admin/system for system-level pipelines
- Management UI at /pipelines (Jinja2 + Alpine.js + Tailwind)
- Pipelines link added to desktop and mobile navigation
- 41 new tests in tests/test_api_pipelines.py (all passing)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add SubscriptionPlan model and subscription_plans table (migration 015)
- Add billing cycle/period/allow_overage fields to UserProfile (migration 016)
- Add subscription_overage_percent config field (replaces overage_factor)
- Rewrite check_upload_allowed: use overage_percent, yearly carry-over, no daily cap
- Add seed_default_plans(), _plan_to_dict(), get_year_file_count(), _months_elapsed()
- Update get_tier/get_all_tiers to be DB-first with TIER_DEFAULTS fallback
- Add TIER_DEFAULTS alias (TIERS kept for backward compat)
- New /api/plans/ CRUD endpoints (admin-only except list/get)
- New /admin/plans Plan Designer page with Alpine.js UI
- Add Plan Designer link to admin navigation in base.html
- Remove 'Files per day' row from pricing comparison table
- Add billing cycle + period start to admin users edit modal
- Seed default plans on startup in lifespan handler
- Rewrite docs/SubscriptionTiers.md with full plan/overage/API docs
- Fix all tests in test_subscription.py (remove daily cap tests, add overage/carry-over tests)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add Free / Starter / Professional / Business tiers with lifetime, daily, and monthly
file limits (app/utils/subscription.py)
- Add subscription_tier column to UserProfile model + migration 014
- Enforce quotas at upload time (HTTP 402 on violation) in /api/ui-upload
- New REST API: GET /api/subscriptions/tiers, /my, /platform (admin)
- New pages: /pricing (marketing, public) and /subscription (per-user status)
- Enhanced dashboard: SaaS stats (files today/month, OCR count, active users)
in multi-user mode; original single-user layout preserved
- Admin users page: show Plan badge, allow tier editing via dropdown
- Navigation: add Pricing link + subscription icon in user header
- Tests: 23 unit tests for subscription tier logic
- Docs: docs/SubscriptionTiers.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add a guided database configuration wizard and a data migration tool that
allows users to:
- Build database connection strings through a step-by-step UI
- Test database connections before applying
- Preview and execute data migrations from SQLite to PostgreSQL/MySQL
- Copy to clipboard for easy .env file updates
New files:
- app/utils/db_wizard.py — connection string builder, parser, and tester
- app/utils/db_migrate.py — table-by-table data migration utility
- app/api/database.py — REST API endpoints for wizard operations
- app/views/db_wizard.py — view route for the wizard page
- frontend/templates/db_wizard.html — multi-tab wizard UI
- tests/test_db_wizard.py — unit tests for db_wizard utilities
- tests/test_db_migrate.py — unit tests for db_migrate utilities
- tests/test_db_wizard_api.py — integration tests for API and views
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add near_duplicate_threshold config setting (default 0.85)
- New GET /api/duplicates endpoint listing all exact-duplicate groups
- New GET /api/files/{id}/duplicates endpoint returning exact + near-duplicates
- POST /api/ui-upload now returns immediate exact-duplicate warning (respects ENABLE_DEDUPLICATION)
- New /duplicates management UI with Exact Duplicates tab and Near-Duplicate Finder tab
- Add Duplicates link in admin nav menu (desktop + mobile)
- Document new config options in ConfigurationGuide.md and .env.demo
- 20 new tests covering all acceptance criteria
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add WebhookConfig model, CRUD API endpoints, HMAC-SHA256 signed delivery,
and Celery-based async dispatch with retry/backoff for document events
(document.uploaded, document.processed, document.failed).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add date range (date_from/date_to), storage provider, and tags filters to GET /api/files
- Add SavedSearch model and migration (005_add_saved_searches)
- Add CRUD API endpoints for saved searches at /api/saved-searches
- Update files.html template with new filter controls and saved searches UI
- Update files view to pass new filter parameters to template
- Add comprehensive tests for all new functionality (26 tests)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Run Black formatter and isort on all app/ files
- Remove unused imports (F401) across multiple files
- Add # noqa: F401 for intentional re-exports in celery_worker.py,
tasks/__init__.py, utils.py, frontend.py, views/base.py
- Fix f-strings without placeholders (F541) in azure.py, notification.py,
check_credentials.py, upload_to_onedrive.py, settings.py
- Fix bare except (E722) in upload_to_sftp.py
- Fix block comment format (E265) in models.py
- Move imports to top of file to fix E402 in celery_app.py, celery_worker.py
- Fix line-too-long (E501) by wrapping strings in multiple files
- Remove unused variable (F841) in upload_to_nextcloud.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Added database logging to all major processing tasks
- Created API endpoints for retrieving processing logs
- Updated frontend to display processing logs per file
- Logging includes: process_document, convert_to_pdf, extract_metadata_with_gpt, embed_metadata_into_pdf, finalize_document_storage, send_to_all_destinations
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Implemented `google_drive_callback.html` for processing Google Drive authorization, including UI for success and error states.
- Added JavaScript functionality for exchanging authorization codes, saving settings, and handling folder selection.
- Created `google_drive_callback_error.html` to display error messages during the authorization process.
copilot-swe-agent[bot]