Commit Graph

34 Commits

Author SHA1 Message Date
Christian Krakau-Louis 8905031d16 Merge branch 'main' into copilot/implement-log-off-everywhere-functionality 2026-03-17 12:12:50 +01:00
copilot-swe-agent[bot] 03712cfb08 feat(ui): add dedicated Devices page for mobile tokens and registered devices
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 11:00:36 +00:00
copilot-swe-agent[bot] 51821092f4 feat(auth): add session management UI, QR login page, mobile QR support, translations
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:38:13 +00:00
copilot-swe-agent[bot] a88d790445 feat(system-reset): add system reset and factory reset feature
- Add FACTORY_RESET_ON_STARTUP and ENABLE_FACTORY_RESET config settings
- Create app/utils/system_reset.py with core reset logic (wipe DB + files, reimport)
- Create app/api/system_reset.py with admin-only API endpoints
- Create app/views/system_reset.py with admin-only UI view
- Create frontend/templates/system_reset.html with confirmation dialogs
- Auto-reset on startup when FACTORY_RESET_ON_STARTUP=true
- Re-import uses watch folder mechanism for re-ingestion
- Register routers in API and views init files
- Add i18n keys and SETTING_METADATA entries
- Add nav links in base.html (desktop + mobile)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:26:17 +00:00
copilot-swe-agent[bot] b0d6f1ab60 feat(profile): add user self-service profile settings page and API
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 13:00:19 +00:00
copilot-swe-agent[bot] 054af636e3 fix(migrations): merge main and rechain compliance migration as 031 after 030_add_mobile_devices
Merge main branch into compliance templates feature branch.

Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.

Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
2026-03-12 00:00:29 +00:00
copilot-swe-agent[bot] 666f739f4e feat(compliance): add GDPR, HIPAA, SOC2 compliance templates with one-click apply and dashboard
- Add ComplianceTemplate model in app/models.py
- Create database migration 027_add_compliance_templates
- Add compliance_enabled feature flag to config and settings metadata
- Create compliance_service.py with pre-built template definitions and evaluation
- Create compliance API endpoints (list, get, apply, status, summary)
- Create compliance admin view and dashboard template
- Add compliance link to admin navigation (desktop and mobile)
- Seed compliance templates at application startup
- Add comprehensive tests (29 passing)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:55:29 +00:00
copilot-swe-agent[bot] 35db9f88de feat(audit): add comprehensive audit logging with SIEM integration
- Add AuditLog model with append-only design (timestamp, user, action, resource, IP, details, severity)
- Add audit_service.py with record/query helpers and SIEM forwarding (Syslog RFC 5424, HTTP/webhook)
- Add /api/audit-logs REST endpoints with filtering and pagination
- Add /admin/audit-logs viewer UI with real-time filters
- Add SIEM config settings (syslog, HTTP for Splunk HEC/Logstash/Grafana Loki)
- Add Alembic migration 027_add_audit_logs
- Add navigation link in admin menu
- Add comprehensive tests (20 tests covering model, service, SIEM, API, view)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:18:18 +00:00
copilot-swe-agent[bot] 5bc1fac79c feat(tasks): add 5 additional scheduled jobs (expire_shared_links, prune_processing_logs, prune_old_notifications, backfill_missing_metadata, sync_search_index)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:25:26 +00:00
Christian Krakau-Louis 201d7c8fdc Merge branch 'main' into copilot/add-document-sharing-feature 2026-03-08 23:12:27 +01:00
copilot-swe-agent[bot] 0f91b8bb7c feat(sharing): add document sharing with expiring links
- Add SharedLink model with token, expiry, view limit, password hash
- Add migration 025_add_shared_links
- Add API endpoints: create, list, revoke (auth) + public info/download
- Add management UI at /shared-links with revoke controls
- Add public share landing page at /share/{token}
- Add Share button on file_view.html
- Add Shared Links to user dropdown in common.js
- Write 35 unit tests covering all scenarios
- Update UserGuide.md with sharing documentation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:51:30 +00:00
copilot-swe-agent[bot] fcefd0978f feat(notifications): add per-user notification system with inbox, email, and webhook targets
- Add UserNotificationTarget, UserNotificationPreference, InAppNotification models
- Add migration 025_add_user_notifications (tables + indexes)
- Add app/utils/user_notification.py dispatch service
- Add app/api/notifications.py REST endpoints (inbox, targets, preferences)
- Add app/views/notifications.py view route
- Add frontend/templates/notifications_dashboard.html Alpine.js dashboard
- Add bell icon with unread badge in base.html nav (desktop + mobile)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 32 unit tests in tests/test_notifications_api.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-08 21:34:25 +00:00
copilot-swe-agent[bot] c3bb93c197 feat(api): add personal API tokens and enhance webhook integration UI
- Add ApiToken model with SHA-256 hashed storage and usage tracking
- Create API token CRUD endpoints (POST/GET/DELETE /api/api-tokens/)
- Add Bearer token authentication to require_login decorator
- Exempt Bearer-authenticated requests from CSRF validation
- Add API tokens management page with create/revoke/copy UI
- Enhance webhook integration type with detailed explanation and code snippets
- Add navigation links (desktop + mobile) to API tokens page
- Include 19 tests covering CRUD, auth resolution, and utility functions
- Create migration 024_add_api_tokens

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:42:55 +00:00
copilot-swe-agent[bot] 2c652dcc3e feat(ui): add unified Sources & Destinations integrations dashboard
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:34:22 +00:00
copilot-swe-agent[bot] 43bcfe5653 feat(imap): add per-user IMAP ingestion accounts with quota enforcement
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:01:14 +00:00
copilot-swe-agent[bot] 2dd1ca0197 feat(backup): add database backup/restore with scheduled retention and admin dashboard
- Add BackupRecord model for tracking backup archives
- Add migration 021_add_backup_records
- Add backup configuration settings (backup_enabled, backup_dir, backup_remote_destination, backup_remote_folder, backup_retain_hourly/daily/weekly)
- Add backup_tasks.py with create_backup, cleanup_old_backups, and helpers
- Register hourly/daily/weekly Celery beat schedules
- Add /api/admin/backup/* REST endpoints (list, create, download, restore, delete, cleanup)
- Add /admin/backup dashboard view and template
- Add backup link to admin dropdown navigation in base.html
- Add backup settings to SETTING_METADATA in settings_service.py
- Add comprehensive test suite (39 tests passing)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 22:03:27 +00:00
copilot-swe-agent[bot] 46b2f17acc feat(docs): add built-in help section with How-To guides embedded in app
- Add MkDocs Material docs build stage to Dockerfile and Dockerfile.local
- Mount pre-built docs as static files at /help/ in FastAPI (app/main.py)
- Add app/views/help.py with /help → /help/ permanent redirect route
- Register help router in app/views/__init__.py
- Add Help nav link to base.html (public + app nav, desktop + mobile)
- Create how-to guides: HP printer, ScanSnap, watched folder, email ingestion, mobile scanning
- Update mkdocs.yml with How-To Guides section and Material theme palette
- Add optional docs service (squidfunk/mkdocs-material) to docker-compose.yaml with docs profile
- Add mkdocs-material to requirements-dev.txt
- Add /docs_build to .gitignore
- Add tests for help view (8 tests, 100% coverage on help.py)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 20:33:10 +00:00
copilot-swe-agent[bot] 02ac55ab1a Merge branch 'main' of https://github.com/christianlouis/DocuElevate into copilot/fix-pricing-page-issues 2026-03-07 14:47:29 +00:00
copilot-swe-agent[bot] 99df0816b0 feat(onboarding): add multi-step user onboarding wizard
Add a 5-step onboarding flow for new users:
- Migration 017: adds onboarding_completed, onboarding_completed_at,
  contact_email, preferred_destination to user_profiles
- app/api/onboarding.py: REST endpoints (status, profile, plan, storage,
  complete) with session-based auth using sub/preferred_username/email/id
  priority chain
- app/views/onboarding.py: GET /onboarding view with configured-destination
  detection helper for all 8 supported storage providers
- frontend/templates/onboarding.html: Alpine.js wizard with progress
  indicator, tier cards (server-rendered), storage destination cards,
  accessible markup (WCAG AA), and all fetch() API calls
- app/auth.py: redirect first-time OAuth users (onboarding_completed=False)
  to /onboarding after login
- 16 unit tests covering all endpoints, auth enforcement, and edge cases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-07 11:04:53 +00:00
copilot-swe-agent[bot] 89e0c2fb50 feat(pipelines): add custom processing pipeline engine
- Add Pipeline and PipelineStep models with user-specific ownership
- Add pipeline_id FK column to FileRecord
- Migration 017_add_pipelines (batch mode for SQLite FK compat)
- Pipeline CRUD API at /api/pipelines with step management endpoints
- Reorder steps PUT endpoint placed before parameterised {step_id} routes
- POST /api/files/{id}/assign-pipeline for per-file pipeline assignment
- Admin-only POST /api/pipelines/admin/system for system-level pipelines
- Management UI at /pipelines (Jinja2 + Alpine.js + Tailwind)
- Pipelines link added to desktop and mobile navigation
- 41 new tests in tests/test_api_pipelines.py (all passing)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 10:32:00 +00:00
copilot-swe-agent[bot] ea7fffa3a1 feat(subscriptions): database-backed plan designer with admin CRUD and overage buffer
- Add SubscriptionPlan model and subscription_plans table (migration 015)
- Add billing cycle/period/allow_overage fields to UserProfile (migration 016)
- Add subscription_overage_percent config field (replaces overage_factor)
- Rewrite check_upload_allowed: use overage_percent, yearly carry-over, no daily cap
- Add seed_default_plans(), _plan_to_dict(), get_year_file_count(), _months_elapsed()
- Update get_tier/get_all_tiers to be DB-first with TIER_DEFAULTS fallback
- Add TIER_DEFAULTS alias (TIERS kept for backward compat)
- New /api/plans/ CRUD endpoints (admin-only except list/get)
- New /admin/plans Plan Designer page with Alpine.js UI
- Add Plan Designer link to admin navigation in base.html
- Remove 'Files per day' row from pricing comparison table
- Add billing cycle + period start to admin users edit modal
- Seed default plans on startup in lifespan handler
- Rewrite docs/SubscriptionTiers.md with full plan/overage/API docs
- Fix all tests in test_subscription.py (remove daily cap tests, add overage/carry-over tests)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-06 18:48:27 +00:00
copilot-swe-agent[bot] 179f6125e8 feat(subscriptions): add SaaS subscription tiers, pricing page, and enforced upload quotas
- Add Free / Starter / Professional / Business tiers with lifetime, daily, and monthly
  file limits (app/utils/subscription.py)
- Add subscription_tier column to UserProfile model + migration 014
- Enforce quotas at upload time (HTTP 402 on violation) in /api/ui-upload
- New REST API: GET /api/subscriptions/tiers, /my, /platform (admin)
- New pages: /pricing (marketing, public) and /subscription (per-user status)
- Enhanced dashboard: SaaS stats (files today/month, OCR count, active users)
  in multi-user mode; original single-user layout preserved
- Admin users page: show Plan badge, allow tier editing via dropdown
- Navigation: add Pricing link + subscription icon in user header
- Tests: 23 unit tests for subscription tier logic
- Docs: docs/SubscriptionTiers.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 15:53:12 +00:00
copilot-swe-agent[bot] 56f7f2351f feat(auth): add admin user management dashboard
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 13:45:35 +00:00
copilot-swe-agent[bot] f6fcaaeccc feat(database): add database configuration wizard and migration tool
Add a guided database configuration wizard and a data migration tool that
allows users to:
- Build database connection strings through a step-by-step UI
- Test database connections before applying
- Preview and execute data migrations from SQLite to PostgreSQL/MySQL
- Copy to clipboard for easy .env file updates

New files:
- app/utils/db_wizard.py — connection string builder, parser, and tester
- app/utils/db_migrate.py — table-by-table data migration utility
- app/api/database.py — REST API endpoints for wizard operations
- app/views/db_wizard.py — view route for the wizard page
- frontend/templates/db_wizard.html — multi-tab wizard UI
- tests/test_db_wizard.py — unit tests for db_wizard utilities
- tests/test_db_migrate.py — unit tests for db_migrate utilities
- tests/test_db_wizard_api.py — integration tests for API and views

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 22:10:14 +00:00
copilot-swe-agent[bot] c7c5718f78 feat(queue): add queue monitoring dashboard and pending banner on files page
- Add /api/queue/stats endpoint with Redis queue lengths, Celery worker
  inspection, and DB processing summaries
- Add /api/queue/pending-count lightweight endpoint for the files page banner
- Add /admin/queue admin-only view with auto-refreshing queue dashboard
- Add queue pending banner on /files page showing queued/processing count
- Add Queue Monitor link to admin dropdown in navigation (desktop + mobile)
- Add comprehensive tests for all new endpoints and views

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-27 00:42:55 +00:00
copilot-swe-agent[bot] e9dd3ef1c2 docs(search): update UserGuide with search page documentation and fix import order
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-26 09:32:08 +00:00
copilot-swe-agent[bot] 09a6337c6f fix(ui): prevent search input from clearing on each keystroke and add dedicated /search page
The debounceSearch() function in files.html called clearFullTextSearch()
when the query was shorter than 2 characters. Since clearFullTextSearch()
sets input.value = '', every single keystroke was immediately erased —
users could paste text but not type.

Fix: debounceSearch now only hides the results panel for short queries
without touching the input value.

Also adds a dedicated /search page with Google-style results showing
content previews (document title, filename, type badges, tag badges,
sender, and OCR text snippets with highlighted matches).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-26 09:18:28 +00:00
copilot-swe-agent[bot] 16e7b6478e feat(admin): add admin-only file manager, admin menu, de-emphasize status, improve dashboard
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-21 09:15:27 +00:00
copilot-swe-agent[bot] d08040ac4a style: fix all flake8 linter errors across app/ directory
- Run Black formatter and isort on all app/ files
- Remove unused imports (F401) across multiple files
- Add # noqa: F401 for intentional re-exports in celery_worker.py,
  tasks/__init__.py, utils.py, frontend.py, views/base.py
- Fix f-strings without placeholders (F541) in azure.py, notification.py,
  check_credentials.py, upload_to_onedrive.py, settings.py
- Fix bare except (E722) in upload_to_sftp.py
- Fix block comment format (E265) in models.py
- Move imports to top of file to fix E402 in celery_app.py, celery_worker.py
- Fix line-too-long (E501) by wrapping strings in multiple files
- Remove unused variable (F841) in upload_to_nextcloud.py

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-08 17:42:33 +00:00
copilot-swe-agent[bot] 1a01811882 Add encryption and setup wizard features
ENCRYPTION:
- Add cryptography library for secure storage
- Implement Fernet encryption for sensitive settings
- Key derived from SESSION_SECRET
- Auto-encrypt/decrypt transparent to app
- "enc:" prefix identifies encrypted values
- Graceful fallback if crypto unavailable

SETUP WIZARD:
- Detect fresh installs needing configuration
- 3-step wizard: Infrastructure, Security, AI Services
- "/" redirects to wizard if setup required
- Auto-generate session_secret option
- Skip option for advanced users
- Beautiful UI with progress indicators

UI IMPROVEMENTS:
- Enhanced sensitive field display
- Lock icon showing encryption status
- Improved show/hide toggle for passwords
- Better visual hierarchy

FILES:
- app/utils/encryption.py - Encryption utilities
- app/utils/setup_wizard.py - Wizard detection logic
- app/views/wizard.py - Wizard routes
- frontend/templates/setup_wizard.html - Wizard UI
- requirements.txt - Added cryptography
- IMPLEMENTATION_CHECKLIST.md - Status tracking

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-08 06:18:49 +00:00
copilot-swe-agent[bot] e1bb9de915 Add settings management infrastructure: models, API, views, and database loading
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 22:31:06 +00:00
Christian Krakau-Louis 05ede35059 feat: Add license and attribution routes, update templates and documentation
- Added a new license route to serve the LGPL license text.
- Introduced a new attribution page to acknowledge third-party software used in the project.
- Updated the base HTML template to include a link to the attribution page.
- Included the license router in the main application router.
- Added the license text file for LGPL to the static licenses directory.
- Updated the NOTICE file to include detailed attributions for third-party libraries.
- Added a new requirements-dev.txt for license compliance checking.
- Updated the requirements.txt to clarify the LGPL license for Paramiko.
2025-04-11 02:39:26 +02:00
Christian Krakau-Louis 4c50c0aae6 Add Google Drive authorization processing and error handling templates
- Implemented `google_drive_callback.html` for processing Google Drive authorization, including UI for success and error states.
- Added JavaScript functionality for exchanging authorization codes, saving settings, and handling folder selection.
- Created `google_drive_callback_error.html` to display error messages during the authorization process.
2025-04-11 00:13:57 +02:00
Christian Krakau-Louis d23a48b1c0 feat: add frontend views and file management functionality, enhance user response structure, and update configuration guide 2025-04-03 21:49:15 +02:00