Commit Graph

473 Commits

Author SHA1 Message Date
github-actions[bot] e5d4f8b58f style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-08 22:12:44 +00:00
Christian Krakau-Louis 201d7c8fdc Merge branch 'main' into copilot/add-document-sharing-feature 2026-03-08 23:12:27 +01:00
Christian Krakau-Louis 6376b6d73b Merge pull request #568 from christianlouis/copilot/add-multi-language-ocr-support
feat(ocr): per-pipeline language override for multi-language OCR
2026-03-08 23:11:41 +01:00
Christian Krakau-Louis 32db11f47f Merge pull request #571 from christianlouis/copilot/build-cli-tool-for-power-users
feat(cli): add `docuelevate` CLI tool for power users
2026-03-08 23:10:49 +01:00
Christian Krakau-Louis 0bd8b71438 Merge pull request #567 from christianlouis/copilot/build-notification-system
feat(notifications): per-user notification system with email, webhook, and in-app inbox
2026-03-08 23:10:27 +01:00
copilot-swe-agent[bot] 06091546b0 fix(sharing): address security review findings
- Use per-password random salt with PBKDF2-HMAC-SHA256 (stored as salt:hash)
- Increase PBKDF2 iterations to 600,000 (OWASP 2023 recommendation)
- Password for downloads now accepted via POST body (never URL query param)
- Fail download request if view count cannot be incremented (prevents bypass)
- Update tests to match new hashing format and POST password download

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:55:55 +00:00
copilot-swe-agent[bot] 0f91b8bb7c feat(sharing): add document sharing with expiring links
- Add SharedLink model with token, expiry, view limit, password hash
- Add migration 025_add_shared_links
- Add API endpoints: create, list, revoke (auth) + public info/download
- Add management UI at /shared-links with revoke controls
- Add public share landing page at /share/{token}
- Add Share button on file_view.html
- Add Shared Links to user dropdown in common.js
- Write 35 unit tests covering all scenarios
- Update UserGuide.md with sharing documentation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:51:30 +00:00
copilot-swe-agent[bot] a3fd74f117 feat(cli): add docuelevate CLI tool for power users
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:50:18 +00:00
copilot-swe-agent[bot] 0b291995b9 fix(ocr): address code review feedback on multi-language OCR
- Fix _get_pipeline_ocr_language: remove redundant `or None` in step_config.get()
- Add Session type hint to _get_pipeline_ocr_language db parameter via TYPE_CHECKING
- Update process_with_ocr to use modern str | None syntax instead of Optional[str]
- Fix test_get_pipeline_ocr_language_explicit_pipeline_takes_priority: properly add
  sys_step to db_session so the system pipeline step is persisted in the test DB

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:49:49 +00:00
copilot-swe-agent[bot] a2a4c6fc9a feat(ocr): add multi-language OCR support with per-pipeline language override
- Add OCR_LANGUAGES constant (28 languages, EN/DE/FR/ES/IT/PT/RU/ZH/JA/KO/AR/etc.)
- Add TESSERACT_TO_EASYOCR mapping for automatic code translation
- Add optional language constructor arg to TesseractOCRProvider/EasyOCRProvider
- Update get_ocr_providers() to accept and pass per-call language override
- Add language parameter to process_with_ocr Celery task
- Add _get_pipeline_ocr_language() helper to resolve OCR language from pipeline step config
- Update process_document to look up and pass pipeline OCR language to process_with_ocr
- Add ocr_language select config field (28 options) to pipeline OCR step schema
- Add language dropdown to pipeline UI (pipelines.html)
- Update docs/UserGuide.md and docs/API.md with language override documentation
- Add 27 new tests covering language constants, provider overrides, and pipeline lookup

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:47:12 +00:00
copilot-swe-agent[bot] d48e36813e feat(notifications): build per-user notification system (email, webhook, in-app)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:44:01 +00:00
copilot-swe-agent[bot] fcefd0978f feat(notifications): add per-user notification system with inbox, email, and webhook targets
- Add UserNotificationTarget, UserNotificationPreference, InAppNotification models
- Add migration 025_add_user_notifications (tables + indexes)
- Add app/utils/user_notification.py dispatch service
- Add app/api/notifications.py REST endpoints (inbox, targets, preferences)
- Add app/views/notifications.py view route
- Add frontend/templates/notifications_dashboard.html Alpine.js dashboard
- Add bell icon with unread badge in base.html nav (desktop + mobile)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 32 unit tests in tests/test_notifications_api.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-08 21:34:25 +00:00
copilot-swe-agent[bot] 58af9e5a29 feat(observability): add Sentry error and performance monitoring integration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:09:17 +00:00
Christian Krakau-Louis 8faf164e9e Merge pull request #560 from christianlouis/copilot/add-webhook-snippet-and-api-tokens
fix(security): resolve CodeQL clear-text logging and weak hashing alerts
2026-03-08 21:42:28 +01:00
copilot-swe-agent[bot] a7eff2c033 fix(security): address CodeQL clear-text logging and weak hashing alerts
- Convert f-string log interpolation to %s-style formatting in
  app/api/pipelines.py and app/api/saved_searches.py to prevent
  clear-text logging of request-derived data (CodeQL: clear-text
  logging of sensitive information)
- Replace plain hashlib.sha256() with PBKDF2-HMAC-SHA256 via
  hash_token() in app/auth.py for Bearer token verification,
  consistent with how tokens are stored in api_tokens.py (CodeQL:
  use of weak cryptographic hashing on sensitive data)
- Remove redundant {exc} from logger.exception() calls (the
  traceback is already captured by logger.exception())
- Update test to verify PBKDF2 hash instead of plain SHA-256

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 19:33:55 +00:00
Christian Krakau-Louis af696daf55 Merge pull request #558 from christianlouis/copilot/fix-user-specific-tokens
feat(auth): make OAuth credentials user-specific via UserIntegration records
2026-03-08 20:23:18 +01:00
Christian Krakau-Louis 1ce9e97ad0 Merge pull request #559 from christianlouis/copilot/refactor-multi-tenant-polling-engine
feat(tasks): Multi-tenant polling engine for IMAP and watch folder ingestion
2026-03-08 20:23:04 +01:00
Christian Krakau-Louis 52ebbad335 Potential fix for code scanning alert no. 344: Use of a broken or weak cryptographic hashing algorithm on sensitive data
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-03-08 20:22:39 +01:00
copilot-swe-agent[bot] 2f95febf71 fix: correct token_prefix comment to match implementation (12 chars)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:54:55 +00:00
copilot-swe-agent[bot] a9829598c5 docs: add API tokens and webhook ingestion documentation
Update API.md with API token management endpoints, usage examples,
and authentication guide. Update UserGuide.md with webhook ingestion
and API tokens sections.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:53:51 +00:00
copilot-swe-agent[bot] c3bb93c197 feat(api): add personal API tokens and enhance webhook integration UI
- Add ApiToken model with SHA-256 hashed storage and usage tracking
- Create API token CRUD endpoints (POST/GET/DELETE /api/api-tokens/)
- Add Bearer token authentication to require_login decorator
- Exempt Bearer-authenticated requests from CSRF validation
- Add API tokens management page with create/revoke/copy UI
- Enhance webhook integration type with detailed explanation and code snippets
- Add navigation links (desktop + mobile) to API tokens page
- Include 19 tests covering CRUD, auth resolution, and utility functions
- Create migration 024_add_api_tokens

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:42:55 +00:00
copilot-swe-agent[bot] 73119e0cef feat(auth): save OAuth credentials per-user to UserIntegration records
- Add 'Authorize' button to integration cards for OAuth types without credentials
- Add isOAuthType() helper and update info box in create/edit modal
- Accept integration_id query param in Dropbox, Google Drive, OneDrive setup views
- Store integration_id in sessionStorage on setup pages
- Add per-user flow in OAuth callbacks: PUT credentials to /api/integrations/{id}
- Preserve existing global flow as fallback when no integration_id is present

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-08 18:41:13 +00:00
copilot-swe-agent[bot] 019807d0f5 feat(tasks): refactor IMAP and watch folder polling to support multi-tenant user attribution
- Add owner_id parameter to pull_inbox() and fetch_attachments_and_enqueue()
  to attribute ingested documents to the correct user
- Add _pull_user_integration_imap() to poll IMAP sources from UserIntegration model
- Add _pull_user_integration_watch_folders() to scan watch folders from UserIntegration model
- Add _is_safe_watch_path() for path traversal security on user-configured paths
- Add _scan_user_watch_folder() that passes owner_id to _enqueue_file()
- Update _enqueue_file() to forward owner_id to process_document/convert_to_pdf
- Update celery beat schedule to always enable IMAP and watch folder polling
  (user integrations can exist without system-level config)
- Ensure individual connection failures don't crash the polling loop
- Update existing tests for new function signatures

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:38:22 +00:00
copilot-swe-agent[bot] 209e82f7e5 feat(support): auto-fill Zammad chat and ticket widgets with user context
Pass authenticated user's name, email, and username to Zammad form
and chat widgets. For the ticket form: pre-fill name/email fields
and append a User Context metadata block to the ticket body via
$.ajaxPrefilter. For the chat widget: pass name/email to the
ZammadChat constructor.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:27:14 +00:00
Christian Krakau-Louis f714a9ed31 Merge pull request #556 from christianlouis/copilot/feature-universal-dashboard
feat(ui): Universal Sources & Destinations integrations dashboard
2026-03-08 19:08:11 +01:00
Christian Krakau-Louis 58733d6450 Merge pull request #555 from christianlouis/copilot/restructure-help-page
feat(help): replace MkDocs redirect with user-facing Help Center + Zammad integration
2026-03-08 18:58:02 +01:00
copilot-swe-agent[bot] e27bacee3b refactor(ui): address code review - optimize queries, extract helpers, improve error handling
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:40:23 +00:00
copilot-swe-agent[bot] 883f123138 fix(security): add SRI to jQuery CDN, escape chat ID, use server-side hostname
- Add integrity/crossorigin attributes to jQuery 3.6.0 CDN script
- Use | int filter on zammad_chat_id to prevent XSS
- Replace request.headers.get('host') with settings.external_hostname
  for canonical URL and Open Graph tags to prevent host-header injection

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:40:15 +00:00
copilot-swe-agent[bot] 351d6f3f4f fix: update docs endpoint test and add Zammad settings metadata
- Update test_docs_endpoint to use new /admin/api-docs path
- Add SETTING_METADATA entries for zammad_url, zammad_chat_enabled,
  zammad_chat_id, zammad_form_enabled, support_email

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:37:29 +00:00
copilot-swe-agent[bot] 2c652dcc3e feat(ui): add unified Sources & Destinations integrations dashboard
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:34:22 +00:00
copilot-swe-agent[bot] af3eed4040 feat(help): restructure /help as user-facing Help Center with Zammad integration
- Replace MkDocs redirect with a branded, SEO-optimised Help Center template
- Add sections: Quick Start, Sources, Destinations, Workflows, FAQ, Support
- Integrate optional Zammad live-chat widget and ticket form
- Add config settings: ZAMMAD_URL, ZAMMAD_CHAT_ENABLED, ZAMMAD_CHAT_ID,
  ZAMMAD_FORM_ENABLED, SUPPORT_EMAIL
- Move MkDocs developer docs from /help to /developer-docs
- Move interactive API docs (Swagger/ReDoc) to /admin/api-docs and /admin/api-redoc
- Add API Docs and Developer Docs links to Admin menu (desktop + mobile)
- Update navigation Help link from /help/ to /help
- Update .env.demo with Zammad configuration examples
- Document new settings in docs/ConfigurationGuide.md
- Rewrite tests to cover new Help Center behaviour

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:25:34 +00:00
Christian Krakau-Louis a7a807bb11 Merge pull request #551 from christianlouis/copilot/verify-stripe-billing-implementation
fix(billing): cast SQLAlchemy Column[str] to str to resolve mypy typeddict-item error
2026-03-08 18:11:58 +01:00
Christian Krakau-Louis 6cdca24333 Potential fix for code scanning alert no. 332: Information exposure through an exception
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-03-08 18:11:38 +01:00
copilot-swe-agent[bot] c4b4643f6e fix(integrations): sanitize error messages and add SSRF protection for WebDAV test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:57:27 +00:00
copilot-swe-agent[bot] 35abdb0e69 feat(integrations): add quota enforcement, connection test, and quota endpoint
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:45:41 +00:00
copilot-swe-agent[bot] 9627667d82 fix(billing): cast plan.name to str to fix mypy typeddict-item error
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:33:57 +00:00
Christian Krakau-Louis 48e3552e1a Potential fix for code scanning alert no. 333: Information exposure through an exception
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-03-08 16:27:53 +01:00
copilot-swe-agent[bot] 002cdf312b fix(billing): address code review - remove duplicate mode logic, accessibility improvements, robust test assertions
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:18:53 +00:00
copilot-swe-agent[bot] 223539bebb feat(billing): expose Stripe price IDs in plan API, add Stripe Setup Wizard and sync endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:14:42 +00:00
Christian Krakau-Louis 89378b2270 Merge pull request #553 from christianlouis/copilot/restrict-status-page-access
feat(status): restrict status page to admin-only, move to admin menu
2026-03-08 16:14:26 +01:00
copilot-swe-agent[bot] feea051c4f feat(status): restrict status page to admin-only and move to admin menu
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:10:46 +00:00
copilot-swe-agent[bot] 743627ecc2 feat(models): add UserIntegration model, encrypt IMAP passwords, add integrations API
- Add IntegrationDirection/IntegrationType constants and UserIntegration SQLAlchemy model
  with owner_id, direction, integration_type, name, config (JSON), credentials (encrypted),
  is_active, last_used_at, last_error, created_at, updated_at fields
- Fix UserImapAccount password encryption: encrypt_value() on create/update,
  decrypt_value() in IMAP tasks and test-connection endpoint (backward compatible)
- Add Alembic migration 023_add_user_integrations
- Add app/api/integrations.py: CRUD + credentials endpoint with owner-scoped access
- Register integrations router in app/api/__init__.py
- Add 39 tests in tests/test_api_integrations.py covering model, CRUD, encryption

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:02:38 +00:00
Christian Krakau-Louis 6175489ba0 Merge pull request #543 from christianlouis/copilot/add-per-user-imap-ingestion
feat(imap): per-user IMAP ingestion accounts with quota enforcement
2026-03-08 15:45:25 +01:00
copilot-swe-agent[bot] 087f3c10e9 fix(auth): cache request body in CSRF middleware to prevent login failures
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:48:17 +00:00
copilot-swe-agent[bot] 73727dc56a fix(imap): address code review feedback - named constants, error context in JS, security docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:08:31 +00:00
copilot-swe-agent[bot] 43bcfe5653 feat(imap): add per-user IMAP ingestion accounts with quota enforcement
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:01:14 +00:00
copilot-swe-agent[bot] c5b330cb4e feat(auth): add comprehensive debug logging for local login failures
Add detailed diagnostic log statements throughout the local authentication
path to help identify why valid local user logins are failing.

Changes:
- app/auth.py: log received username, multi_user_enabled status, LocalUser
  DB lookup result, is_active status, password verification outcome, and
  the specific failure reason (empty_username / wrong_password / no_match)
  at every decision point. Also log form keys and Content-Type header on
  empty-username failures to detect Starlette body-consumption issues.
- app/middleware/csrf.py: log Content-Type, form field names, and whether
  the CSRF token was present in _get_submitted_token() to reveal if the
  middleware is consuming form data before the endpoint can read it.
- app/utils/local_auth.py: verify_password() now logs DEBUG on mismatch
  and WARNING (with exception type) on unexpected bcrypt errors instead
  of silently swallowing exceptions.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 12:48:38 +00:00
copilot-swe-agent[bot] 86f9f5f9b1 fix(auth): activate account on password reset and fix is_active check order
- reset_password sets is_active=True so users with unverified accounts
  can log in after using the forgot-password flow
- admin set_password also sets is_active=True for the same reason
- auth() now checks is_active before verifying the password, ensuring
  inactive users always see the email-verification prompt regardless of
  password correctness (avoids leaking password validity)"

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 12:07:25 +00:00
copilot-swe-agent[bot] 2d754d52ef fix(merge): resolve conflicts with main v0.92.0 keeping path-param regression tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 11:19:22 +00:00
copilot-swe-agent[bot] c31b72810e fix(merge): resolve tests/test_auth.py conflict keeping all tests from both branches
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 11:11:21 +00:00