Commit Graph

222 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 82d67c56b3 feat(storage): add Apple iCloud Drive storage provider
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-10 00:43:00 +00:00
Christian Krakau-Louis 5fd3f0661b Merge pull request #577 from christianlouis/copilot/create-user-auth-workflow
fix(templates): remove orphan `</div>` in google_drive.html breaking HTML accessibility lint
2026-03-10 00:00:39 +01:00
copilot-swe-agent[bot] 256d9cb674 fix(templates): remove orphan </div> in google_drive.html causing HTML accessibility lint failure
The extra </div> at line 260 (before the {% endif %} closing the admin
auth-method block) was introduced when the per-user OAuth wizard PR
restructured the admin section. Removing it balances the div tree so
that the outer container div is properly closed by line 427.

djlint frontend/templates/ --lint now reports 0 errors across 58 files.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 21:46:38 +00:00
copilot-swe-agent[bot] 0529ae53ff fix: 100% coverage, WCAG accessibility fixes, and documentation updates for per-user OAuth wizards
Test coverage:
- Add config=None branch test for dropbox, onedrive, google_drive views (100% coverage)
- Add WATCH_FOLDER source-type test (folder_path vs folder key)
- Add integration-not-found fallback-to-admin-mode test

WCAG 2.1 AA fixes:
- Add aria-labelledby="modalTitle" to role="dialog" modals in setup templates
- Add aria-hidden="true" to decorative SVGs in callback templates
- Add role="status" aria-label="Loading" to spinner divs
- Add aria-live="polite" to processing-message and success/folder-selection regions
- Add role="alert" aria-live="assertive" to error containers
- Update "Return to Setup" link to preserve integration_id in user mode

Docs: update DropboxSetup.md, GoogleDriveSetup.md, OneDriveSetup.md with per-user OAuth flow section

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 20:19:19 +00:00
copilot-swe-agent[bot] 2d01e71afd test(views): add user-mode coverage tests and fix dashboard helper duplication
- Add tests for user-mode with invalid JSON config (covers exception handling path)
- Add tests for user-mode with valid config (verifies folder path pre-population)
- Extract _watchFolderOAuthSource() helper to reduce isOAuthType/oauthLink duplication

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:32:04 +00:00
copilot-swe-agent[bot] 5bc1fac79c feat(tasks): add 5 additional scheduled jobs (expire_shared_links, prune_processing_logs, prune_old_notifications, backfill_missing_metadata, sync_search_index)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:25:26 +00:00
copilot-swe-agent[bot] c71f33a214 feat(integrations): add per-user OAuth wizard with user-mode for Dropbox, OneDrive, Google Drive
- Add user_mode to dropbox/onedrive/google_drive setup views that loads integration config
- Show user-friendly auth wizard when integration_id is provided (user mode)
- In user mode: show integration name, current folder, back-to-integrations link
- In callback templates: only save credentials (not config) for user integrations
- In integrations dashboard: show Authorize/Re-Authorize button for all OAuth types
- Add WATCH_FOLDER OAuth support: detect source_type in config for auth button
- isOAuthType() and oauthLink() now accept full integration object

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:24:07 +00:00
copilot-swe-agent[bot] c2d56d7475 feat(tasks): add scheduled batch processing infrastructure (model, migration, tasks, API, view, template)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:01:34 +00:00
copilot-swe-agent[bot] e7d1289f1b feat(integrations): add IMAP Gmail labels/delete and Watch Folder cloud source support
- Add delete_after_process and gmail_apply_labels checkboxes to IMAP integration UI
- Add source_type selector (Local/S3/Dropbox/Google Drive/OneDrive/Nextcloud/WebDAV) to Watch Folder UI
- Add per-source-type config and credential fields for watch folders
- Add delete_after_process checkbox to Watch Folder settings
- Fix field name mismatch (path → folder_path) in Watch Folder UI
- Update pull_inbox() to accept and respect gmail_apply_labels parameter
- Update _pull_user_integration_imap() to read gmail_apply_labels from config
- Add per-user cloud scanning functions for S3, Dropbox, Google Drive, OneDrive, Nextcloud, WebDAV
- Update _pull_user_integration_watch_folders() to dispatch based on source_type
- Update IMAP and WATCH_FOLDER config shape documentation in models.py

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 02:15:03 +00:00
Christian Krakau-Louis 201d7c8fdc Merge branch 'main' into copilot/add-document-sharing-feature 2026-03-08 23:12:27 +01:00
Christian Krakau-Louis 6376b6d73b Merge pull request #568 from christianlouis/copilot/add-multi-language-ocr-support
feat(ocr): per-pipeline language override for multi-language OCR
2026-03-08 23:11:41 +01:00
copilot-swe-agent[bot] 06091546b0 fix(sharing): address security review findings
- Use per-password random salt with PBKDF2-HMAC-SHA256 (stored as salt:hash)
- Increase PBKDF2 iterations to 600,000 (OWASP 2023 recommendation)
- Password for downloads now accepted via POST body (never URL query param)
- Fail download request if view count cannot be incremented (prevents bypass)
- Update tests to match new hashing format and POST password download

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:55:55 +00:00
copilot-swe-agent[bot] 0f91b8bb7c feat(sharing): add document sharing with expiring links
- Add SharedLink model with token, expiry, view limit, password hash
- Add migration 025_add_shared_links
- Add API endpoints: create, list, revoke (auth) + public info/download
- Add management UI at /shared-links with revoke controls
- Add public share landing page at /share/{token}
- Add Share button on file_view.html
- Add Shared Links to user dropdown in common.js
- Write 35 unit tests covering all scenarios
- Update UserGuide.md with sharing documentation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:51:30 +00:00
copilot-swe-agent[bot] a2a4c6fc9a feat(ocr): add multi-language OCR support with per-pipeline language override
- Add OCR_LANGUAGES constant (28 languages, EN/DE/FR/ES/IT/PT/RU/ZH/JA/KO/AR/etc.)
- Add TESSERACT_TO_EASYOCR mapping for automatic code translation
- Add optional language constructor arg to TesseractOCRProvider/EasyOCRProvider
- Update get_ocr_providers() to accept and pass per-call language override
- Add language parameter to process_with_ocr Celery task
- Add _get_pipeline_ocr_language() helper to resolve OCR language from pipeline step config
- Update process_document to look up and pass pipeline OCR language to process_with_ocr
- Add ocr_language select config field (28 options) to pipeline OCR step schema
- Add language dropdown to pipeline UI (pipelines.html)
- Update docs/UserGuide.md and docs/API.md with language override documentation
- Add 27 new tests covering language constants, provider overrides, and pipeline lookup

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:47:12 +00:00
copilot-swe-agent[bot] d48e36813e feat(notifications): build per-user notification system (email, webhook, in-app)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:44:01 +00:00
copilot-swe-agent[bot] fcefd0978f feat(notifications): add per-user notification system with inbox, email, and webhook targets
- Add UserNotificationTarget, UserNotificationPreference, InAppNotification models
- Add migration 025_add_user_notifications (tables + indexes)
- Add app/utils/user_notification.py dispatch service
- Add app/api/notifications.py REST endpoints (inbox, targets, preferences)
- Add app/views/notifications.py view route
- Add frontend/templates/notifications_dashboard.html Alpine.js dashboard
- Add bell icon with unread badge in base.html nav (desktop + mobile)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 32 unit tests in tests/test_notifications_api.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-08 21:34:25 +00:00
Christian Krakau-Louis 8faf164e9e Merge pull request #560 from christianlouis/copilot/add-webhook-snippet-and-api-tokens
fix(security): resolve CodeQL clear-text logging and weak hashing alerts
2026-03-08 21:42:28 +01:00
Christian Krakau-Louis af696daf55 Merge pull request #558 from christianlouis/copilot/fix-user-specific-tokens
feat(auth): make OAuth credentials user-specific via UserIntegration records
2026-03-08 20:23:18 +01:00
copilot-swe-agent[bot] c3bb93c197 feat(api): add personal API tokens and enhance webhook integration UI
- Add ApiToken model with SHA-256 hashed storage and usage tracking
- Create API token CRUD endpoints (POST/GET/DELETE /api/api-tokens/)
- Add Bearer token authentication to require_login decorator
- Exempt Bearer-authenticated requests from CSRF validation
- Add API tokens management page with create/revoke/copy UI
- Enhance webhook integration type with detailed explanation and code snippets
- Add navigation links (desktop + mobile) to API tokens page
- Include 19 tests covering CRUD, auth resolution, and utility functions
- Create migration 024_add_api_tokens

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:42:55 +00:00
copilot-swe-agent[bot] 73119e0cef feat(auth): save OAuth credentials per-user to UserIntegration records
- Add 'Authorize' button to integration cards for OAuth types without credentials
- Add isOAuthType() helper and update info box in create/edit modal
- Accept integration_id query param in Dropbox, Google Drive, OneDrive setup views
- Store integration_id in sessionStorage on setup pages
- Add per-user flow in OAuth callbacks: PUT credentials to /api/integrations/{id}
- Preserve existing global flow as fallback when no integration_id is present

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-08 18:41:13 +00:00
copilot-swe-agent[bot] 8098856186 test(support): improve Zammad user context tests with meaningful assertions
Enable Zammad settings via mock and verify user data appears in the
rendered HTML. Add tests for chat widget, email-only fallback, and
display_name fallback. Also improve JS variable naming in help.html.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:33:39 +00:00
copilot-swe-agent[bot] 209e82f7e5 feat(support): auto-fill Zammad chat and ticket widgets with user context
Pass authenticated user's name, email, and username to Zammad form
and chat widgets. For the ticket form: pre-fill name/email fields
and append a User Context metadata block to the ticket body via
$.ajaxPrefilter. For the chat widget: pass name/email to the
ZammadChat constructor.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:27:14 +00:00
copilot-swe-agent[bot] ba387b8e8b fix(ui): correct OAuth setup page links in integrations dashboard
The oauthLink() function in integrations_dashboard.html returned
/dropbox, /google-drive, /onedrive which are not valid routes
(404). The actual view routes are /dropbox-setup,
/google-drive-setup, /onedrive-setup.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:21:52 +00:00
Christian Krakau-Louis f714a9ed31 Merge pull request #556 from christianlouis/copilot/feature-universal-dashboard
feat(ui): Universal Sources & Destinations integrations dashboard
2026-03-08 19:08:11 +01:00
Christian Krakau-Louis 58733d6450 Merge pull request #555 from christianlouis/copilot/restructure-help-page
feat(help): replace MkDocs redirect with user-facing Help Center + Zammad integration
2026-03-08 18:58:02 +01:00
copilot-swe-agent[bot] 0edb94158d feat(ui): replace Email Ingestion nav with unified Integrations link
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:41:49 +00:00
copilot-swe-agent[bot] e27bacee3b refactor(ui): address code review - optimize queries, extract helpers, improve error handling
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:40:23 +00:00
copilot-swe-agent[bot] 883f123138 fix(security): add SRI to jQuery CDN, escape chat ID, use server-side hostname
- Add integrity/crossorigin attributes to jQuery 3.6.0 CDN script
- Use | int filter on zammad_chat_id to prevent XSS
- Replace request.headers.get('host') with settings.external_hostname
  for canonical URL and Open Graph tags to prevent host-header injection

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:40:15 +00:00
copilot-swe-agent[bot] 2c652dcc3e feat(ui): add unified Sources & Destinations integrations dashboard
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:34:22 +00:00
copilot-swe-agent[bot] af3eed4040 feat(help): restructure /help as user-facing Help Center with Zammad integration
- Replace MkDocs redirect with a branded, SEO-optimised Help Center template
- Add sections: Quick Start, Sources, Destinations, Workflows, FAQ, Support
- Integrate optional Zammad live-chat widget and ticket form
- Add config settings: ZAMMAD_URL, ZAMMAD_CHAT_ENABLED, ZAMMAD_CHAT_ID,
  ZAMMAD_FORM_ENABLED, SUPPORT_EMAIL
- Move MkDocs developer docs from /help to /developer-docs
- Move interactive API docs (Swagger/ReDoc) to /admin/api-docs and /admin/api-redoc
- Add API Docs and Developer Docs links to Admin menu (desktop + mobile)
- Update navigation Help link from /help/ to /help
- Update .env.demo with Zammad configuration examples
- Document new settings in docs/ConfigurationGuide.md
- Rewrite tests to cover new Help Center behaviour

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:25:34 +00:00
Christian Krakau-Louis a7a807bb11 Merge pull request #551 from christianlouis/copilot/verify-stripe-billing-implementation
fix(billing): cast SQLAlchemy Column[str] to str to resolve mypy typeddict-item error
2026-03-08 18:11:58 +01:00
copilot-swe-agent[bot] 002cdf312b fix(billing): address code review - remove duplicate mode logic, accessibility improvements, robust test assertions
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:18:53 +00:00
copilot-swe-agent[bot] 223539bebb feat(billing): expose Stripe price IDs in plan API, add Stripe Setup Wizard and sync endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:14:42 +00:00
Christian Krakau-Louis 89378b2270 Merge pull request #553 from christianlouis/copilot/restrict-status-page-access
feat(status): restrict status page to admin-only, move to admin menu
2026-03-08 16:14:26 +01:00
copilot-swe-agent[bot] feea051c4f feat(status): restrict status page to admin-only and move to admin menu
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:10:46 +00:00
copilot-swe-agent[bot] 8aa8a93a6a fix(ui): move IMAP modals inside Alpine.js x-data scope so create/edit/delete buttons work
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:04:23 +00:00
copilot-swe-agent[bot] 73727dc56a fix(imap): address code review feedback - named constants, error context in JS, security docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:08:31 +00:00
copilot-swe-agent[bot] 43bcfe5653 feat(imap): add per-user IMAP ingestion accounts with quota enforcement
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:01:14 +00:00
copilot-swe-agent[bot] d36ba88de7 feat(auth): password reset, forgot username, and admin user management for local accounts
- Add /forgot-password and /forgot-username page routes and templates
- Update login page label to "Username or Email" (both already accepted by backend)
- Add "Forgot password?" and "Forgot username?" links to login page
- Add POST /api/auth/forgot-username endpoint + send_forgot_username_email() utility
- Add admin endpoints: PATCH /local/{id}, POST /local/{id}/send-password-reset, POST /local/{id}/set-password
- Update admin_users.html with Edit, Password, and Reset action buttons + modals
- Add 23 tests; fix code review issues (import style, display_name clearing behaviour)
- Update docs/API.md and docs/UserGuide.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 10:06:08 +00:00
copilot-swe-agent[bot] 44ea43f9cf chore: update plan to include forgot-username and login label clarification
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 09:44:59 +00:00
Christian Krakau-Louis bd9da65511 Merge pull request #504 from christianlouis/copilot/add-watch-folder-support 2026-03-08 10:05:10 +01:00
Christian Krakau-Louis ada6c368e4 Merge pull request #506 from christianlouis/copilot/fix-admin-user-buttons-functionality 2026-03-08 10:04:43 +01:00
Christian Krakau-Louis 5e6f44e6bf Merge pull request #507 from christianlouis/copilot/add-backup-and-restore-functionality 2026-03-08 10:04:18 +01:00
copilot-swe-agent[bot] 1877fc0000 fix(backup): address code review feedback - accessibility, CSRF, docs, imports
- Move `import os` to top-level in app/views/backup.py
- Fix docstring in BackupRecord model to remove non-existent 'location' field
- Replace browser confirm() dialogs with accessible modal dialog (role=dialog, aria-modal, aria-labelledby)
- Add csrfToken() helper that validates token presence instead of silently falling back to empty string
- Fix aria-live region to remain in DOM (screen-reader friendly) rather than using x-show
- Add Backup & Restore section to docs/ConfigurationGuide.md with retention table
- Add backup env vars to .env.demo with comments

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 22:08:47 +00:00
copilot-swe-agent[bot] 2dd1ca0197 feat(backup): add database backup/restore with scheduled retention and admin dashboard
- Add BackupRecord model for tracking backup archives
- Add migration 021_add_backup_records
- Add backup configuration settings (backup_enabled, backup_dir, backup_remote_destination, backup_remote_folder, backup_retain_hourly/daily/weekly)
- Add backup_tasks.py with create_backup, cleanup_old_backups, and helpers
- Register hourly/daily/weekly Celery beat schedules
- Add /api/admin/backup/* REST endpoints (list, create, download, restore, delete, cleanup)
- Add /admin/backup dashboard view and template
- Add backup link to admin dropdown navigation in base.html
- Add backup settings to SETTING_METADATA in settings_service.py
- Add comprehensive test suite (39 tests passing)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 22:03:27 +00:00
copilot-swe-agent[bot] 56f346ac20 fix(ui): replace Tailwind v3-only peer toggles with Alpine.js-driven toggles in admin users modal
The "Block this user" and "Complimentary plan" toggle switches were always
greyed out because the app loads Tailwind CSS v2.2.19 from CDN, but the
toggles used Tailwind v3-only features (peer, peer-checked:*, after:content-[''],
arbitrary value syntax like after:top-[2px], etc.).

Replaced both toggles with button[role=switch] elements driven by Alpine.js
@click handlers and :class bindings — fully compatible with Tailwind v2.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 21:22:58 +00:00
copilot-swe-agent[bot] db5f3d51d9 fix(ui): make pricing page CTAs link to /signup when local signup is enabled
All four CTA anchor elements on pricing.html were hardcoded to /login.
They now use /signup when allow_signup is True (MULTI_USER_ENABLED and
ALLOW_LOCAL_SIGNUP both true), falling back to /login when signup is
disabled. Bottom CTA button text also updates accordingly.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 21:18:17 +00:00
copilot-swe-agent[bot] bdb67de5cb feat(watch-folders): add local/FTP/SFTP watch folder ingest with settings, tasks, tests, docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 21:06:42 +00:00
Christian Krakau-Louis e929930bae Merge pull request #499 from christianlouis/copilot/add-subscription-management-features
Merge main → subscription-management-features; fix migration chain collision
2026-03-07 22:03:02 +01:00
Christian Krakau-Louis f53574dc33 Merge pull request #503 from christianlouis/copilot/add-help-forum-and-blog
feat(help): embed MkDocs documentation site at /help/ with How-To guides
2026-03-07 21:58:16 +01:00