Commit Graph

204 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 59d851272d feat(i18n): update language support to 77 languages
- Remove Russian (ru) per requirement
- Keep Esperanto (eo) per user feedback
- Add 29 new languages: am, az, bn, gu, ha, hi, hy, id, ig, jv, ka,
  kk, km, ml, mn, mr, ms, ne, si, sw, ta, te, th, tl, ur, uz, yo,
  zh-TW, zu
- Create 29 new translation JSON files seeded from en.json
- Remove frontend/translations/ru.json
- Update SUPPORTED_LANGUAGES list and docstring (49 → 77)
- Update tests: count 49→77, code set updated
- Update docs/InternationalizationGuide.md: count and table

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 18:36:07 +00:00
copilot-swe-agent[bot] 87d85624be feat(i18n): expand supported languages from 31 to 49 with flags and locale formats
Add 18 previously missing language entries to SUPPORTED_LANGUAGES in
app/utils/i18n.py so every translation JSON file in frontend/translations/
is properly indexed and served by the language selector:

New languages: af, ar, cy, eo, fa, fy, gl, he, ja, kn, ko, li, nds, no, pa, sr, vi, vls

Each new language entry includes:
- Correct ISO 639-1/639-3 code matching its JSON filename
- Native name and display name
- Appropriate country/language flag emoji
- Locale-specific date and number formatting rules in _LOCALE_FORMATS

Also: rename nb "Norwegian" → "Norwegian Bokmål" to distinguish it from no "Norwegian".

Update tests/test_i18n.py:
- Count assertions: 31 → 49
- Expected code set expanded to all 49 codes
- Fix test_unsupported_language_fallback (ja/ko now supported, use xx/yy)

Update docs/InternationalizationGuide.md:
- Language count: 10 → 49
- Full language table with flags, native names, tiers
- Complete file structure listing all 49 JSON files

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 14:03:31 +00:00
copilot-swe-agent[bot] f70fa65969 Merge branch 'main' into copilot/add-conditional-routing
Resolve conflicts in app/api/__init__.py and app/models.py.
Renumber migration 027_add_routing_rules → 035_add_routing_rules.
Fix migration chain: down_revision → 034_add_user_profile_settings.
Add PipelineRoutingRule to migrations/env.py.
2026-03-12 22:07:54 +00:00
Christian Krakau-Louis 7f644a2388 Merge pull request #604 from christianlouis/copilot/configure-attachment-ingestion
feat(imap): fine-grained attachment ingestion profiles with per-category selection
2026-03-12 08:53:45 +01:00
Christian Krakau-Louis 87fca21c7f Merge pull request #605 from christianlouis/copilot/enable-disable-storage-locations
feat(storage): explicit enable/disable flag per global storage destination
2026-03-12 08:51:14 +01:00
copilot-swe-agent[bot] 2e087731b9 fix(imap): address code review feedback on ingestion profiles
- Improve warning log in _resolve_categories_for_profile() to include
  exception type name for better troubleshooting
- Add SQLAlchemy IS NULL comment to imap_profiles.py filter
- Pass default_categories from server to template to avoid hardcoded
  category list in JS (now uses {{ default_categories | tojson }})
- Simplify view profiles query (remove redundant unauthenticated path)
- Update docs: ConfigurationGuide.md and EmailIngestion.md with
  full profiles documentation including category table and API reference

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 01:19:32 +00:00
copilot-swe-agent[bot] fd75961580 docs(storage): document explicit enable/disable flags in ConfigurationGuide and .env.demo
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 01:10:31 +00:00
copilot-swe-agent[bot] 554bb21d32 feat(imap): add attachment type filter for IMAP ingestion
Add a configurable switch to control which attachment types are ingested
via IMAP. Images are excluded by default; office files and PDFs are ingested.

- Add global `IMAP_ATTACHMENT_FILTER` config setting (default: `documents_only`)
- Add `attachment_filter` column to `UserImapAccount` model for per-user override
- Migration 032 adds the column to `user_imap_accounts` table
- Update `fetch_attachments_and_enqueue()` to respect filter (documents_only/all)
- Update `pull_inbox()`, `_pull_user_imap_accounts()`, and
  `_pull_user_integration_imap()` to pass the resolved filter
- Update IMAP accounts API (schemas, create/update handlers, response serializer)
- Update IMAP accounts UI to show attachment filter dropdown in modal and
  display filter badges on account cards
- Add 6 new tests covering attachment filter behaviour
- Update ConfigurationGuide.md, EmailIngestion.md, and .env.demo

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 00:59:54 +00:00
Christian Krakau-Louis 0aa1df3920 Merge pull request #588 from christianlouis/copilot/add-social-logins-configuration
feat(auth): add social login support for Google, Microsoft, Apple, and Dropbox
2026-03-12 01:40:04 +01:00
Christian Krakau-Louis 80dd117ebf Merge pull request #587 from christianlouis/copilot/add-apple-drive-icloud-support
feat(storage): add Apple iCloud Drive storage provider
2026-03-12 01:14:37 +01:00
copilot-swe-agent[bot] 054af636e3 fix(migrations): merge main and rechain compliance migration as 031 after 030_add_mobile_devices
Merge main branch into compliance templates feature branch.

Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.

Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
2026-03-12 00:00:29 +00:00
copilot-swe-agent[bot] 528f0a624d fix: remove accidental pip artifact file and update docs for iCloud Drive
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 23:38:35 +00:00
copilot-swe-agent[bot] a41ded535f feat(api): add GraphQL endpoint at /graphql with Strawberry
- Add strawberry-graphql[fastapi] dependency
- Implement GraphQL schema covering documents, pipelines, settings, users
- Enable GraphiQL playground at /graphql
- Mount GraphQL router in main.py
- Add 24 tests for all query types and auth enforcement
- Update docs/API.md with GraphQL documentation section

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 22:17:25 +00:00
copilot-swe-agent[bot] 6bcabfef77 fix(merge): resolve conflicts in app/api/__init__.py and migrations/env.py against main
Merge main into branch to pull in translation files, audit log templates,
CI workflow updates, and other changes from PRs #595, #597, #598.
Keep mobile_router and MobileDevice additions from this branch.
2026-03-11 21:53:00 +00:00
copilot-swe-agent[bot] a66eb4b2a2 fix: merge main into i18n branch and relink migration to 029
- Resolve conflict in app/api/__init__.py (keep both audit_logs_router and i18n_router)
- Incorporate AuditLog model, audit_service, audit_logs API/views from main
- Relink migration from 026→027 to 028→029 (chain after 028_add_audit_logs)
- Update migrations/env.py with full model import list from main

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-10 21:59:58 +00:00
copilot-swe-agent[bot] d538c0879d feat(mobile): add iOS/Android mobile app with SSO login, camera upload, and push notifications
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-10 09:52:02 +00:00
copilot-swe-agent[bot] ac6e052788 feat(auth): add social login support for Google, Microsoft, Apple, and Dropbox
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-10 00:12:15 +00:00
copilot-swe-agent[bot] 491c424580 docs(compliance): add ComplianceGuide.md and update ConfigurationGuide with compliance_enabled setting
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-10 00:05:44 +00:00
copilot-swe-agent[bot] 1e5e35a26a refactor(routing): address code review feedback - simplify list filter, fix docs example
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:38:25 +00:00
copilot-swe-agent[bot] e95693d684 docs(routing): add routing rules documentation to API.md and UserGuide.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:36:13 +00:00
copilot-swe-agent[bot] c286e1b394 test(i18n): add comprehensive tests and documentation for i18n system
- Add 45 tests covering translation files, translate(), Accept-Language
  parsing, language detection, l10n formatters, and API endpoints
- Create InternationalizationGuide.md documentation
- Fix linting issues (E741, PLW2901)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:35:32 +00:00
Christian Krakau-Louis 5fd3f0661b Merge pull request #577 from christianlouis/copilot/create-user-auth-workflow
fix(templates): remove orphan `</div>` in google_drive.html breaking HTML accessibility lint
2026-03-10 00:00:39 +01:00
copilot-swe-agent[bot] 0529ae53ff fix: 100% coverage, WCAG accessibility fixes, and documentation updates for per-user OAuth wizards
Test coverage:
- Add config=None branch test for dropbox, onedrive, google_drive views (100% coverage)
- Add WATCH_FOLDER source-type test (folder_path vs folder key)
- Add integration-not-found fallback-to-admin-mode test

WCAG 2.1 AA fixes:
- Add aria-labelledby="modalTitle" to role="dialog" modals in setup templates
- Add aria-hidden="true" to decorative SVGs in callback templates
- Add role="status" aria-label="Loading" to spinner divs
- Add aria-live="polite" to processing-message and success/folder-selection regions
- Add role="alert" aria-live="assertive" to error containers
- Update "Return to Setup" link to preserve integration_id in user mode

Docs: update DropboxSetup.md, GoogleDriveSetup.md, OneDriveSetup.md with per-user OAuth flow section

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 20:19:19 +00:00
copilot-swe-agent[bot] 5bc1fac79c feat(tasks): add 5 additional scheduled jobs (expire_shared_links, prune_processing_logs, prune_old_notifications, backfill_missing_metadata, sync_search_index)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:25:26 +00:00
copilot-swe-agent[bot] 8c62a6b714 test(integrations): add tests for gmail_apply_labels and cloud watch folder dispatch
docs(integrations): update UserGuide and ConfigurationGuide for new IMAP and Watch Folder features

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 02:35:30 +00:00
Christian Krakau-Louis 0b9bba7158 Merge pull request #562 from christianlouis/copilot/refactor-dynamic-routing-user-destinations
fix(tasks): register upload_to_user_integration in Celery and add handler coverage
2026-03-09 01:05:36 +01:00
Christian Krakau-Louis e916001e08 Merge pull request #573 from christianlouis/copilot/fix-sentry-settings-startup
fix(sentry): initialize Sentry after DB settings load in lifespan
2026-03-09 00:46:36 +01:00
copilot-swe-agent[bot] 5864a4ed06 fix(sentry): initialize Sentry after DB settings are loaded in lifespan
Move init_sentry() from module level into the FastAPI lifespan context
manager, immediately after load_settings_from_db() completes. This
ensures that SENTRY_DSN and other Sentry settings configured via the
database admin UI are picked up on every restart.

Also update tests and docs accordingly.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 22:17:18 +00:00
Christian Krakau-Louis 201d7c8fdc Merge branch 'main' into copilot/add-document-sharing-feature 2026-03-08 23:12:27 +01:00
Christian Krakau-Louis 6376b6d73b Merge pull request #568 from christianlouis/copilot/add-multi-language-ocr-support
feat(ocr): per-pipeline language override for multi-language OCR
2026-03-08 23:11:41 +01:00
Christian Krakau-Louis 32db11f47f Merge pull request #571 from christianlouis/copilot/build-cli-tool-for-power-users
feat(cli): add `docuelevate` CLI tool for power users
2026-03-08 23:10:49 +01:00
Christian Krakau-Louis 0bd8b71438 Merge pull request #567 from christianlouis/copilot/build-notification-system
feat(notifications): per-user notification system with email, webhook, and in-app inbox
2026-03-08 23:10:27 +01:00
copilot-swe-agent[bot] 0f91b8bb7c feat(sharing): add document sharing with expiring links
- Add SharedLink model with token, expiry, view limit, password hash
- Add migration 025_add_shared_links
- Add API endpoints: create, list, revoke (auth) + public info/download
- Add management UI at /shared-links with revoke controls
- Add public share landing page at /share/{token}
- Add Share button on file_view.html
- Add Shared Links to user dropdown in common.js
- Write 35 unit tests covering all scenarios
- Update UserGuide.md with sharing documentation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:51:30 +00:00
copilot-swe-agent[bot] a3fd74f117 feat(cli): add docuelevate CLI tool for power users
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:50:18 +00:00
copilot-swe-agent[bot] a2a4c6fc9a feat(ocr): add multi-language OCR support with per-pipeline language override
- Add OCR_LANGUAGES constant (28 languages, EN/DE/FR/ES/IT/PT/RU/ZH/JA/KO/AR/etc.)
- Add TESSERACT_TO_EASYOCR mapping for automatic code translation
- Add optional language constructor arg to TesseractOCRProvider/EasyOCRProvider
- Update get_ocr_providers() to accept and pass per-call language override
- Add language parameter to process_with_ocr Celery task
- Add _get_pipeline_ocr_language() helper to resolve OCR language from pipeline step config
- Update process_document to look up and pass pipeline OCR language to process_with_ocr
- Add ocr_language select config field (28 options) to pipeline OCR step schema
- Add language dropdown to pipeline UI (pipelines.html)
- Update docs/UserGuide.md and docs/API.md with language override documentation
- Add 27 new tests covering language constants, provider overrides, and pipeline lookup

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:47:12 +00:00
copilot-swe-agent[bot] ae075908d1 docs: add per-user notification system documentation to ConfigurationGuide.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:44:54 +00:00
copilot-swe-agent[bot] 89bf07d2ef feat(tasks): dynamic routing to user-specific destination integrations
- Add app/tasks/upload_to_user_integration.py: new Celery task that
  uploads a processed document to a specific UserIntegration using its
  own per-user config and Fernet-decrypted credentials. Supports all
  DESTINATION types: Dropbox, S3, Google Drive, OneDrive, WebDAV,
  Nextcloud, FTP, SFTP, Paperless-ngx, Email (SMTP), and Rclone.

- Extend app/tasks/send_to_all.py: add send_to_user_destinations task
  (queries active DESTINATION UserIntegrations for an owner and
  dispatches one upload_to_user_integration task per integration) and
  get_user_destination_count helper used by finalize_document_storage.

- Refactor app/tasks/finalize_document_storage.py: after processing,
  look up the document owner; if the owner has active DESTINATION
  integrations route exclusively to those (user-specific routing),
  otherwise fall back to the global send_to_all_destinations.

- Update tests/test_finalize_storage.py: add autouse fixture to prevent
  Redis hangs, update all existing tests with new mock parameters, add
  TestFinalizeDocumentStorageUserRouting class with four new tests that
  validate user-specific vs global routing decisions.

- Add tests/test_user_integration_upload.py: 14 new unit tests covering
  upload_to_user_integration (handler dispatch, error persistence,
  last_used_at update, credential decryption, skip for unknown types)
  and send_to_user_destinations / get_user_destination_count.

- Update docs/StorageArchitecture.md: document the user-specific
  destination routing feature, supported types, multiple-destination
  behaviour, and global fallback semantics.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:24:14 +00:00
Christian Krakau-Louis caf860dd10 Merge pull request #565 from christianlouis/copilot/add-sentry-integration
feat(observability): integrate Sentry for error tracking and performance monitoring
2026-03-08 22:24:13 +01:00
copilot-swe-agent[bot] 58af9e5a29 feat(observability): add Sentry error and performance monitoring integration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:09:17 +00:00
copilot-swe-agent[bot] 6d533aae33 feat(ci): integrate Codecov for test coverage analysis and reporting
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:00:57 +00:00
Christian Krakau-Louis 8faf164e9e Merge pull request #560 from christianlouis/copilot/add-webhook-snippet-and-api-tokens
fix(security): resolve CodeQL clear-text logging and weak hashing alerts
2026-03-08 21:42:28 +01:00
Christian Krakau-Louis 1ce9e97ad0 Merge pull request #559 from christianlouis/copilot/refactor-multi-tenant-polling-engine
feat(tasks): Multi-tenant polling engine for IMAP and watch folder ingestion
2026-03-08 20:23:04 +01:00
copilot-swe-agent[bot] a9829598c5 docs: add API tokens and webhook ingestion documentation
Update API.md with API token management endpoints, usage examples,
and authentication guide. Update UserGuide.md with webhook ingestion
and API tokens sections.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:53:51 +00:00
copilot-swe-agent[bot] fc453be71f docs(config): document per-user IMAP and watch folder integration polling
Update ConfigurationGuide.md to document:
- Per-user IMAP integrations via the Integrations dashboard
- Per-user watch folder integrations with path traversal protection
- Updated unclaimed documents section to clarify attribution behavior

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:50:49 +00:00
copilot-swe-agent[bot] 209e82f7e5 feat(support): auto-fill Zammad chat and ticket widgets with user context
Pass authenticated user's name, email, and username to Zammad form
and chat widgets. For the ticket form: pre-fill name/email fields
and append a User Context metadata block to the ticket body via
$.ajaxPrefilter. For the chat widget: pass name/email to the
ZammadChat constructor.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:27:14 +00:00
Christian Krakau-Louis f714a9ed31 Merge pull request #556 from christianlouis/copilot/feature-universal-dashboard
feat(ui): Universal Sources & Destinations integrations dashboard
2026-03-08 19:08:11 +01:00
Christian Krakau-Louis 58733d6450 Merge pull request #555 from christianlouis/copilot/restructure-help-page
feat(help): replace MkDocs redirect with user-facing Help Center + Zammad integration
2026-03-08 18:58:02 +01:00
copilot-swe-agent[bot] e27bacee3b refactor(ui): address code review - optimize queries, extract helpers, improve error handling
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:40:23 +00:00
copilot-swe-agent[bot] af3eed4040 feat(help): restructure /help as user-facing Help Center with Zammad integration
- Replace MkDocs redirect with a branded, SEO-optimised Help Center template
- Add sections: Quick Start, Sources, Destinations, Workflows, FAQ, Support
- Integrate optional Zammad live-chat widget and ticket form
- Add config settings: ZAMMAD_URL, ZAMMAD_CHAT_ENABLED, ZAMMAD_CHAT_ID,
  ZAMMAD_FORM_ENABLED, SUPPORT_EMAIL
- Move MkDocs developer docs from /help to /developer-docs
- Move interactive API docs (Swagger/ReDoc) to /admin/api-docs and /admin/api-redoc
- Add API Docs and Developer Docs links to Admin menu (desktop + mobile)
- Update navigation Help link from /help/ to /help
- Update .env.demo with Zammad configuration examples
- Document new settings in docs/ConfigurationGuide.md
- Rewrite tests to cover new Help Center behaviour

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:25:34 +00:00
Christian Krakau-Louis a7a807bb11 Merge pull request #551 from christianlouis/copilot/verify-stripe-billing-implementation
fix(billing): cast SQLAlchemy Column[str] to str to resolve mypy typeddict-item error
2026-03-08 18:11:58 +01:00