Commit Graph

59 Commits

Author SHA1 Message Date
Christian Krakau-Louis 6f5a73f98a Merge pull request #586 from christianlouis/copilot/fix-watch-folder-settings
feat(integrations): simplify OAuth watch folder setup with system credentials and folder browser
2026-03-21 00:37:49 +01:00
copilot-swe-agent[bot] ae675485af Merge remote-tracking branch 'origin/main' into copilot/fix-watch-folder-settings
# Conflicts:
#	app/views/dropbox.py
#	frontend/templates/dropbox.html
#	frontend/templates/integrations_dashboard.html
#	tests/test_api_dropbox.py
#	tests/test_views_dropbox.py
2026-03-20 23:33:58 +00:00
copilot-swe-agent[bot] d6c21b8026 docs: update setup guides and API docs for folder browser and system credentials
Updated DropboxSetup.md, OneDriveSetup.md, and GoogleDriveSetup.md to
document the new system credentials toggle and folder browser features.
Added API documentation for POST /api/dropbox/list-folders and
POST /api/onedrive/list-folders endpoints.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 13:43:08 +00:00
copilot-swe-agent[bot] 1afd42bc57 fix: merge main (v0.161.0) into classification feature branch
Resolve all 40 merge conflicts from merging origin/main into the
classification feature branch. Key resolutions:

- Auto-generated files (BUILD_DATE, VERSION, etc.): use main's version
- API tokens: take main's version (token expiry, reactivation, hard-delete)
- Auth: take main's Dropbox credential sharing + token expiry checking
- Config: take main's social_auth_dropbox_use_global_credentials option
- Files API: take main's improved duplicate handling + rate limiting
- Models: keep ClassificationRuleModel alongside main's new models
- Mobile: take main's mature implementation
- Templates/translations: take main's versions (device deletion, reactivation keys)
- Migration: renumber 038_add_classification_rules → 039_add_classification_rules
  to chain after main's 038_add_api_token_expires_at
- Requirements: take main's version (adds segno QR library)
- Tests: take main's more complete token tests, keep classification imports
2026-03-20 13:08:41 +00:00
Christian Krakau-Louis 91eecd9396 Merge pull request #753 from christianlouis/copilot/scale-worker-and-api-pods
feat(scaling): enable horizontal scaling for API and worker pods
2026-03-20 13:55:31 +01:00
copilot-swe-agent[bot] d1f9819f4e feat(integrations): add Dropbox connection test and global-credential sharing
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-19 19:17:12 +00:00
copilot-swe-agent[bot] d5c18ccf07 fix(upload): reject exact duplicates at upload time and prevent duplicate mobile uploads
- Move duplicate check before task enqueue in ui_upload endpoint
- Clean up temp file and return status "duplicate" for exact duplicates
- Add URI-level dedup guard in mobile UploadScreen to prevent repeated uploads
- Improve ShareContext URI normalization (collapse slashes, decode percent-encoding)
- Guard +not-found.tsx effect against re-firing for the same pathname
- Update mobile UploadResponse type and handlers for duplicate status
- Update web frontend upload.js to show duplicate status
- Update API and Configuration docs

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-19 14:07:43 +00:00
copilot-swe-agent[bot] 2941f6e177 fix: resolve merge conflict in database.py
Merge origin/main into branch, resolving conflict in app/database.py.
Combined improvements from both branches:
- Keep pool_pre_ping=True and structured variable approach from feature branch
- Add explicit QueuePool import and poolclass assignment from main
2026-03-19 10:40:43 +00:00
copilot-swe-agent[bot] e4749b4e7c feat(api): allow disabled tokens/devices to be deleted & reactivated; add token lifetime
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-18 23:43:16 +00:00
copilot-swe-agent[bot] 571cc81789 feat(api): add per-user health-aware upload rate limiting
Introduces a Redis-backed sliding-window rate limiter for upload
endpoints (/api/ui-upload, /api/process-url) that:
- Enforces per-user limits (default: 20 uploads / 60 s)
- Dynamically reduces limits under system stress (queue depth, CPU load)
- Returns 429 with Retry-After header when exceeded
- Fails open when Redis is unavailable
- Works with the existing client-side adaptive back-off

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 12:12:50 +00:00
copilot-swe-agent[bot] c7d67031d4 fix: resolve merge conflicts with main branch
Merge origin/main into feature branch, resolving 18 conflicts:
- app/api/__init__.py: add classification_rules_router alongside system_reset_router
- app/config.py: keep system reset settings from main
- app/models.py: keep ClassificationRuleModel alongside main's models
- app/utils/settings_service.py: keep factory reset settings from main
- app/views/__init__.py: keep system_reset_router from main
- tests/conftest.py: add ClassificationRuleModel import
- migrations/env.py: add ClassificationRuleModel import
- .env.demo, docs/*, frontend/*, mobile/*: keep additions from main
- BUILD_DATE, GIT_SHA, RUNTIME_INFO, VERSION, CHANGELOG.md: accept main's version

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 11:09:39 +00:00
copilot-swe-agent[bot] 4d019d53d9 docs: update scaling, health probe, and beat scheduler documentation
- Update DeploymentGuide.md with scaling instructions and beat service info
- Update KubernetesDeployment.md with unauthenticated probe paths and beat note
- Update ProductionReadiness.md with new health endpoints table and beat guidance
- Update API.md with new healthz/live and healthz/ready endpoint docs

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 00:08:17 +00:00
copilot-swe-agent[bot] 651b48658c fix: resolve merge conflicts with main branch
Merge origin/main into feature branch, resolving 3 conflicts:
- app/api/__init__.py: add classification_rules_router alongside new
  routers from main (audit_logs, i18n, mobile, compliance, translation)
- app/models.py: keep ClassificationRuleModel alongside new models from
  main (MobileDevice, ComplianceTemplate, PipelineRoutingRule)
- tests/conftest.py: import both ClassificationRuleModel and new models
  from main (AuditLog, ComplianceTemplate)

Also renumber migration from 027 to 037 to chain from the latest
migration on main (036_add_document_translation_fields).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:33:59 +00:00
copilot-swe-agent[bot] 96bfba8057 test(system-reset): add comprehensive tests and documentation
- 21 tests covering unit, integration, API, and view layers
- Update ConfigurationGuide.md with System Reset section
- Update API.md with system reset endpoint docs
- All tests pass, ruff clean

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:32:31 +00:00
copilot-swe-agent[bot] f70fa65969 Merge branch 'main' into copilot/add-conditional-routing
Resolve conflicts in app/api/__init__.py and app/models.py.
Renumber migration 027_add_routing_rules → 035_add_routing_rules.
Fix migration chain: down_revision → 034_add_user_profile_settings.
Add PipelineRoutingRule to migrations/env.py.
2026-03-12 22:07:54 +00:00
copilot-swe-agent[bot] a41ded535f feat(api): add GraphQL endpoint at /graphql with Strawberry
- Add strawberry-graphql[fastapi] dependency
- Implement GraphQL schema covering documents, pipelines, settings, users
- Enable GraphiQL playground at /graphql
- Mount GraphQL router in main.py
- Add 24 tests for all query types and auth enforcement
- Update docs/API.md with GraphQL documentation section

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 22:17:25 +00:00
copilot-swe-agent[bot] d538c0879d feat(mobile): add iOS/Android mobile app with SSO login, camera upload, and push notifications
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-10 09:52:02 +00:00
copilot-swe-agent[bot] bd898605de docs: add classification rules API and user guide documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:39:09 +00:00
copilot-swe-agent[bot] e95693d684 docs(routing): add routing rules documentation to API.md and UserGuide.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:36:13 +00:00
Christian Krakau-Louis 6376b6d73b Merge pull request #568 from christianlouis/copilot/add-multi-language-ocr-support
feat(ocr): per-pipeline language override for multi-language OCR
2026-03-08 23:11:41 +01:00
copilot-swe-agent[bot] a3fd74f117 feat(cli): add docuelevate CLI tool for power users
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:50:18 +00:00
copilot-swe-agent[bot] a2a4c6fc9a feat(ocr): add multi-language OCR support with per-pipeline language override
- Add OCR_LANGUAGES constant (28 languages, EN/DE/FR/ES/IT/PT/RU/ZH/JA/KO/AR/etc.)
- Add TESSERACT_TO_EASYOCR mapping for automatic code translation
- Add optional language constructor arg to TesseractOCRProvider/EasyOCRProvider
- Update get_ocr_providers() to accept and pass per-call language override
- Add language parameter to process_with_ocr Celery task
- Add _get_pipeline_ocr_language() helper to resolve OCR language from pipeline step config
- Update process_document to look up and pass pipeline OCR language to process_with_ocr
- Add ocr_language select config field (28 options) to pipeline OCR step schema
- Add language dropdown to pipeline UI (pipelines.html)
- Update docs/UserGuide.md and docs/API.md with language override documentation
- Add 27 new tests covering language constants, provider overrides, and pipeline lookup

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:47:12 +00:00
copilot-swe-agent[bot] a9829598c5 docs: add API tokens and webhook ingestion documentation
Update API.md with API token management endpoints, usage examples,
and authentication guide. Update UserGuide.md with webhook ingestion
and API tokens sections.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:53:51 +00:00
copilot-swe-agent[bot] 71350f2ad6 docs(api): add integrations quota enforcement and test endpoint documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:54:45 +00:00
copilot-swe-agent[bot] d36ba88de7 feat(auth): password reset, forgot username, and admin user management for local accounts
- Add /forgot-password and /forgot-username page routes and templates
- Update login page label to "Username or Email" (both already accepted by backend)
- Add "Forgot password?" and "Forgot username?" links to login page
- Add POST /api/auth/forgot-username endpoint + send_forgot_username_email() utility
- Add admin endpoints: PATCH /local/{id}, POST /local/{id}/send-password-reset, POST /local/{id}/set-password
- Update admin_users.html with Edit, Password, and Reset action buttons + modals
- Add 23 tests; fix code review issues (import style, display_name clearing behaviour)
- Update docs/API.md and docs/UserGuide.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 10:06:08 +00:00
copilot-swe-agent[bot] fdc48c7fe9 feat(notifications): admin push notifications and webhooks for user signup, plan changes, and payment issues
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 20:15:39 +00:00
copilot-swe-agent[bot] 1203a4b75f refactor(pipelines): address code review - shared get_current_user_id, aria-live, deduplicate user ID logic
- Extract _get_user_id into shared auth.get_current_user_id() used by both
  pipelines API and the assign-pipeline endpoint in files API
- Fix aria-live attribute: use two separate static containers (polite/assertive)
  instead of dynamic Alpine.js binding for correct screen reader announcements
- Fix migration comment to accurately describe batch-mode FK creation
- Remove redundant tags parameter from reorder endpoint decorator
- Rename _make_file test helper to _make_test_file_record for clarity
- Update docs/UserGuide.md and docs/API.md with full Pipelines reference

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 10:39:19 +00:00
copilot-swe-agent[bot] 56f7f2351f feat(auth): add admin user management dashboard
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 13:45:35 +00:00
Christian Krakau-Louis 26b4d04d76 Merge pull request #482 from christianlouis/copilot/add-database-configuration-wizard
feat(database): integrate wizard into settings page, improve accessibility and test coverage
2026-03-06 11:28:44 +01:00
copilot-swe-agent[bot] cb3bf809cc docs(database): add wizard and migration tool documentation
Update DatabaseConfiguration.md with sections for the new Database
Configuration Wizard and Database Migration Tool. Also update API.md
with the new /api/database/ endpoint documentation.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 22:19:39 +00:00
copilot-swe-agent[bot] c8bc4afc93 fix: address code review — use modern type hints and Callable annotation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 22:03:30 +00:00
copilot-swe-agent[bot] 3601e2ca5c feat(ui): add user autocomplete widget for default_owner_id, user search API, and documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 21:40:41 +00:00
copilot-swe-agent[bot] 8e955f3c81 fix(similarity): truncate text to fit embedding model context window, fix step tracking
- Add EMBEDDING_MAX_TOKENS config (default 8000) for safe text truncation
- Use conservative 3 chars/token estimate (was 4) to prevent ContextWindowExceededError
- Add compute_embedding to REAL_MAIN_STEPS in both get_file_overall_status and get_step_summary
- Fix test_near_duplicates_returned to use pre-computed embeddings
- Update .env.demo and docs with EMBEDDING_MAX_TOKENS setting

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 13:27:34 +00:00
Christian Krakau-Louis 1e1ba1d8d7 Merge branch 'main' into copilot/add-document-similarity-detection 2026-03-01 22:06:55 +01:00
copilot-swe-agent[bot] a85992ee79 docs(webhooks): fix spelling - Behaviour to Behavior
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:47:51 +00:00
copilot-swe-agent[bot] b7c78177e9 docs(similarity): add API documentation and fix template accessibility
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:46:13 +00:00
copilot-swe-agent[bot] a953b726fc docs(webhooks): add webhook API and configuration documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:45:20 +00:00
Christian Krakau-Louis 68ea6aabc7 Merge pull request #448 from christianlouis/copilot/add-retry-logic-backoff 2026-03-01 20:30:45 +01:00
copilot-swe-agent[bot] 85da309740 feat(api): add GET /api/diagnostic/health endpoint for monitoring
- Add health check endpoint at GET /api/diagnostic/health
- Auth-protected via @require_login (no-op when AUTH_ENABLED=False)
- Checks database (SELECT 1) and Redis (ping) with 2s timeouts
- Returns healthy/degraded/unhealthy with per-check detail
- Returns HTTP 503 when database is down, 200 otherwise
- 7 new unit tests covering all status scenarios
- Update docs/API.md with Grafana/monitoring integration notes
- Fixes test_cors_headers_absent_when_disabled CI timeout

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 18:36:38 +00:00
copilot-swe-agent[bot] af34ce88df feat(api): implement proper pagination for file list API
- Change default per_page from 50 to 25
- Rename total_items → total, total_pages → pages in pagination response
- Add next/previous URL fields to pagination response
- Update view and template to use new field names
- Update tests and API docs

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 17:31:56 +00:00
Christian Krakau-Louis 96b14ec74f Merge pull request #440 from christianlouis/copilot/update-search-functionality
feat(search): add content-finding filters, saved searches, and text quality to Search view
2026-03-01 15:19:57 +01:00
copilot-swe-agent[bot] de09dba285 docs: update API and User Guide with search filters and saved searches
- Document new /api/search endpoint with all filter parameters
- Update saved searches docs with new allowed filter keys
- Update User Guide with search view filters and saved searches usage

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:55:55 +00:00
copilot-swe-agent[bot] 83c3405c98 feat: persist ocr_quality_score and use it for numeric filtering
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:46:43 +00:00
copilot-swe-agent[bot] 705b970522 feat: add bulk download, cloud OCR, and basic OCR quality filter
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:38:42 +00:00
Christian Krakau-Louis 05c35a6775 Merge pull request #437 from christianlouis/copilot/add-file-download-endpoint
fix(api): change download endpoint default version from `original` to `processed`
2026-03-01 13:30:30 +01:00
copilot-swe-agent[bot] cc9fd1a7fe docs(api): document advanced filtering and saved searches endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 11:29:47 +00:00
copilot-swe-agent[bot] 7b5494bafb feat(api): default download endpoint to processed file version
- Change default `version` param from 'original' to 'processed' so
  GET /api/files/{id}/download (no param) returns the processed file
- Update docstring to reflect new default
- Add tests: ?version=processed, default→processed, invalid→400
- Fix test_file_download_missing_mime_type to use explicit ?version=original
- Add File Download section to docs/API.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 11:15:14 +00:00
copilot-swe-agent[bot] 4dd018f210 refactor(queue): address code review feedback — extract constants and sync refresh interval
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-27 00:47:58 +00:00
copilot-swe-agent[bot] b03ebab747 chore: apply ruff formatting and fix whitespace issues
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-13 09:12:11 +00:00
copilot-swe-agent[bot] 817dc31e12 feat(browser): implement browser extension for sending files to DocuElevate
- Add complete browser extension with popup UI and background workers
- Support Chrome, Firefox, Edge, and other Chromium-based browsers
- Include context menu integration for quick file sending
- Add comprehensive documentation for users and administrators
- Update main README and API docs to include browser extension
- Implement secure configuration storage in browser extension storage
- Add SSRF-protected URL upload endpoint integration

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-12 03:04:36 +00:00