Commit Graph

43 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 4b6412734b fix(comments): address code review feedback
- Add onupdate=sa.func.now() to migration updated_at columns
- Use UserProfile.is_blocked.is_(False) instead of == False
- Fix British to American spelling (organised → organized)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/3894af37-0f19-457b-8811-f1feb18b17ef
2026-03-21 18:30:05 +00:00
copilot-swe-agent[bot] ad795e200a feat(comments): add document comments, annotations, and @mention support
- Add DocumentComment and DocumentAnnotation models to app/models.py
- Create migration 041_add_document_comments_and_annotations
- Add API endpoints for CRUD operations on comments and annotations
- Add threaded comment support with parent_id relationships
- Add @mention extraction from comment body text
- Add resolve/unresolve comment thread endpoint
- Add mentionable users endpoint (GET /api/users/mentionable)
- Add 43 unit tests covering all endpoints and edge cases
- Add 29 i18n translation keys to en.json
- Update API documentation in docs/API.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/3894af37-0f19-457b-8811-f1feb18b17ef
2026-03-21 18:28:01 +00:00
copilot-swe-agent[bot] e518bce922 fix: merge main branch and renumber migration 037→040
Resolve all merge conflicts between our automation feature branch and
current main (v0.163.0, 920 commits ahead).

Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
  (classification_rules, qr_auth, sessions, system_reset)
- app/config.py: add main's new settings (dropbox_use_global_credentials,
  factory_reset_on_startup, enable_factory_reset)
- app/models.py: add main's new models (ClassificationRuleModel, UserSession,
  QRLoginChallenge, SharePoint integration type)
- app/utils/settings_service.py: merge automation_hooks_enabled with main's
  new metadata entries
- docs/API.md: merge automation API docs with main's classification rules docs
- docs/ConfigurationGuide.md: add factory reset settings
- tests/conftest.py: import both AutomationHook and new main models

Migration renumbered:
- 037_add_automation_hooks → 040_add_automation_hooks
- down_revision: 039_add_classification_rules (was 036_add_document_translation_fields)
- Chain: 036 → 037 → 038 → 039 → 040 (automation hooks)

For all non-automation files with conflicts, main's version was taken since
our branch did not modify those files (conflicts were from a stale prior merge).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/cb62f012-3b69-4415-835e-3857ce3e9f45
2026-03-20 23:54:04 +00:00
copilot-swe-agent[bot] 204000aabc fix: merge main branch and renumber migration 027→037
Resolve 3 merge conflicts and renumber the automation_hooks migration
to follow main's migration chain (036_add_document_translation_fields).

Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
- app/utils/settings_service.py: add automation_hooks_enabled alongside compliance_enabled
- tests/conftest.py: add AutomationHook alongside AuditLog/ComplianceTemplate imports

Migration renumbered:
- 027_add_automation_hooks → 037_add_automation_hooks
- down_revision: 026_add_scheduled_jobs → 036_add_document_translation_fields

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:40:15 +00:00
copilot-swe-agent[bot] d167be8274 feat(automation): add Zapier and Make.com integration
Add REST hooks subscription endpoints, incoming action endpoints, and
Zapier-compatible flat payload format for automation platform integration.

- AutomationHook model for webhook subscriptions
- POST /api/automation/hooks/subscribe and DELETE /hooks/{id}
- GET /api/automation/triggers/sample/{event} for Zapier field mapping
- POST /api/automation/actions/upload for incoming document uploads
- Celery task with retry for async hook delivery
- Integration with existing webhook dispatch flow
- 30 passing tests covering all new functionality

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:50:12 +00:00
copilot-swe-agent[bot] c2d56d7475 feat(tasks): add scheduled batch processing infrastructure (model, migration, tasks, API, view, template)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:01:34 +00:00
copilot-swe-agent[bot] c374ba3ae9 fix(migrations): resolve duplicate Alembic heads causing test failures
Two migration files (025_add_shared_links and 025_add_user_notifications)
both had down_revision pointing to 024_add_api_tokens, creating multiple
migration heads. This caused alembic stamp/upgrade head to fail with
'Multiple heads are present; please specify a single target revision'
whenever init_db() ran (e.g. in TestClient fixtures).

Fix: chain 025_add_user_notifications from 025_add_shared_links so the
migration history is linear:
024_add_api_tokens → 025_add_shared_links → 025_add_user_notifications

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 01:11:26 +00:00
Christian Krakau-Louis 201d7c8fdc Merge branch 'main' into copilot/add-document-sharing-feature 2026-03-08 23:12:27 +01:00
copilot-swe-agent[bot] 0f91b8bb7c feat(sharing): add document sharing with expiring links
- Add SharedLink model with token, expiry, view limit, password hash
- Add migration 025_add_shared_links
- Add API endpoints: create, list, revoke (auth) + public info/download
- Add management UI at /shared-links with revoke controls
- Add public share landing page at /share/{token}
- Add Share button on file_view.html
- Add Shared Links to user dropdown in common.js
- Write 35 unit tests covering all scenarios
- Update UserGuide.md with sharing documentation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:51:30 +00:00
copilot-swe-agent[bot] fcefd0978f feat(notifications): add per-user notification system with inbox, email, and webhook targets
- Add UserNotificationTarget, UserNotificationPreference, InAppNotification models
- Add migration 025_add_user_notifications (tables + indexes)
- Add app/utils/user_notification.py dispatch service
- Add app/api/notifications.py REST endpoints (inbox, targets, preferences)
- Add app/views/notifications.py view route
- Add frontend/templates/notifications_dashboard.html Alpine.js dashboard
- Add bell icon with unread badge in base.html nav (desktop + mobile)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 32 unit tests in tests/test_notifications_api.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-08 21:34:25 +00:00
copilot-swe-agent[bot] c3bb93c197 feat(api): add personal API tokens and enhance webhook integration UI
- Add ApiToken model with SHA-256 hashed storage and usage tracking
- Create API token CRUD endpoints (POST/GET/DELETE /api/api-tokens/)
- Add Bearer token authentication to require_login decorator
- Exempt Bearer-authenticated requests from CSRF validation
- Add API tokens management page with create/revoke/copy UI
- Enhance webhook integration type with detailed explanation and code snippets
- Add navigation links (desktop + mobile) to API tokens page
- Include 19 tests covering CRUD, auth resolution, and utility functions
- Create migration 024_add_api_tokens

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:42:55 +00:00
copilot-swe-agent[bot] 743627ecc2 feat(models): add UserIntegration model, encrypt IMAP passwords, add integrations API
- Add IntegrationDirection/IntegrationType constants and UserIntegration SQLAlchemy model
  with owner_id, direction, integration_type, name, config (JSON), credentials (encrypted),
  is_active, last_used_at, last_error, created_at, updated_at fields
- Fix UserImapAccount password encryption: encrypt_value() on create/update,
  decrypt_value() in IMAP tasks and test-connection endpoint (backward compatible)
- Add Alembic migration 023_add_user_integrations
- Add app/api/integrations.py: CRUD + credentials endpoint with owner-scoped access
- Register integrations router in app/api/__init__.py
- Add 39 tests in tests/test_api_integrations.py covering model, CRUD, encryption

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:02:38 +00:00
copilot-swe-agent[bot] 43bcfe5653 feat(imap): add per-user IMAP ingestion accounts with quota enforcement
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:01:14 +00:00
copilot-swe-agent[bot] 2dd1ca0197 feat(backup): add database backup/restore with scheduled retention and admin dashboard
- Add BackupRecord model for tracking backup archives
- Add migration 021_add_backup_records
- Add backup configuration settings (backup_enabled, backup_dir, backup_remote_destination, backup_remote_folder, backup_retain_hourly/daily/weekly)
- Add backup_tasks.py with create_backup, cleanup_old_backups, and helpers
- Register hourly/daily/weekly Celery beat schedules
- Add /api/admin/backup/* REST endpoints (list, create, download, restore, delete, cleanup)
- Add /admin/backup dashboard view and template
- Add backup link to admin dropdown navigation in base.html
- Add backup settings to SETTING_METADATA in settings_service.py
- Add comprehensive test suite (39 tests passing)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 22:03:27 +00:00
copilot-swe-agent[bot] 699a93e193 chore: merge main into branch, resolve conflict in models.py
Both sets of UserProfile columns are retained:
- is_complimentary (from main, migration 019_add_is_complimentary)
- subscription_change_pending_tier / subscription_change_pending_date
  (our branch, renamed to migration 020_add_subscription_change_pending
   with down_revision updated to chain after 019_add_is_complimentary)
2026-03-07 20:53:42 +00:00
copilot-swe-agent[bot] 97f85ce74e feat(auth): auto-create admin user profiles with highest tier and complimentary flag
- Add `is_complimentary` column to UserProfile model (migration 019)
- Update `_ensure_user_profile` to accept `is_admin` param; admins get
  highest subscription tier, is_complimentary=True, onboarding skipped
- Call `_ensure_user_profile` from all login paths (OAuth, local user, admin creds)
- Add `is_complimentary` to UserProfileUpsert schema, response helpers,
  list_users, get_user, upsert_user_profile in admin API
- Add complimentary toggle to admin users UI with gift badge in table
- Write 18 new tests covering complimentary plan and admin auto-creation
- Update SubscriptionTiers.md documentation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 20:14:34 +00:00
copilot-swe-agent[bot] 231f983429 feat(subscriptions): add subscription change management with upgrade/downgrade scheduling
- Add subscription_change_pending_tier and subscription_change_pending_date fields to UserProfile
- Create migration 019_add_subscription_change_pending
- Add apply_pending_subscription_changes(), request_subscription_change(), cancel_pending_subscription_change() utilities
- Add POST /api/subscriptions/change and DELETE /api/subscriptions/change endpoints
- Update GET /api/subscriptions/my to apply pending changes and return pending change info
- Update subscription view to apply pending changes and pass period_start + pending info
- Update subscription.html: per-tier action buttons (upgrade/downgrade/cancel), pending-change banner, period start date
- Add Celery daily task apply_pending_subscription_changes_all at 00:05 UTC
- Add 19 new tests covering all new utility functions and API endpoints

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 20:14:12 +00:00
copilot-swe-agent[bot] b1ce28f804 fix: resolve merge conflict with main, fix test failures
- Merge main (pipelines feature) into branch, resolving conflicts in
  app/api/__init__.py and app/views/__init__.py by keeping all routers
  (onboarding + billing from our branch, pipelines from main)
- Fix migration 018 down_revision to depend on both 017_add_onboarding_fields
  and 017_add_pipelines (Alembic multi-head merge pattern)
- Fix test_auth_module.py: add multi_user_enabled=False to three admin-auth
  tests that call auth() directly without FastAPI DI
- Add missing SETTING_METADATA entries for allow_local_signup and all five
  Stripe config keys (fixes test_all_config_settings_have_metadata)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 15:02:43 +00:00
copilot-swe-agent[bot] 02ac55ab1a Merge branch 'main' of https://github.com/christianlouis/DocuElevate into copilot/fix-pricing-page-issues 2026-03-07 14:47:29 +00:00
copilot-swe-agent[bot] 6a967051ba fix: address code review feedback
- Use astimezone() instead of replace() for timezone conversion in is_token_expired
- Log cleanup exceptions with logger.exception() in signup
- Add security warning when STRIPE_WEBHOOK_SECRET is not configured
- Increase Stripe price ID column length from 64 to 128 characters
- Replace alert() with aria-live assertive region in pricing.html
- Convert auth() login tests to use pytest.mark.asyncio and await

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-07 13:18:33 +00:00
copilot-swe-agent[bot] 52e3852129 feat(auth): add local user signup, email verification, and Stripe billing
- Add LocalUser model with bcrypt password hashing, email verification
  tokens, and password reset tokens
- Add ALLOW_LOCAL_SIGNUP config flag (requires SMTP to be configured)
- Add Stripe billing config fields (STRIPE_SECRET_KEY, etc.)
- Add stripe_customer_id to UserProfile and stripe_price_id_monthly/
  stripe_price_id_yearly to SubscriptionPlan
- Create migration 018_add_local_users_and_billing
- Add app/utils/local_auth.py: hash_password, verify_password,
  generate_token, is_token_expired, send_verification_email,
  send_password_reset_email, build_session_user
- Add app/api/local_auth.py: signup, email verification, password reset
  endpoints plus signup/verify-email-sent/reset-password page routes
- Add app/api/billing.py: Stripe Checkout, Customer Portal, and webhook
  endpoints; syncs subscription tier from webhook events
- Update auth() to check LocalUser table before admin credentials fallback
- Update login() to pass allow_signup context variable to template
- Add signup.html, verify_email_sent.html, password_reset_form.html,
  billing_success.html templates (Alpine.js, Tailwind, WCAG 2.1 AA)
- Update login.html to show 'Create account' link when signup enabled
- Update pricing.html CTA buttons to use Stripe Checkout for paid tiers
- Add docs/BillingSetup.md with setup guide, webhook config, compliance
- Add tests/test_local_auth.py (42 tests) and tests/test_billing.py
  (31 tests); all 106 tests in the modified test suite pass
- Add stripe>=7.0.0,<15.0.0 to requirements.txt

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-07 13:15:15 +00:00
copilot-swe-agent[bot] 99df0816b0 feat(onboarding): add multi-step user onboarding wizard
Add a 5-step onboarding flow for new users:
- Migration 017: adds onboarding_completed, onboarding_completed_at,
  contact_email, preferred_destination to user_profiles
- app/api/onboarding.py: REST endpoints (status, profile, plan, storage,
  complete) with session-based auth using sub/preferred_username/email/id
  priority chain
- app/views/onboarding.py: GET /onboarding view with configured-destination
  detection helper for all 8 supported storage providers
- frontend/templates/onboarding.html: Alpine.js wizard with progress
  indicator, tier cards (server-rendered), storage destination cards,
  accessible markup (WCAG AA), and all fetch() API calls
- app/auth.py: redirect first-time OAuth users (onboarding_completed=False)
  to /onboarding after login
- 16 unit tests covering all endpoints, auth enforcement, and edge cases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-07 11:04:53 +00:00
copilot-swe-agent[bot] 1203a4b75f refactor(pipelines): address code review - shared get_current_user_id, aria-live, deduplicate user ID logic
- Extract _get_user_id into shared auth.get_current_user_id() used by both
  pipelines API and the assign-pipeline endpoint in files API
- Fix aria-live attribute: use two separate static containers (polite/assertive)
  instead of dynamic Alpine.js binding for correct screen reader announcements
- Fix migration comment to accurately describe batch-mode FK creation
- Remove redundant tags parameter from reorder endpoint decorator
- Rename _make_file test helper to _make_test_file_record for clarity
- Update docs/UserGuide.md and docs/API.md with full Pipelines reference

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 10:39:19 +00:00
copilot-swe-agent[bot] 89e0c2fb50 feat(pipelines): add custom processing pipeline engine
- Add Pipeline and PipelineStep models with user-specific ownership
- Add pipeline_id FK column to FileRecord
- Migration 017_add_pipelines (batch mode for SQLite FK compat)
- Pipeline CRUD API at /api/pipelines with step management endpoints
- Reorder steps PUT endpoint placed before parameterised {step_id} routes
- POST /api/files/{id}/assign-pipeline for per-file pipeline assignment
- Admin-only POST /api/pipelines/admin/system for system-level pipelines
- Management UI at /pipelines (Jinja2 + Alpine.js + Tailwind)
- Pipelines link added to desktop and mobile navigation
- 41 new tests in tests/test_api_pipelines.py (all passing)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 10:32:00 +00:00
copilot-swe-agent[bot] ea7fffa3a1 feat(subscriptions): database-backed plan designer with admin CRUD and overage buffer
- Add SubscriptionPlan model and subscription_plans table (migration 015)
- Add billing cycle/period/allow_overage fields to UserProfile (migration 016)
- Add subscription_overage_percent config field (replaces overage_factor)
- Rewrite check_upload_allowed: use overage_percent, yearly carry-over, no daily cap
- Add seed_default_plans(), _plan_to_dict(), get_year_file_count(), _months_elapsed()
- Update get_tier/get_all_tiers to be DB-first with TIER_DEFAULTS fallback
- Add TIER_DEFAULTS alias (TIERS kept for backward compat)
- New /api/plans/ CRUD endpoints (admin-only except list/get)
- New /admin/plans Plan Designer page with Alpine.js UI
- Add Plan Designer link to admin navigation in base.html
- Remove 'Files per day' row from pricing comparison table
- Add billing cycle + period start to admin users edit modal
- Seed default plans on startup in lifespan handler
- Rewrite docs/SubscriptionTiers.md with full plan/overage/API docs
- Fix all tests in test_subscription.py (remove daily cap tests, add overage/carry-over tests)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-06 18:48:27 +00:00
copilot-swe-agent[bot] 179f6125e8 feat(subscriptions): add SaaS subscription tiers, pricing page, and enforced upload quotas
- Add Free / Starter / Professional / Business tiers with lifetime, daily, and monthly
  file limits (app/utils/subscription.py)
- Add subscription_tier column to UserProfile model + migration 014
- Enforce quotas at upload time (HTTP 402 on violation) in /api/ui-upload
- New REST API: GET /api/subscriptions/tiers, /my, /platform (admin)
- New pages: /pricing (marketing, public) and /subscription (per-user status)
- Enhanced dashboard: SaaS stats (files today/month, OCR count, active users)
  in multi-user mode; original single-user layout preserved
- Admin users page: show Plan badge, allow tier editing via dropdown
- Navigation: add Pricing link + subscription icon in user header
- Tests: 23 unit tests for subscription tier logic
- Docs: docs/SubscriptionTiers.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 15:53:12 +00:00
copilot-swe-agent[bot] 56f7f2351f feat(auth): add admin user management dashboard
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 13:45:35 +00:00
copilot-swe-agent[bot] a8d44b189c refactor(multi-user): address code review - module imports, explicit false(), string length
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 11:37:54 +00:00
copilot-swe-agent[bot] 71f437e43a feat(multi-user): add multi-user feature flag, owner_id model field, and user-scoped queries
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 11:26:24 +00:00
copilot-swe-agent[bot] eea99eb01d feat(pdfa): add PDF/A archival conversion using ocrmypdf
- Add ENABLE_PDFA_CONVERSION, PDFA_FORMAT, PDFA_UPLOAD_TO_PROVIDERS config settings
- Add original_pdfa_path and processed_pdfa_path columns to FileRecord model
- Create Alembic migration 011_add_pdfa_paths
- Create app/tasks/convert_to_pdfa.py Celery task using ocrmypdf + Ghostscript
- Integrate PDF/A conversion into finalize_document_storage pipeline
- Add comprehensive unit tests (15 tests)
- Update .env.demo and docs/ConfigurationGuide.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 13:35:13 +00:00
copilot-swe-agent[bot] f7cf7e2a4c fix(migrations): resolve multiple Alembic heads causing CI test crash
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 09:50:59 +00:00
Christian Krakau-Louis 1e1ba1d8d7 Merge branch 'main' into copilot/add-document-similarity-detection 2026-03-01 22:06:55 +01:00
copilot-swe-agent[bot] e60914127c feat(webhooks): add webhook support for external integrations
Add WebhookConfig model, CRUD API endpoints, HMAC-SHA256 signed delivery,
and Celery-based async dispatch with retry/backoff for document events
(document.uploaded, document.processed, document.failed).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:42:57 +00:00
copilot-swe-agent[bot] 9748103782 feat(similarity): add document similarity detection with embeddings and cosine similarity
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:38:52 +00:00
copilot-swe-agent[bot] 8d81ed6c74 fix: address code review feedback - use specific exceptions, dynamic revision IDs in tests
- Replace bare Exception catches with sa.exc.OperationalError in migration 007
- Use ScriptDirectory.get_heads() for dynamic revision ID assertions in tests
- Make migration count assertion flexible (>= 8 instead of == 8)
- Rewrite pending migrations test to actually test upgrade from revision 006

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 16:57:19 +00:00
copilot-swe-agent[bot] 87d1b9d935 refactor(database): enforce Alembic-only database migrations, deprecate manual schema migrations
- Create alembic.ini and migrations/env.py for full Alembic CLI + programmatic support
- Add Alembic migration 006: detail column on processing_logs
- Add Alembic migration 007: ocr_quality_score column + drop unique filehash index
- Add Alembic migration 008: performance indexes
- Replace _run_schema_migrations() call in init_db() with _run_alembic_upgrade()
- Deprecate _run_schema_migrations() with DeprecationWarning
- Update tests for new Alembic-based approach and deprecation
- Update DatabaseConfiguration.md documentation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 16:47:41 +00:00
copilot-swe-agent[bot] e5a4c6c64a feat(api): add advanced filtering and saved searches
- Add date range (date_from/date_to), storage provider, and tags filters to GET /api/files
- Add SavedSearch model and migration (005_add_saved_searches)
- Add CRUD API endpoints for saved searches at /api/saved-searches
- Update files.html template with new filter controls and saved searches UI
- Update files view to pass new filter parameters to template
- Add comprehensive tests for all new functionality (26 tests)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 11:22:07 +00:00
copilot-swe-agent[bot] ff96340154 fix(config): default meilisearch_url to http://meilisearch:7700 for Docker/K8s service discovery
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-25 13:27:09 +00:00
copilot-swe-agent[bot] b03ebab747 chore: apply ruff formatting and fix whitespace issues
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-13 09:12:11 +00:00
Christian Krakau-Louis 4c11cfd6dd feat(deduplication): implement configurable duplicate file detection
- Add enable_deduplication and show_deduplication_step config options
- Rename hash_file step to check_for_duplicates
- Make deduplication step conditional based on configuration
- Add is_duplicate and duplicate_of_id fields to FileRecord model
- Create database migration for new deduplication fields
- Update process_document task to log deduplication results
- Update step visualization to show/hide step based on config
- Update status calculations to include deduplication step conditionally
- Default: deduplication enabled, step displayed
- Can be configured to hide from UI while still processing
2026-02-12 02:38:00 +01:00
copilot-swe-agent[bot] 73a0222e56 feat(storage): implement immutable original/processed file storage and collision handling
- Add original_file_path and processed_file_path columns to FileRecord model
- Create database migration for new fields
- Implement get_unique_filepath_with_counter() with -0001 suffix format
- Update process_document to save immutable copy to /workdir/original
- Add force_cloud_ocr parameter to process_document for forced OCR
- Update embed_metadata to use new collision handling
- Update metadata JSON to include file path references
- Add /files/{file_id}/reprocess-with-cloud-ocr API endpoint
- Update processed_file_path in database during embedding

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-11 20:10:19 +00:00
copilot-swe-agent[bot] ad74bc852d feat: Add explicit FileProcessingStep table for status tracking
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-11 19:45:09 +00:00
Christian Krakau-Louis 92a387a759 feat: add initial migration setup and configuration files 2025-03-31 10:51:49 +02:00