Commit Graph

22 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] e518bce922 fix: merge main branch and renumber migration 037→040
Resolve all merge conflicts between our automation feature branch and
current main (v0.163.0, 920 commits ahead).

Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
  (classification_rules, qr_auth, sessions, system_reset)
- app/config.py: add main's new settings (dropbox_use_global_credentials,
  factory_reset_on_startup, enable_factory_reset)
- app/models.py: add main's new models (ClassificationRuleModel, UserSession,
  QRLoginChallenge, SharePoint integration type)
- app/utils/settings_service.py: merge automation_hooks_enabled with main's
  new metadata entries
- docs/API.md: merge automation API docs with main's classification rules docs
- docs/ConfigurationGuide.md: add factory reset settings
- tests/conftest.py: import both AutomationHook and new main models

Migration renumbered:
- 037_add_automation_hooks → 040_add_automation_hooks
- down_revision: 039_add_classification_rules (was 036_add_document_translation_fields)
- Chain: 036 → 037 → 038 → 039 → 040 (automation hooks)

For all non-automation files with conflicts, main's version was taken since
our branch did not modify those files (conflicts were from a stale prior merge).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/cb62f012-3b69-4415-835e-3857ce3e9f45
2026-03-20 23:54:04 +00:00
copilot-swe-agent[bot] 204000aabc fix: merge main branch and renumber migration 027→037
Resolve 3 merge conflicts and renumber the automation_hooks migration
to follow main's migration chain (036_add_document_translation_fields).

Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
- app/utils/settings_service.py: add automation_hooks_enabled alongside compliance_enabled
- tests/conftest.py: add AutomationHook alongside AuditLog/ComplianceTemplate imports

Migration renumbered:
- 027_add_automation_hooks → 037_add_automation_hooks
- down_revision: 026_add_scheduled_jobs → 036_add_document_translation_fields

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:40:15 +00:00
copilot-swe-agent[bot] d167be8274 feat(automation): add Zapier and Make.com integration
Add REST hooks subscription endpoints, incoming action endpoints, and
Zapier-compatible flat payload format for automation platform integration.

- AutomationHook model for webhook subscriptions
- POST /api/automation/hooks/subscribe and DELETE /hooks/{id}
- GET /api/automation/triggers/sample/{event} for Zapier field mapping
- POST /api/automation/actions/upload for incoming document uploads
- Celery task with retry for async hook delivery
- Integration with existing webhook dispatch flow
- 30 passing tests covering all new functionality

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:50:12 +00:00
copilot-swe-agent[bot] 5bc1fac79c feat(tasks): add 5 additional scheduled jobs (expire_shared_links, prune_processing_logs, prune_old_notifications, backfill_missing_metadata, sync_search_index)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:25:26 +00:00
copilot-swe-agent[bot] c3bb93c197 feat(api): add personal API tokens and enhance webhook integration UI
- Add ApiToken model with SHA-256 hashed storage and usage tracking
- Create API token CRUD endpoints (POST/GET/DELETE /api/api-tokens/)
- Add Bearer token authentication to require_login decorator
- Exempt Bearer-authenticated requests from CSRF validation
- Add API tokens management page with create/revoke/copy UI
- Enhance webhook integration type with detailed explanation and code snippets
- Add navigation links (desktop + mobile) to API tokens page
- Include 19 tests covering CRUD, auth resolution, and utility functions
- Create migration 024_add_api_tokens

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:42:55 +00:00
copilot-swe-agent[bot] 743627ecc2 feat(models): add UserIntegration model, encrypt IMAP passwords, add integrations API
- Add IntegrationDirection/IntegrationType constants and UserIntegration SQLAlchemy model
  with owner_id, direction, integration_type, name, config (JSON), credentials (encrypted),
  is_active, last_used_at, last_error, created_at, updated_at fields
- Fix UserImapAccount password encryption: encrypt_value() on create/update,
  decrypt_value() in IMAP tasks and test-connection endpoint (backward compatible)
- Add Alembic migration 023_add_user_integrations
- Add app/api/integrations.py: CRUD + credentials endpoint with owner-scoped access
- Register integrations router in app/api/__init__.py
- Add 39 tests in tests/test_api_integrations.py covering model, CRUD, encryption

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:02:38 +00:00
copilot-swe-agent[bot] 43bcfe5653 feat(imap): add per-user IMAP ingestion accounts with quota enforcement
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:01:14 +00:00
copilot-swe-agent[bot] 89e0c2fb50 feat(pipelines): add custom processing pipeline engine
- Add Pipeline and PipelineStep models with user-specific ownership
- Add pipeline_id FK column to FileRecord
- Migration 017_add_pipelines (batch mode for SQLite FK compat)
- Pipeline CRUD API at /api/pipelines with step management endpoints
- Reorder steps PUT endpoint placed before parameterised {step_id} routes
- POST /api/files/{id}/assign-pipeline for per-file pipeline assignment
- Admin-only POST /api/pipelines/admin/system for system-level pipelines
- Management UI at /pipelines (Jinja2 + Alpine.js + Tailwind)
- Pipelines link added to desktop and mobile navigation
- 41 new tests in tests/test_api_pipelines.py (all passing)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 10:32:00 +00:00
copilot-swe-agent[bot] 56f7f2351f feat(auth): add admin user management dashboard
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 13:45:35 +00:00
copilot-swe-agent[bot] e60914127c feat(webhooks): add webhook support for external integrations
Add WebhookConfig model, CRUD API endpoints, HMAC-SHA256 signed delivery,
and Celery-based async dispatch with retry/backoff for document events
(document.uploaded, document.processed, document.failed).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:42:57 +00:00
copilot-swe-agent[bot] e5a4c6c64a feat(api): add advanced filtering and saved searches
- Add date range (date_from/date_to), storage provider, and tags filters to GET /api/files
- Add SavedSearch model and migration (005_add_saved_searches)
- Add CRUD API endpoints for saved searches at /api/saved-searches
- Update files.html template with new filter controls and saved searches UI
- Update files view to pass new filter parameters to template
- Add comprehensive tests for all new functionality (26 tests)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 11:22:07 +00:00
copilot-swe-agent[bot] 43bc58770d refactor: consolidate linting tools into Ruff
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-13 09:10:52 +00:00
copilot-swe-agent[bot] 94daf9b2fe test: implement mock OAuth2 server infrastructure for auth testing
- Add MockOAuth2ServerContainer using testcontainers
- Create conftest_oauth.py with OAuth test fixtures
- Add comprehensive OAuth integration tests
- Support both mock (default) and real (CI secrets) OAuth modes
- Add documentation for OAuth testing setup

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-12 11:09:37 +00:00
anthropic-code-agent[bot] 104757fd4f fix(tests): fix remaining 8 tests - storage reorganization and file detail enhancements
- Fix test_storage_reorganization.py: Use valid PDF content with EOF marker
- Fix test_file_detail_enhancements.py: Add missing sample_pdf_file fixture
- All 12 tests now passing (2 storage + 7 file detail + 3 other)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-12 02:08:50 +00:00
copilot-swe-agent[bot] e658dec83c feat(test): add comprehensive external API integration tests with PDF generation
- Add fpdf2 dependency for dynamic test PDF generation
- Create tests/test_external_integrations.py with end-to-end pipeline tests:
  - OpenAI: key validation, metadata extraction via chat completion
  - Azure Document Intelligence: admin connectivity, full OCR on generated PDF
  - S3: bucket access, upload/download/delete pipeline
  - Dropbox: token refresh, upload/download/delete pipeline
  - OneDrive: token refresh, upload/download/delete pipeline
  - Authentik: OIDC discovery endpoint, credential consistency
  - Full pipeline: Azure OCR → OpenAI metadata extraction
- Update tests/conftest.py to capture original env vars before test overrides
- Add has_real_env() helper and original_env fixture for credential detection
- All external tests use @pytest.mark.requires_external and skipif guards
- Test files are dynamically generated with unique content per run
- Uploaded test files are cleaned up in finally blocks

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-11 15:16:53 +00:00
copilot-swe-agent[bot] fb50c8b375 test: add full-stack integration testing with real infrastructure (PostgreSQL, Redis, Gotenberg, WebDAV, SFTP, MinIO)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-10 15:24:35 +00:00
copilot-swe-agent[bot] f2d70e90cf test: add comprehensive WebDAV tests with real server and Redis integration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-10 15:18:17 +00:00
copilot-swe-agent[bot] a63983a26c style: fix formatting and linting issues in conftest.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-09 15:38:31 +00:00
copilot-swe-agent[bot] 98cf9e0e0b refactor: consolidate get_db into single module
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-09 15:37:23 +00:00
copilot-swe-agent[bot] 388bde2ff0 Fix SQLAlchemy model registration and database session injection in tests
- Import all models (DocumentMetadata, FileRecord, ProcessingLog) in conftest.py to register them with SQLAlchemy Base
- Override all three get_db functions used across the app (app.database, app.api.common, app.views.base) to ensure tests use the test database
- Fixes 21 of 24 failing tests (from "no such table" errors to passing)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 17:54:05 +00:00
copilot-swe-agent[bot] 8eea420bc5 Add throttling to /processall endpoint to prevent overwhelming downstream APIs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 14:59:19 +00:00
copilot-swe-agent[bot] f0f48b39a9 Add security fixes, testing infrastructure, and CI/CD improvements
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-06 21:55:19 +00:00