Commit Graph

134 Commits

Author SHA1 Message Date
Christian Krakau-Louis a27a0d6f01 Merge pull request #502 from christianlouis/copilot/update-plan-descriptions
fix(ui): rename "Business" → "Power" and rewrite plan copy for per-user pricing
2026-03-07 21:44:40 +01:00
copilot-swe-agent[bot] 9d11d741f4 fix(ui): update plan descriptions to reflect per-user pricing
- Rewrite taglines and feature lists for all four tiers with
  concrete, use-case-driven copy (freelancers, knowledge workers,
  power users — no team or enterprise framing)
- Rename 'Business' display name to 'Power' (plan_id stays
  'business' for backwards DB compatibility)
- Replace 'enterprise' language in pricing page hero with
  'per person, per month' copy
- Swap fa-building icon for fa-bolt on the Power tier
- Fix support level for Power tier to 'Priority' (was 'Dedicated')
- Update docs/SubscriptionTiers.md with new names, table, and
  intended-use-case section
- Add test_business_tier_display_name_is_power assertion

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 20:12:28 +00:00
copilot-swe-agent[bot] aa6e2fe001 feat(auth): enable local user signup without SMTP, add admin user creation
- Remove SMTP hard-requirement from /api/auth/signup: when SMTP is not
  configured accounts are activated immediately (no email verification).
  When SMTP is configured the existing email-verification flow is kept.
- Inject allow_signup into global template context via app/views/base.py
- Add data-allow-signup attribute to base.html body tag
- Update common.js _renderLoggedOutAuth to show Sign Up (→ /signup) when
  signup is enabled, otherwise Get Started (→ /pricing)
- Add admin API endpoints before the /{user_id:path} catch-all:
    GET  /api/admin/users/local       – list all local accounts
    POST /api/admin/users/local       – admin-create local account (active immediately)
    DELETE /api/admin/users/local/{id} – delete local account + profile
- Add LocalUserCreate / LocalUserResponse Pydantic schemas
- Update admin_users.html with Local User Accounts section and modals
- Update .env.demo to document ALLOW_LOCAL_SIGNUP
- Update docs/BillingSetup.md: SMTP is optional, document both flows
- Update tests: test_signup_smtp_not_configured now asserts 201 + immediate
  activation; add 7 new integration tests for admin local user endpoints

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 19:59:10 +00:00
copilot-swe-agent[bot] 02ac55ab1a Merge branch 'main' of https://github.com/christianlouis/DocuElevate into copilot/fix-pricing-page-issues 2026-03-07 14:47:29 +00:00
copilot-swe-agent[bot] 52e3852129 feat(auth): add local user signup, email verification, and Stripe billing
- Add LocalUser model with bcrypt password hashing, email verification
  tokens, and password reset tokens
- Add ALLOW_LOCAL_SIGNUP config flag (requires SMTP to be configured)
- Add Stripe billing config fields (STRIPE_SECRET_KEY, etc.)
- Add stripe_customer_id to UserProfile and stripe_price_id_monthly/
  stripe_price_id_yearly to SubscriptionPlan
- Create migration 018_add_local_users_and_billing
- Add app/utils/local_auth.py: hash_password, verify_password,
  generate_token, is_token_expired, send_verification_email,
  send_password_reset_email, build_session_user
- Add app/api/local_auth.py: signup, email verification, password reset
  endpoints plus signup/verify-email-sent/reset-password page routes
- Add app/api/billing.py: Stripe Checkout, Customer Portal, and webhook
  endpoints; syncs subscription tier from webhook events
- Update auth() to check LocalUser table before admin credentials fallback
- Update login() to pass allow_signup context variable to template
- Add signup.html, verify_email_sent.html, password_reset_form.html,
  billing_success.html templates (Alpine.js, Tailwind, WCAG 2.1 AA)
- Update login.html to show 'Create account' link when signup enabled
- Update pricing.html CTA buttons to use Stripe Checkout for paid tiers
- Add docs/BillingSetup.md with setup guide, webhook config, compliance
- Add tests/test_local_auth.py (42 tests) and tests/test_billing.py
  (31 tests); all 106 tests in the modified test suite pass
- Add stripe>=7.0.0,<15.0.0 to requirements.txt

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-07 13:15:15 +00:00
copilot-swe-agent[bot] 1203a4b75f refactor(pipelines): address code review - shared get_current_user_id, aria-live, deduplicate user ID logic
- Extract _get_user_id into shared auth.get_current_user_id() used by both
  pipelines API and the assign-pipeline endpoint in files API
- Fix aria-live attribute: use two separate static containers (polite/assertive)
  instead of dynamic Alpine.js binding for correct screen reader announcements
- Fix migration comment to accurately describe batch-mode FK creation
- Remove redundant tags parameter from reorder endpoint decorator
- Rename _make_file test helper to _make_test_file_record for clarity
- Update docs/UserGuide.md and docs/API.md with full Pipelines reference

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-07 10:39:19 +00:00
Christian Krakau-Louis 10bb533f64 Merge pull request #490 from christianlouis/copilot/add-subscription-page
fix: add subscription_overage_percent to SETTING_METADATA and document in ConfigurationGuide/.env.demo
2026-03-07 10:59:01 +01:00
copilot-swe-agent[bot] 9853a27d82 fix: add subscription_overage_percent to SETTING_METADATA and docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 21:29:44 +00:00
copilot-swe-agent[bot] ea7fffa3a1 feat(subscriptions): database-backed plan designer with admin CRUD and overage buffer
- Add SubscriptionPlan model and subscription_plans table (migration 015)
- Add billing cycle/period/allow_overage fields to UserProfile (migration 016)
- Add subscription_overage_percent config field (replaces overage_factor)
- Rewrite check_upload_allowed: use overage_percent, yearly carry-over, no daily cap
- Add seed_default_plans(), _plan_to_dict(), get_year_file_count(), _months_elapsed()
- Update get_tier/get_all_tiers to be DB-first with TIER_DEFAULTS fallback
- Add TIER_DEFAULTS alias (TIERS kept for backward compat)
- New /api/plans/ CRUD endpoints (admin-only except list/get)
- New /admin/plans Plan Designer page with Alpine.js UI
- Add Plan Designer link to admin navigation in base.html
- Remove 'Files per day' row from pricing comparison table
- Add billing cycle + period start to admin users edit modal
- Seed default plans on startup in lifespan handler
- Rewrite docs/SubscriptionTiers.md with full plan/overage/API docs
- Fix all tests in test_subscription.py (remove daily cap tests, add overage/carry-over tests)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-06 18:48:27 +00:00
Christian Krakau-Louis 59f1c46b94 Merge pull request #489 from christianlouis/copilot/combine-quick-and-integration-tests
ci: consolidate quick and integration tests into a single test run
2026-03-06 19:24:27 +01:00
copilot-swe-agent[bot] 179f6125e8 feat(subscriptions): add SaaS subscription tiers, pricing page, and enforced upload quotas
- Add Free / Starter / Professional / Business tiers with lifetime, daily, and monthly
  file limits (app/utils/subscription.py)
- Add subscription_tier column to UserProfile model + migration 014
- Enforce quotas at upload time (HTTP 402 on violation) in /api/ui-upload
- New REST API: GET /api/subscriptions/tiers, /my, /platform (admin)
- New pages: /pricing (marketing, public) and /subscription (per-user status)
- Enhanced dashboard: SaaS stats (files today/month, OCR count, active users)
  in multi-user mode; original single-user layout preserved
- Admin users page: show Plan badge, allow tier editing via dropdown
- Navigation: add Pricing link + subscription icon in user header
- Tests: 23 unit tests for subscription tier logic
- Docs: docs/SubscriptionTiers.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 15:53:12 +00:00
copilot-swe-agent[bot] 9670e84862 ci: merge quick and integration tests into a single test step
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 15:32:43 +00:00
copilot-swe-agent[bot] 56f7f2351f feat(auth): add admin user management dashboard
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 13:45:35 +00:00
Christian Krakau-Louis 26b4d04d76 Merge pull request #482 from christianlouis/copilot/add-database-configuration-wizard
feat(database): integrate wizard into settings page, improve accessibility and test coverage
2026-03-06 11:28:44 +01:00
copilot-swe-agent[bot] 7dae15fd29 docs(database): update configuration guides with wizard cross-references, clean up review feedback
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-06 10:13:40 +00:00
copilot-swe-agent[bot] cb3bf809cc docs(database): add wizard and migration tool documentation
Update DatabaseConfiguration.md with sections for the new Database
Configuration Wizard and Database Migration Tool. Also update API.md
with the new /api/database/ endpoint documentation.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 22:19:39 +00:00
copilot-swe-agent[bot] c8bc4afc93 fix: address code review — use modern type hints and Callable annotation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 22:03:30 +00:00
copilot-swe-agent[bot] 3601e2ca5c feat(ui): add user autocomplete widget for default_owner_id, user search API, and documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 21:40:41 +00:00
copilot-swe-agent[bot] d7b7f1478f test(multi-user): add comprehensive tests for multi-user isolation and feature flag
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-05 11:34:41 +00:00
copilot-swe-agent[bot] 3080e37ddf docs(settings): update SettingsManagement.md with new categories and setting types
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 16:45:15 +00:00
Christian Krakau-Louis 8fee5c8d0b Merge branch 'main' into copilot/add-pdfa-export-option 2026-03-02 16:26:31 +01:00
copilot-swe-agent[bot] a03b3af933 feat(pdfa): add FreeTSA timestamping, per-provider folder overrides, individual upload toggles
- Add RFC 3161 timestamping via FreeTSA (PDFA_TIMESTAMP_ENABLED, PDFA_TIMESTAMP_URL)
- Replace PDFA_UPLOAD_TO_PROVIDERS with individual PDFA_UPLOAD_ORIGINAL and PDFA_UPLOAD_PROCESSED
- Add PDFA_UPLOAD_FOLDER setting for per-provider subfolder configuration
- Add GOOGLE_DRIVE_PDFA_FOLDER_ID for Google Drive-specific folder override
- Add folder_override parameter to all 8 folder-using upload tasks
- Add folder_overrides dict parameter to send_to_all_destinations
- Add _compute_pdfa_folder_overrides() and _timestamp_file() helpers
- Expand tests to 26 (timestamping, folder overrides, individual toggles)
- Update docs/ConfigurationGuide.md and .env.demo

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 13:55:34 +00:00
copilot-swe-agent[bot] eea99eb01d feat(pdfa): add PDF/A archival conversion using ocrmypdf
- Add ENABLE_PDFA_CONVERSION, PDFA_FORMAT, PDFA_UPLOAD_TO_PROVIDERS config settings
- Add original_pdfa_path and processed_pdfa_path columns to FileRecord model
- Create Alembic migration 011_add_pdfa_paths
- Create app/tasks/convert_to_pdfa.py Celery task using ocrmypdf + Ghostscript
- Integrate PDF/A conversion into finalize_document_storage pipeline
- Add comprehensive unit tests (15 tests)
- Update .env.demo and docs/ConfigurationGuide.md

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 13:35:13 +00:00
copilot-swe-agent[bot] 8e955f3c81 fix(similarity): truncate text to fit embedding model context window, fix step tracking
- Add EMBEDDING_MAX_TOKENS config (default 8000) for safe text truncation
- Use conservative 3 chars/token estimate (was 4) to prevent ContextWindowExceededError
- Add compute_embedding to REAL_MAIN_STEPS in both get_file_overall_status and get_step_summary
- Fix test_near_duplicates_returned to use pre-computed embeddings
- Update .env.demo and docs with EMBEDDING_MAX_TOKENS setting

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 13:27:34 +00:00
copilot-swe-agent[bot] ebea83a750 feat(duplicates): add duplicate document detection and management
- Add near_duplicate_threshold config setting (default 0.85)
- New GET /api/duplicates endpoint listing all exact-duplicate groups
- New GET /api/files/{id}/duplicates endpoint returning exact + near-duplicates
- POST /api/ui-upload now returns immediate exact-duplicate warning (respects ENABLE_DEDUPLICATION)
- New /duplicates management UI with Exact Duplicates tab and Near-Duplicate Finder tab
- Add Duplicates link in admin nav menu (desktop + mobile)
- Document new config options in ConfigurationGuide.md and .env.demo
- 20 new tests covering all acceptance criteria

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 10:21:57 +00:00
Christian Krakau-Louis 1e1ba1d8d7 Merge branch 'main' into copilot/add-document-similarity-detection 2026-03-01 22:06:55 +01:00
copilot-swe-agent[bot] a85992ee79 docs(webhooks): fix spelling - Behaviour to Behavior
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:47:51 +00:00
copilot-swe-agent[bot] b7c78177e9 docs(similarity): add API documentation and fix template accessibility
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:46:13 +00:00
copilot-swe-agent[bot] a953b726fc docs(webhooks): add webhook API and configuration documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:45:20 +00:00
Christian Krakau-Louis ecca0ecdfa Merge pull request #461 from christianlouis/copilot/enhance-release-versioning 2026-03-01 20:31:17 +01:00
Christian Krakau-Louis 68ea6aabc7 Merge pull request #448 from christianlouis/copilot/add-retry-logic-backoff 2026-03-01 20:30:45 +01:00
copilot-swe-agent[bot] 85da309740 feat(api): add GET /api/diagnostic/health endpoint for monitoring
- Add health check endpoint at GET /api/diagnostic/health
- Auth-protected via @require_login (no-op when AUTH_ENABLED=False)
- Checks database (SELECT 1) and Redis (ping) with 2s timeouts
- Returns healthy/degraded/unhealthy with per-check detail
- Returns HTTP 503 when database is down, 200 otherwise
- 7 new unit tests covering all status scenarios
- Update docs/API.md with Grafana/monitoring integration notes
- Fixes test_cors_headers_absent_when_disabled CI timeout

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 18:36:38 +00:00
copilot-swe-agent[bot] d18c10996a feat(release): add named release anchors with codenames and roadmap integration
- Add release_names.json mapping version ranges to codenames
- Add release_name property to Settings in app/config.py
- Update build metadata script to include codename in RUNTIME_INFO
- Display release codename in status dashboard and page footer
- Inject release_name globally via template response wrapper
- Update ROADMAP.md with codenames for all milestone releases
- Add docs/ReleaseNaming.md with naming guide and best practices
- Add comprehensive tests for release name resolution

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 18:32:46 +00:00
copilot-swe-agent[bot] 5ff7b72a80 feat(tasks): add retry logic with exponential backoff and jitter
- Rewrite app/tasks/retry_config.py with compute_countdown() function
  implementing per-retry delays with ±20% jitter (default: 60s, 300s, 900s)
- Add BaseTaskWithRetry.retry() override to inject proper countdown
- Add OcrTaskWithRetry (120s, 600s, 1800s) for OCR/AI tasks
- Add UploadTaskWithRetry for cloud-storage upload tasks
- Add config settings: TASK_RETRY_MAX_RETRIES, TASK_RETRY_DELAYS, TASK_RETRY_JITTER
- Update process_with_ocr and process_with_azure tasks to use OcrTaskWithRetry
- Update all 11 upload tasks to use UploadTaskWithRetry
- Add 38 unit tests in tests/test_retry_config.py
- Update docs/ConfigurationGuide.md and .env.demo

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 17:37:41 +00:00
copilot-swe-agent[bot] af34ce88df feat(api): implement proper pagination for file list API
- Change default per_page from 50 to 25
- Rename total_items → total, total_pages → pages in pagination response
- Add next/previous URL fields to pagination response
- Update view and template to use new field names
- Update tests and API docs

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 17:31:56 +00:00
Christian Krakau-Louis 0134ed37d5 Merge pull request #446 from christianlouis/copilot/optimize-test-timeouts
fix(ci): split test pipeline into quick/integration stages, fix hanging test
2026-03-01 18:23:06 +01:00
Christian Krakau-Louis 8ac4179b8b Merge pull request #445 from christianlouis/copilot/enforce-database-upgrades-migration
refactor(database): enforce Alembic-only schema migrations, deprecate manual migration code
2026-03-01 18:22:48 +01:00
copilot-swe-agent[bot] 87d1b9d935 refactor(database): enforce Alembic-only database migrations, deprecate manual schema migrations
- Create alembic.ini and migrations/env.py for full Alembic CLI + programmatic support
- Add Alembic migration 006: detail column on processing_logs
- Add Alembic migration 007: ocr_quality_score column + drop unique filehash index
- Add Alembic migration 008: performance indexes
- Replace _run_schema_migrations() call in init_db() with _run_alembic_upgrade()
- Deprecate _run_schema_migrations() with DeprecationWarning
- Update tests for new Alembic-based approach and deprecation
- Update DatabaseConfiguration.md documentation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 16:47:41 +00:00
copilot-swe-agent[bot] 6a21adcdd4 fix(ci): increase quick test job timeout to 15 minutes for CI runner headroom
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 16:46:57 +00:00
copilot-swe-agent[bot] c100250b23 docs: update CI workflow and integration test documentation for two-stage pipeline
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 16:45:44 +00:00
Christian Krakau-Louis 91bd4ad837 Merge pull request #444 from christianlouis/copilot/optimize-database-queries
fix(db): check column existence before creating indexes; fix mypy error in cache.py
2026-03-01 17:27:03 +01:00
copilot-swe-agent[bot] ce87b53331 style: fix spelling (British to American English)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 14:41:29 +00:00
copilot-swe-agent[bot] d5a95e9270 docs: add Performance & Caching section to ConfigurationGuide
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 14:40:19 +00:00
copilot-swe-agent[bot] f8c5dd539d feat(imap): add IMAP_READONLY_MODE feature flag to safeguard shared mailboxes
When enabled, IMAP processing will fetch and process attachments but
will NOT modify the mailbox state (no starring, labeling, deleting,
or flag changes). This allows preprod instances to safely share a
Gmail inbox with production without interfering with production
email processing.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 14:28:51 +00:00
Christian Krakau-Louis 96b14ec74f Merge pull request #440 from christianlouis/copilot/update-search-functionality
feat(search): add content-finding filters, saved searches, and text quality to Search view
2026-03-01 15:19:57 +01:00
copilot-swe-agent[bot] de09dba285 docs: update API and User Guide with search filters and saved searches
- Document new /api/search endpoint with all filter parameters
- Update saved searches docs with new allowed filter keys
- Update User Guide with search view filters and saved searches usage

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:55:55 +00:00
copilot-swe-agent[bot] 83c3405c98 feat: persist ocr_quality_score and use it for numeric filtering
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:46:43 +00:00
copilot-swe-agent[bot] 705b970522 feat: add bulk download, cloud OCR, and basic OCR quality filter
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:38:42 +00:00
Christian Krakau-Louis 05c35a6775 Merge pull request #437 from christianlouis/copilot/add-file-download-endpoint
fix(api): change download endpoint default version from `original` to `processed`
2026-03-01 13:30:30 +01:00
copilot-swe-agent[bot] cc9fd1a7fe docs(api): document advanced filtering and saved searches endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 11:29:47 +00:00