- Move `import os` to top-level in app/views/backup.py
- Fix docstring in BackupRecord model to remove non-existent 'location' field
- Replace browser confirm() dialogs with accessible modal dialog (role=dialog, aria-modal, aria-labelledby)
- Add csrfToken() helper that validates token presence instead of silently falling back to empty string
- Fix aria-live region to remain in DOM (screen-reader friendly) rather than using x-show
- Add Backup & Restore section to docs/ConfigurationGuide.md with retention table
- Add backup env vars to .env.demo with comments
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Both sets of UserProfile columns are retained:
- is_complimentary (from main, migration 019_add_is_complimentary)
- subscription_change_pending_tier / subscription_change_pending_date
(our branch, renamed to migration 020_add_subscription_change_pending
with down_revision updated to chain after 019_add_is_complimentary)
- Add MkDocs Material docs build stage to Dockerfile and Dockerfile.local
- Mount pre-built docs as static files at /help/ in FastAPI (app/main.py)
- Add app/views/help.py with /help → /help/ permanent redirect route
- Register help router in app/views/__init__.py
- Add Help nav link to base.html (public + app nav, desktop + mobile)
- Create how-to guides: HP printer, ScanSnap, watched folder, email ingestion, mobile scanning
- Update mkdocs.yml with How-To Guides section and Material theme palette
- Add optional docs service (squidfunk/mkdocs-material) to docker-compose.yaml with docs profile
- Add mkdocs-material to requirements-dev.txt
- Add /docs_build to .gitignore
- Add tests for help view (8 tests, 100% coverage on help.py)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add subscription_change_pending_tier and subscription_change_pending_date fields to UserProfile
- Create migration 019_add_subscription_change_pending
- Add apply_pending_subscription_changes(), request_subscription_change(), cancel_pending_subscription_change() utilities
- Add POST /api/subscriptions/change and DELETE /api/subscriptions/change endpoints
- Update GET /api/subscriptions/my to apply pending changes and return pending change info
- Update subscription view to apply pending changes and pass period_start + pending info
- Update subscription.html: per-tier action buttons (upgrade/downgrade/cancel), pending-change banner, period start date
- Add Celery daily task apply_pending_subscription_changes_all at 00:05 UTC
- Add 19 new tests covering all new utility functions and API endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Remove SMTP hard-requirement from /api/auth/signup: when SMTP is not
configured accounts are activated immediately (no email verification).
When SMTP is configured the existing email-verification flow is kept.
- Inject allow_signup into global template context via app/views/base.py
- Add data-allow-signup attribute to base.html body tag
- Update common.js _renderLoggedOutAuth to show Sign Up (→ /signup) when
signup is enabled, otherwise Get Started (→ /pricing)
- Add admin API endpoints before the /{user_id:path} catch-all:
GET /api/admin/users/local – list all local accounts
POST /api/admin/users/local – admin-create local account (active immediately)
DELETE /api/admin/users/local/{id} – delete local account + profile
- Add LocalUserCreate / LocalUserResponse Pydantic schemas
- Update admin_users.html with Local User Accounts section and modals
- Update .env.demo to document ALLOW_LOCAL_SIGNUP
- Update docs/BillingSetup.md: SMTP is optional, document both flows
- Update tests: test_signup_smtp_not_configured now asserts 201 + immediate
activation; add 7 new integration tests for admin local user endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Inject is_logged_in, multi_user_enabled, auth_enabled into all templates
via app/views/base.py _inject_global_context() helper
- Multi-user + logged-out: show only Pricing, About, Log In, Get Started
- Logged-in or single-user: full app nav (Dashboard, Upload, Files,
Search, Pipelines, Admin dropdown, Status)
- Upload link is visually accented (blue) as the primary action
- Account dropdown (avatar, name, email, subscription, sign-out) for
logged-in users in desktop and mobile
- Admin dropdown Similarity icon changed to purple to differentiate
from Queue Monitor
- data-multi-user attribute on <body> so JS reads the mode at runtime
- 5 new unit tests for is_logged_in/multi_user_enabled injection
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The file detail page (/files/{id}/detail) and file view page (/files/{id})
previously showed no information about which processing pipeline was used.
Changes:
- _STEP_TYPE_TO_STAGES mapping: pipeline step_type → Celery log stage keys
(with maintenance comment requiring updates when new step types are added)
- _ALWAYS_SHOW_STAGES: stages always visible regardless of pipeline
- _resolve_pipeline(db, file_record): resolves the pipeline for a file —
uses explicit pipeline_id when set, falls back to active system default
- _compute_processing_flow: new pipeline_steps parameter; when provided,
filters flow graph to only show stages for the pipeline's enabled steps
(+ always-show stages + any stage that actually ran). Also adds
convert_to_pdf to the flow stage catalogue.
- file_detail_page: passes pipeline_info + pipeline-filtered flow_data
- file_view_page: passes pipeline_info
Templates:
- file_detail.html: 'Processing Pipeline' detail row with name link and
colour-coded badge (System Default / System / Custom)
- file_view.html: 'Pipeline' info row in sidebar with (default)/(custom) tag
Tests:
- TestPipelineInfoInViews with 14 tests covering _resolve_pipeline,
_compute_processing_flow filtering, completeness assertion for
_STEP_TYPE_TO_STAGES, and HTTP-level view tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- 5-step wizard: Welcome → Profile → Plan → Storage → All Set!
- New migration 017: onboarding_completed, contact_email, preferred_destination fields
- REST API at /api/onboarding/{status,profile,plan,storage,complete}
- GET /onboarding view with configured-destinations helper
- OAuth callback redirects first-time users to onboarding
- 16 unit tests for all endpoints; 65 total tests pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Pipeline and PipelineStep models with user-specific ownership
- Add pipeline_id FK column to FileRecord
- Migration 017_add_pipelines (batch mode for SQLite FK compat)
- Pipeline CRUD API at /api/pipelines with step management endpoints
- Reorder steps PUT endpoint placed before parameterised {step_id} routes
- POST /api/files/{id}/assign-pipeline for per-file pipeline assignment
- Admin-only POST /api/pipelines/admin/system for system-level pipelines
- Management UI at /pipelines (Jinja2 + Alpine.js + Tailwind)
- Pipelines link added to desktop and mobile navigation
- 41 new tests in tests/test_api_pipelines.py (all passing)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add SubscriptionPlan model and subscription_plans table (migration 015)
- Add billing cycle/period/allow_overage fields to UserProfile (migration 016)
- Add subscription_overage_percent config field (replaces overage_factor)
- Rewrite check_upload_allowed: use overage_percent, yearly carry-over, no daily cap
- Add seed_default_plans(), _plan_to_dict(), get_year_file_count(), _months_elapsed()
- Update get_tier/get_all_tiers to be DB-first with TIER_DEFAULTS fallback
- Add TIER_DEFAULTS alias (TIERS kept for backward compat)
- New /api/plans/ CRUD endpoints (admin-only except list/get)
- New /admin/plans Plan Designer page with Alpine.js UI
- Add Plan Designer link to admin navigation in base.html
- Remove 'Files per day' row from pricing comparison table
- Add billing cycle + period start to admin users edit modal
- Seed default plans on startup in lifespan handler
- Rewrite docs/SubscriptionTiers.md with full plan/overage/API docs
- Fix all tests in test_subscription.py (remove daily cap tests, add overage/carry-over tests)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add Free / Starter / Professional / Business tiers with lifetime, daily, and monthly
file limits (app/utils/subscription.py)
- Add subscription_tier column to UserProfile model + migration 014
- Enforce quotas at upload time (HTTP 402 on violation) in /api/ui-upload
- New REST API: GET /api/subscriptions/tiers, /my, /platform (admin)
- New pages: /pricing (marketing, public) and /subscription (per-user status)
- Enhanced dashboard: SaaS stats (files today/month, OCR count, active users)
in multi-user mode; original single-user layout preserved
- Admin users page: show Plan badge, allow tier editing via dropdown
- Navigation: add Pricing link + subscription icon in user header
- Tests: 23 unit tests for subscription tier logic
- Docs: docs/SubscriptionTiers.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add a guided database configuration wizard and a data migration tool that
allows users to:
- Build database connection strings through a step-by-step UI
- Test database connections before applying
- Preview and execute data migrations from SQLite to PostgreSQL/MySQL
- Copy to clipboard for easy .env file updates
New files:
- app/utils/db_wizard.py — connection string builder, parser, and tester
- app/utils/db_migrate.py — table-by-table data migration utility
- app/api/database.py — REST API endpoints for wizard operations
- app/views/db_wizard.py — view route for the wizard page
- frontend/templates/db_wizard.html — multi-tab wizard UI
- tests/test_db_wizard.py — unit tests for db_wizard utilities
- tests/test_db_migrate.py — unit tests for db_migrate utilities
- tests/test_db_wizard_api.py — integration tests for API and views
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add GET /api/similarity/pairs endpoint for corpus-wide pair discovery
- Add /similarity view route and similarity_dashboard.html template
- Add Similarity link to desktop and mobile nav menus
- Register compute_embedding as a tracked FileProcessingStep
- Update compute_embedding task with update_step_status calls
- Add compute_embedding to flow visualization in _compute_processing_flow
- Add backfill_missing_embeddings periodic beat task (every 5 min)
- Return clear message when embedding not yet computed in similar docs API
- Fix all tests to use pre-computed embeddings (no lazy API calls)
- Add tests for similarity pairs, backfill task, and embedding-not-computed
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add near_duplicate_threshold config setting (default 0.85)
- New GET /api/duplicates endpoint listing all exact-duplicate groups
- New GET /api/files/{id}/duplicates endpoint returning exact + near-duplicates
- POST /api/ui-upload now returns immediate exact-duplicate warning (respects ENABLE_DEDUPLICATION)
- New /duplicates management UI with Exact Duplicates tab and Near-Duplicate Finder tab
- Add Duplicates link in admin nav menu (desktop + mobile)
- Document new config options in ConfigurationGuide.md and .env.demo
- 20 new tests covering all acceptance criteria
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add release_names.json mapping version ranges to codenames
- Add release_name property to Settings in app/config.py
- Update build metadata script to include codename in RUNTIME_INFO
- Display release codename in status dashboard and page footer
- Inject release_name globally via template response wrapper
- Update ROADMAP.md with codenames for all milestone releases
- Add docs/ReleaseNaming.md with naming guide and best practices
- Add comprehensive tests for release name resolution
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Change default per_page from 50 to 25
- Rename total_items → total, total_pages → pages in pagination response
- Add next/previous URL fields to pagination response
- Update view and template to use new field names
- Update tests and API docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add date range (date_from/date_to), storage provider, and tags filters to GET /api/files
- Add SavedSearch model and migration (005_add_saved_searches)
- Add CRUD API endpoints for saved searches at /api/saved-searches
- Update files.html template with new filter controls and saved searches UI
- Update files view to pass new filter parameters to template
- Add comprehensive tests for all new functionality (26 tests)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Revert search.html and files.html links back to /files/{id}
- Add GET /files/{file_id} route (file_view_page) with workdir path-
containment guards (os.path.commonpath) to prevent traversal
- Create file_view.html: document-centric page showing AI metadata,
inline PDF preview, extracted OCR text, download actions, file info,
status pill, and link to /files/{id}/detail for process pipeline view
- Existing /files/{id}/detail route is unchanged
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The debounceSearch() function in files.html called clearFullTextSearch()
when the query was shorter than 2 characters. Since clearFullTextSearch()
sets input.value = '', every single keystroke was immediately erased —
users could paste text but not type.
Fix: debounceSearch now only hides the results panel for short queries
without touching the input value.
Also adds a dedicated /search page with Google-style results showing
content previews (document title, filename, type badges, tag badges,
sender, and OCR text snippets with highlighted matches).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Update _compute_processing_flow to recognize process_with_ocr as the OCR
stage and remap legacy process_with_azure_document_intelligence log entries
for backward compatibility
- Normalize legacy OCR step name in _compute_step_summary log fallback
- Add process_with_ocr to REAL_MAIN_STEPS/REAL_STEPS in step_manager,
file_status, and file_queries (keeping legacy name for old DB entries)
- Update retry logic in api/files.py to retry failed OCR via process_with_ocr
(handles both step names as aliases)
- Fix process_document.py to log process_with_ocr as skipped (not azure step)
for the local text extraction path
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
A) Per-option Save Button
- Add per-setting Save button in settings.html (visible only when value changed)
- Button calls POST /api/settings/{key} directly; existing bulk Save retained
- Add Audit Log link in settings page header
B) Immediate Worker Sync
- New app/utils/settings_sync.py with notify_settings_updated() (Redis version key)
and register_settings_reload_signal() (Celery task_prerun handler)
- Register signal in celery_worker.py at startup
- All API write paths call notify_settings_updated() after successful saves
C) Audit Log
- Add SettingsAuditLog model (key, old_value, new_value, changed_by, changed_at, action)
- save_setting_to_db / delete_setting_from_db accept changed_by and write audit entries
- New get_audit_log() service function (masks sensitive values)
- New GET /api/settings/audit-log endpoint (admin-only)
- New GET /admin/settings/audit-log view + audit_log.html template
- Visible to all admins (per clarified requirement)
D) Config Rollback / History
- New get_setting_history() and rollback_setting() service functions
- New GET /api/settings/{key}/history endpoint
- New POST /api/settings/{key}/rollback/{history_id} endpoint
- Rollback buttons in audit_log.html with confirmation dialog
- Tests: 25 new tests covering audit log, rollback, worker sync helpers, and API endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix ruff format: add blank line before nested function in app/views/filemanager.py
- Fix ruff format: use double-quote escaping in tests/test_config.py
- Remove deploy job (Portainer webhook) from ci.yml
- Add update-k8s-manifest job: after main-branch build, updates
apps/docuelevate/preprod/docuelevate-stack.yaml in christianlouis/k8s-cluster-state
with the new GHCR image tag (ghcr.io/christianlouis/docuelevate:main-<short-sha>)
using mikefarah/yq@v4.44.6 and GH_PAT secret for cross-repo write access
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]