- Use astimezone() instead of replace() for timezone conversion in is_token_expired
- Log cleanup exceptions with logger.exception() in signup
- Add security warning when STRIPE_WEBHOOK_SECRET is not configured
- Increase Stripe price ID column length from 64 to 128 characters
- Replace alert() with aria-live assertive region in pricing.html
- Convert auth() login tests to use pytest.mark.asyncio and await
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The file detail page (/files/{id}/detail) and file view page (/files/{id})
previously showed no information about which processing pipeline was used.
Changes:
- _STEP_TYPE_TO_STAGES mapping: pipeline step_type → Celery log stage keys
(with maintenance comment requiring updates when new step types are added)
- _ALWAYS_SHOW_STAGES: stages always visible regardless of pipeline
- _resolve_pipeline(db, file_record): resolves the pipeline for a file —
uses explicit pipeline_id when set, falls back to active system default
- _compute_processing_flow: new pipeline_steps parameter; when provided,
filters flow graph to only show stages for the pipeline's enabled steps
(+ always-show stages + any stage that actually ran). Also adds
convert_to_pdf to the flow stage catalogue.
- file_detail_page: passes pipeline_info + pipeline-filtered flow_data
- file_view_page: passes pipeline_info
Templates:
- file_detail.html: 'Processing Pipeline' detail row with name link and
colour-coded badge (System Default / System / Custom)
- file_view.html: 'Pipeline' info row in sidebar with (default)/(custom) tag
Tests:
- TestPipelineInfoInViews with 14 tests covering _resolve_pipeline,
_compute_processing_flow filtering, completeness assertion for
_STEP_TYPE_TO_STAGES, and HTTP-level view tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- 5-step wizard: Welcome → Profile → Plan → Storage → All Set!
- New migration 017: onboarding_completed, contact_email, preferred_destination fields
- REST API at /api/onboarding/{status,profile,plan,storage,complete}
- GET /onboarding view with configured-destinations helper
- OAuth callback redirects first-time users to onboarding
- 16 unit tests for all endpoints; 65 total tests pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix monthly/annual price toggle by moving x-data scope to outer div
- Auto-create UserProfile in DB on first Authentik OAuth login
- Update and expand tests for oauth_callback and _ensure_user_profile
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Extract _get_user_id into shared auth.get_current_user_id() used by both
pipelines API and the assign-pipeline endpoint in files API
- Fix aria-live attribute: use two separate static containers (polite/assertive)
instead of dynamic Alpine.js binding for correct screen reader announcements
- Fix migration comment to accurately describe batch-mode FK creation
- Remove redundant tags parameter from reorder endpoint decorator
- Rename _make_file test helper to _make_test_file_record for clarity
- Update docs/UserGuide.md and docs/API.md with full Pipelines reference
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Pipeline and PipelineStep models with user-specific ownership
- Add pipeline_id FK column to FileRecord
- Migration 017_add_pipelines (batch mode for SQLite FK compat)
- Pipeline CRUD API at /api/pipelines with step management endpoints
- Reorder steps PUT endpoint placed before parameterised {step_id} routes
- POST /api/files/{id}/assign-pipeline for per-file pipeline assignment
- Admin-only POST /api/pipelines/admin/system for system-level pipelines
- Management UI at /pipelines (Jinja2 + Alpine.js + Tailwind)
- Pipelines link added to desktop and mobile navigation
- 41 new tests in tests/test_api_pipelines.py (all passing)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- app/utils/subscription.py: add Any type annotation to _scalar_count()
query parameter (mypy no-untyped-def error at line 316)
- frontend/templates/admin_plans.html: remove empty <div></div> at line 344
(djlint H020 empty tag pair error)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add SubscriptionPlan model and subscription_plans table (migration 015)
- Add billing cycle/period/allow_overage fields to UserProfile (migration 016)
- Add subscription_overage_percent config field (replaces overage_factor)
- Rewrite check_upload_allowed: use overage_percent, yearly carry-over, no daily cap
- Add seed_default_plans(), _plan_to_dict(), get_year_file_count(), _months_elapsed()
- Update get_tier/get_all_tiers to be DB-first with TIER_DEFAULTS fallback
- Add TIER_DEFAULTS alias (TIERS kept for backward compat)
- New /api/plans/ CRUD endpoints (admin-only except list/get)
- New /admin/plans Plan Designer page with Alpine.js UI
- Add Plan Designer link to admin navigation in base.html
- Remove 'Files per day' row from pricing comparison table
- Add billing cycle + period start to admin users edit modal
- Seed default plans on startup in lifespan handler
- Rewrite docs/SubscriptionTiers.md with full plan/overage/API docs
- Fix all tests in test_subscription.py (remove daily cap tests, add overage/carry-over tests)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Use shared _require_admin from admin_users in subscriptions API endpoint
- Remove unnecessary Alpine.js hidden-div workaround in pricing.html
- Replace fragile string replace for OCR page count with proper Jinja {:,} format
- Improve comment wording in upload quota cleanup code
- Extract _scalar_count() helper in subscription.py to reduce repetition
- Add aria-valuemin='0' to all progressbar elements in subscription/index templates
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Free / Starter / Professional / Business tiers with lifetime, daily, and monthly
file limits (app/utils/subscription.py)
- Add subscription_tier column to UserProfile model + migration 014
- Enforce quotas at upload time (HTTP 402 on violation) in /api/ui-upload
- New REST API: GET /api/subscriptions/tiers, /my, /platform (admin)
- New pages: /pricing (marketing, public) and /subscription (per-user status)
- Enhanced dashboard: SaaS stats (files today/month, OCR count, active users)
in multi-user mode; original single-user layout preserved
- Admin users page: show Plan badge, allow tier editing via dropdown
- Navigation: add Pricing link + subscription icon in user header
- Tests: 23 unit tests for subscription tier logic
- Docs: docs/SubscriptionTiers.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add "DB Wizard" link button to settings page header
- Add help_link to database_url SETTING_METADATA pointing to /database-wizard
- Add help_link rendering in settings template for any setting with a help_link
- Fix SQLite whitespace path handling in build_connection_string
- Add dark mode CSS overrides for wizard template
- Add aria-describedby for all form inputs with help text
- Add prefers-reduced-motion media query for smooth scrolling
- Expand test coverage: 106 tests (up from 49)
- db_wizard.py: 100% coverage
- db_wizard view: 100% coverage
- database.py API: 97.37% coverage
- db_migrate.py: 96.60% coverage
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add inline safety comment for noqa: S608 (table_name from inspect)
- Fix HTTPException detail to be a string (not dict)
- Add aria-label to migration progress bar
- Rename _noop to _NoOpContextManager in tests
- Add explanatory comment for zip(strict=False)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add a guided database configuration wizard and a data migration tool that
allows users to:
- Build database connection strings through a step-by-step UI
- Test database connections before applying
- Preview and execute data migrations from SQLite to PostgreSQL/MySQL
- Copy to clipboard for easy .env file updates
New files:
- app/utils/db_wizard.py — connection string builder, parser, and tester
- app/utils/db_migrate.py — table-by-table data migration utility
- app/api/database.py — REST API endpoints for wizard operations
- app/views/db_wizard.py — view route for the wizard page
- frontend/templates/db_wizard.html — multi-tab wizard UI
- tests/test_db_wizard.py — unit tests for db_wizard utilities
- tests/test_db_migrate.py — unit tests for db_migrate utilities
- tests/test_db_wizard_api.py — integration tests for API and views
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replace the fragile global DOM query approach in `updateOverallStatus`
with per-batch closure-based counters inside `processFiles`.
- Add `total`, `done`, `updateStatus()`, and `markDone()` as closure
variables/functions within each `processFiles` invocation
- Change `_uploadSingleFile` to accept an `onTerminal` callback instead
of `statusMessage`, called when a file reaches a terminal state
- Pass `markDone` as the `onTerminal` callback from `scheduleNext`
- Remove the now-unused global `updateOverallStatus` function
The previous implementation queried `document.querySelectorAll('.file-status')`
globally and relied on text `startsWith` checks to count completed files.
This was fragile and could produce a stale done=0 count in practice.
The new approach uses deterministic closure counters, so the displayed
"Uploading files (X/N)" count correctly increments as files complete.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace with CSS spacing classes for accessibility
- Use |tojson filter for search index to prevent XSS
- Add IntersectionObserver cleanup via Alpine $cleanup
- Add sr-only setting key text for mobile screen readers
- Respect prefers-reduced-motion for smooth scrolling
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add real-time search/filter bar for finding settings by name, key, or description
- Add sidebar navigation with category icons and setting counts
- Make category sections collapsible with smooth animations
- Add mobile-friendly category dropdown selector
- Show setting key as code badge for quick reference
- Compact header with inline precedence/legend info
- Add intersection observer for active category tracking in sidebar
- Add no-results state with clear search action
- Maintain all existing functionality (save, revert, bulk save, alerts)
- Full dark mode compatibility via existing CSS overrides
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add GET /api/similarity/pairs endpoint for corpus-wide pair discovery
- Add /similarity view route and similarity_dashboard.html template
- Add Similarity link to desktop and mobile nav menus
- Register compute_embedding as a tracked FileProcessingStep
- Update compute_embedding task with update_step_status calls
- Add compute_embedding to flow visualization in _compute_processing_flow
- Add backfill_missing_embeddings periodic beat task (every 5 min)
- Return clear message when embedding not yet computed in similar docs API
- Fix all tests to use pre-computed embeddings (no lazy API calls)
- Add tests for similarity pairs, backfill task, and embedding-not-computed
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add near_duplicate_threshold config setting (default 0.85)
- New GET /api/duplicates endpoint listing all exact-duplicate groups
- New GET /api/files/{id}/duplicates endpoint returning exact + near-duplicates
- POST /api/ui-upload now returns immediate exact-duplicate warning (respects ENABLE_DEDUPLICATION)
- New /duplicates management UI with Exact Duplicates tab and Near-Duplicate Finder tab
- Add Duplicates link in admin nav menu (desktop + mobile)
- Document new config options in ConfigurationGuide.md and .env.demo
- 20 new tests covering all acceptance criteria
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add release_names.json mapping version ranges to codenames
- Add release_name property to Settings in app/config.py
- Update build metadata script to include codename in RUNTIME_INFO
- Display release codename in status dashboard and page footer
- Inject release_name globally via template response wrapper
- Update ROADMAP.md with codenames for all milestone releases
- Add docs/ReleaseNaming.md with naming guide and best practices
- Add comprehensive tests for release name resolution
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Change default per_page from 50 to 25
- Rename total_items → total, total_pages → pages in pagination response
- Add next/previous URL fields to pagination response
- Update view and template to use new field names
- Update tests and API docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add tags, sender, text_quality filters to search API and Meilisearch client
- Add sender and ocr_text_length to Meilisearch filterable attributes
- Expand saved search allowed filter keys to include q, document_type, language, sender, text_quality
- Add filters panel and saved searches UI to the Search view template
- Add tests for new search filters, saved search keys, and search view elements
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add date range (date_from/date_to), storage provider, and tags filters to GET /api/files
- Add SavedSearch model and migration (005_add_saved_searches)
- Add CRUD API endpoints for saved searches at /api/saved-searches
- Update files.html template with new filter controls and saved searches UI
- Update files view to pass new filter parameters to template
- Add comprehensive tests for all new functionality (26 tests)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace iframe in file_view.html with pdf.js for PDF files
- Add image viewer with zoom/pan controls to file_view.html
- Add text file viewer with line numbers to file_view.html
- Add preview side-panel modal to files.html (file list)
- Fix file_detail.html bottom preview section variable names
- Add aria-labels and WCAG compliance to all new elements
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]