Added tests for POST /api/dropbox/list-folders (7 tests) and
POST /api/onedrive/list-folders (7 tests) covering success, subfolder
navigation, empty directories, auth errors, API errors, path
normalization, and alphabetical sorting.
Added view tests for system credentials toggle visibility in Dropbox
setup wizard.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
When system-wide Dropbox, Google Drive, or OneDrive app credentials are
configured by the admin, user-mode OAuth wizards now default to using
them. A toggle lets users switch to custom credentials if needed. This
removes the need for end users to register their own cloud provider apps.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add user_mode to dropbox/onedrive/google_drive setup views that loads integration config
- Show user-friendly auth wizard when integration_id is provided (user mode)
- In user mode: show integration name, current folder, back-to-integrations link
- In callback templates: only save credentials (not config) for user integrations
- In integrations dashboard: show Authorize/Re-Authorize button for all OAuth types
- Add WATCH_FOLDER OAuth support: detect source_type in config for auth button
- isOAuthType() and oauthLink() now accept full integration object
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add SharedLink model with token, expiry, view limit, password hash
- Add migration 025_add_shared_links
- Add API endpoints: create, list, revoke (auth) + public info/download
- Add management UI at /shared-links with revoke controls
- Add public share landing page at /share/{token}
- Add Share button on file_view.html
- Add Shared Links to user dropdown in common.js
- Write 35 unit tests covering all scenarios
- Update UserGuide.md with sharing documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add 'Authorize' button to integration cards for OAuth types without credentials
- Add isOAuthType() helper and update info box in create/edit modal
- Accept integration_id query param in Dropbox, Google Drive, OneDrive setup views
- Store integration_id in sessionStorage on setup pages
- Add per-user flow in OAuth callbacks: PUT credentials to /api/integrations/{id}
- Preserve existing global flow as fallback when no integration_id is present
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pass authenticated user's name, email, and username to Zammad form
and chat widgets. For the ticket form: pre-fill name/email fields
and append a User Context metadata block to the ticket body via
$.ajaxPrefilter. For the chat widget: pass name/email to the
ZammadChat constructor.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add integrity/crossorigin attributes to jQuery 3.6.0 CDN script
- Use | int filter on zammad_chat_id to prevent XSS
- Replace request.headers.get('host') with settings.external_hostname
for canonical URL and Open Graph tags to prevent host-header injection
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace MkDocs redirect with a branded, SEO-optimised Help Center template
- Add sections: Quick Start, Sources, Destinations, Workflows, FAQ, Support
- Integrate optional Zammad live-chat widget and ticket form
- Add config settings: ZAMMAD_URL, ZAMMAD_CHAT_ENABLED, ZAMMAD_CHAT_ID,
ZAMMAD_FORM_ENABLED, SUPPORT_EMAIL
- Move MkDocs developer docs from /help to /developer-docs
- Move interactive API docs (Swagger/ReDoc) to /admin/api-docs and /admin/api-redoc
- Add API Docs and Developer Docs links to Admin menu (desktop + mobile)
- Update navigation Help link from /help/ to /help
- Update .env.demo with Zammad configuration examples
- Document new settings in docs/ConfigurationGuide.md
- Rewrite tests to cover new Help Center behaviour
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Move `import os` to top-level in app/views/backup.py
- Fix docstring in BackupRecord model to remove non-existent 'location' field
- Replace browser confirm() dialogs with accessible modal dialog (role=dialog, aria-modal, aria-labelledby)
- Add csrfToken() helper that validates token presence instead of silently falling back to empty string
- Fix aria-live region to remain in DOM (screen-reader friendly) rather than using x-show
- Add Backup & Restore section to docs/ConfigurationGuide.md with retention table
- Add backup env vars to .env.demo with comments
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Both sets of UserProfile columns are retained:
- is_complimentary (from main, migration 019_add_is_complimentary)
- subscription_change_pending_tier / subscription_change_pending_date
(our branch, renamed to migration 020_add_subscription_change_pending
with down_revision updated to chain after 019_add_is_complimentary)
- Add MkDocs Material docs build stage to Dockerfile and Dockerfile.local
- Mount pre-built docs as static files at /help/ in FastAPI (app/main.py)
- Add app/views/help.py with /help → /help/ permanent redirect route
- Register help router in app/views/__init__.py
- Add Help nav link to base.html (public + app nav, desktop + mobile)
- Create how-to guides: HP printer, ScanSnap, watched folder, email ingestion, mobile scanning
- Update mkdocs.yml with How-To Guides section and Material theme palette
- Add optional docs service (squidfunk/mkdocs-material) to docker-compose.yaml with docs profile
- Add mkdocs-material to requirements-dev.txt
- Add /docs_build to .gitignore
- Add tests for help view (8 tests, 100% coverage on help.py)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add subscription_change_pending_tier and subscription_change_pending_date fields to UserProfile
- Create migration 019_add_subscription_change_pending
- Add apply_pending_subscription_changes(), request_subscription_change(), cancel_pending_subscription_change() utilities
- Add POST /api/subscriptions/change and DELETE /api/subscriptions/change endpoints
- Update GET /api/subscriptions/my to apply pending changes and return pending change info
- Update subscription view to apply pending changes and pass period_start + pending info
- Update subscription.html: per-tier action buttons (upgrade/downgrade/cancel), pending-change banner, period start date
- Add Celery daily task apply_pending_subscription_changes_all at 00:05 UTC
- Add 19 new tests covering all new utility functions and API endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Remove SMTP hard-requirement from /api/auth/signup: when SMTP is not
configured accounts are activated immediately (no email verification).
When SMTP is configured the existing email-verification flow is kept.
- Inject allow_signup into global template context via app/views/base.py
- Add data-allow-signup attribute to base.html body tag
- Update common.js _renderLoggedOutAuth to show Sign Up (→ /signup) when
signup is enabled, otherwise Get Started (→ /pricing)
- Add admin API endpoints before the /{user_id:path} catch-all:
GET /api/admin/users/local – list all local accounts
POST /api/admin/users/local – admin-create local account (active immediately)
DELETE /api/admin/users/local/{id} – delete local account + profile
- Add LocalUserCreate / LocalUserResponse Pydantic schemas
- Update admin_users.html with Local User Accounts section and modals
- Update .env.demo to document ALLOW_LOCAL_SIGNUP
- Update docs/BillingSetup.md: SMTP is optional, document both flows
- Update tests: test_signup_smtp_not_configured now asserts 201 + immediate
activation; add 7 new integration tests for admin local user endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Inject is_logged_in, multi_user_enabled, auth_enabled into all templates
via app/views/base.py _inject_global_context() helper
- Multi-user + logged-out: show only Pricing, About, Log In, Get Started
- Logged-in or single-user: full app nav (Dashboard, Upload, Files,
Search, Pipelines, Admin dropdown, Status)
- Upload link is visually accented (blue) as the primary action
- Account dropdown (avatar, name, email, subscription, sign-out) for
logged-in users in desktop and mobile
- Admin dropdown Similarity icon changed to purple to differentiate
from Queue Monitor
- data-multi-user attribute on <body> so JS reads the mode at runtime
- 5 new unit tests for is_logged_in/multi_user_enabled injection
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The file detail page (/files/{id}/detail) and file view page (/files/{id})
previously showed no information about which processing pipeline was used.
Changes:
- _STEP_TYPE_TO_STAGES mapping: pipeline step_type → Celery log stage keys
(with maintenance comment requiring updates when new step types are added)
- _ALWAYS_SHOW_STAGES: stages always visible regardless of pipeline
- _resolve_pipeline(db, file_record): resolves the pipeline for a file —
uses explicit pipeline_id when set, falls back to active system default
- _compute_processing_flow: new pipeline_steps parameter; when provided,
filters flow graph to only show stages for the pipeline's enabled steps
(+ always-show stages + any stage that actually ran). Also adds
convert_to_pdf to the flow stage catalogue.
- file_detail_page: passes pipeline_info + pipeline-filtered flow_data
- file_view_page: passes pipeline_info
Templates:
- file_detail.html: 'Processing Pipeline' detail row with name link and
colour-coded badge (System Default / System / Custom)
- file_view.html: 'Pipeline' info row in sidebar with (default)/(custom) tag
Tests:
- TestPipelineInfoInViews with 14 tests covering _resolve_pipeline,
_compute_processing_flow filtering, completeness assertion for
_STEP_TYPE_TO_STAGES, and HTTP-level view tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- 5-step wizard: Welcome → Profile → Plan → Storage → All Set!
- New migration 017: onboarding_completed, contact_email, preferred_destination fields
- REST API at /api/onboarding/{status,profile,plan,storage,complete}
- GET /onboarding view with configured-destinations helper
- OAuth callback redirects first-time users to onboarding
- 16 unit tests for all endpoints; 65 total tests pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Pipeline and PipelineStep models with user-specific ownership
- Add pipeline_id FK column to FileRecord
- Migration 017_add_pipelines (batch mode for SQLite FK compat)
- Pipeline CRUD API at /api/pipelines with step management endpoints
- Reorder steps PUT endpoint placed before parameterised {step_id} routes
- POST /api/files/{id}/assign-pipeline for per-file pipeline assignment
- Admin-only POST /api/pipelines/admin/system for system-level pipelines
- Management UI at /pipelines (Jinja2 + Alpine.js + Tailwind)
- Pipelines link added to desktop and mobile navigation
- 41 new tests in tests/test_api_pipelines.py (all passing)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add SubscriptionPlan model and subscription_plans table (migration 015)
- Add billing cycle/period/allow_overage fields to UserProfile (migration 016)
- Add subscription_overage_percent config field (replaces overage_factor)
- Rewrite check_upload_allowed: use overage_percent, yearly carry-over, no daily cap
- Add seed_default_plans(), _plan_to_dict(), get_year_file_count(), _months_elapsed()
- Update get_tier/get_all_tiers to be DB-first with TIER_DEFAULTS fallback
- Add TIER_DEFAULTS alias (TIERS kept for backward compat)
- New /api/plans/ CRUD endpoints (admin-only except list/get)
- New /admin/plans Plan Designer page with Alpine.js UI
- Add Plan Designer link to admin navigation in base.html
- Remove 'Files per day' row from pricing comparison table
- Add billing cycle + period start to admin users edit modal
- Seed default plans on startup in lifespan handler
- Rewrite docs/SubscriptionTiers.md with full plan/overage/API docs
- Fix all tests in test_subscription.py (remove daily cap tests, add overage/carry-over tests)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add Free / Starter / Professional / Business tiers with lifetime, daily, and monthly
file limits (app/utils/subscription.py)
- Add subscription_tier column to UserProfile model + migration 014
- Enforce quotas at upload time (HTTP 402 on violation) in /api/ui-upload
- New REST API: GET /api/subscriptions/tiers, /my, /platform (admin)
- New pages: /pricing (marketing, public) and /subscription (per-user status)
- Enhanced dashboard: SaaS stats (files today/month, OCR count, active users)
in multi-user mode; original single-user layout preserved
- Admin users page: show Plan badge, allow tier editing via dropdown
- Navigation: add Pricing link + subscription icon in user header
- Tests: 23 unit tests for subscription tier logic
- Docs: docs/SubscriptionTiers.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add UserProfile model (app/models.py) with per-user settings: display_name, daily_upload_limit, notes, is_blocked
- Add Alembic migration 013_add_user_profiles for the new table
- Add REST API at /api/admin/users/ with list, get, upsert (PUT), delete endpoints (admin-only)
- Add HTML template admin_users.html with Alpine.js: filterable user list, paginated table, edit/create modal, delete confirmation modal
- Add view handler at /admin/users (admin-only redirect guard)
- Register routers in app/api/__init__.py and app/views/__init__.py
- Add 'Users' link to admin nav dropdown in base.html (desktop + mobile)
- Add 27 tests covering auth, list, get, upsert, delete, and model constraints
- Register UserProfile in conftest.py model imports
- Document new endpoints in docs/API.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add a guided database configuration wizard and a data migration tool that
allows users to:
- Build database connection strings through a step-by-step UI
- Test database connections before applying
- Preview and execute data migrations from SQLite to PostgreSQL/MySQL
- Copy to clipboard for easy .env file updates
New files:
- app/utils/db_wizard.py — connection string builder, parser, and tester
- app/utils/db_migrate.py — table-by-table data migration utility
- app/api/database.py — REST API endpoints for wizard operations
- app/views/db_wizard.py — view route for the wizard page
- frontend/templates/db_wizard.html — multi-tab wizard UI
- tests/test_db_wizard.py — unit tests for db_wizard utilities
- tests/test_db_migrate.py — unit tests for db_migrate utilities
- tests/test_db_wizard_api.py — integration tests for API and views
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add GET /api/similarity/pairs endpoint for corpus-wide pair discovery
- Add /similarity view route and similarity_dashboard.html template
- Add Similarity link to desktop and mobile nav menus
- Register compute_embedding as a tracked FileProcessingStep
- Update compute_embedding task with update_step_status calls
- Add compute_embedding to flow visualization in _compute_processing_flow
- Add backfill_missing_embeddings periodic beat task (every 5 min)
- Return clear message when embedding not yet computed in similar docs API
- Fix all tests to use pre-computed embeddings (no lazy API calls)
- Add tests for similarity pairs, backfill task, and embedding-not-computed
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add near_duplicate_threshold config setting (default 0.85)
- New GET /api/duplicates endpoint listing all exact-duplicate groups
- New GET /api/files/{id}/duplicates endpoint returning exact + near-duplicates
- POST /api/ui-upload now returns immediate exact-duplicate warning (respects ENABLE_DEDUPLICATION)
- New /duplicates management UI with Exact Duplicates tab and Near-Duplicate Finder tab
- Add Duplicates link in admin nav menu (desktop + mobile)
- Document new config options in ConfigurationGuide.md and .env.demo
- 20 new tests covering all acceptance criteria
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add release_names.json mapping version ranges to codenames
- Add release_name property to Settings in app/config.py
- Update build metadata script to include codename in RUNTIME_INFO
- Display release codename in status dashboard and page footer
- Inject release_name globally via template response wrapper
- Update ROADMAP.md with codenames for all milestone releases
- Add docs/ReleaseNaming.md with naming guide and best practices
- Add comprehensive tests for release name resolution
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Change default per_page from 50 to 25
- Rename total_items → total, total_pages → pages in pagination response
- Add next/previous URL fields to pagination response
- Update view and template to use new field names
- Update tests and API docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]