Add a configurable switch to control which attachment types are ingested
via IMAP. Images are excluded by default; office files and PDFs are ingested.
- Add global `IMAP_ATTACHMENT_FILTER` config setting (default: `documents_only`)
- Add `attachment_filter` column to `UserImapAccount` model for per-user override
- Migration 032 adds the column to `user_imap_accounts` table
- Update `fetch_attachments_and_enqueue()` to respect filter (documents_only/all)
- Update `pull_inbox()`, `_pull_user_imap_accounts()`, and
`_pull_user_integration_imap()` to pass the resolved filter
- Update IMAP accounts API (schemas, create/update handlers, response serializer)
- Update IMAP accounts UI to show attachment filter dropdown in modal and
display filter badges on account cards
- Add 6 new tests covering attachment filter behaviour
- Update ConfigurationGuide.md, EmailIngestion.md, and .env.demo
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge main branch into compliance templates feature branch.
Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.
Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge main into branch to pull in translation files, audit log templates,
CI workflow updates, and other changes from PRs #595, #597, #598.
Keep mobile_router and MobileDevice additions from this branch.
- Replace MkDocs redirect with a branded, SEO-optimised Help Center template
- Add sections: Quick Start, Sources, Destinations, Workflows, FAQ, Support
- Integrate optional Zammad live-chat widget and ticket form
- Add config settings: ZAMMAD_URL, ZAMMAD_CHAT_ENABLED, ZAMMAD_CHAT_ID,
ZAMMAD_FORM_ENABLED, SUPPORT_EMAIL
- Move MkDocs developer docs from /help to /developer-docs
- Move interactive API docs (Swagger/ReDoc) to /admin/api-docs and /admin/api-redoc
- Add API Docs and Developer Docs links to Admin menu (desktop + mobile)
- Update navigation Help link from /help/ to /help
- Update .env.demo with Zammad configuration examples
- Document new settings in docs/ConfigurationGuide.md
- Rewrite tests to cover new Help Center behaviour
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Create app/templates/email/default.html (fixes 'default.html not found' error)
- Add DEST_EMAIL_* settings to app/config.py (decoupled from shared EMAIL_* settings)
- Update upload_to_email task to use dest_email_* settings exclusively
- Update _should_upload_to_email() to check dest_email_* settings
- Update config validator, providers, and settings_service for dest_email_*
- Update .env.demo and docs/ConfigurationGuide.md
- Update all tests to use dest_email_* settings where appropriate"
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Move `import os` to top-level in app/views/backup.py
- Fix docstring in BackupRecord model to remove non-existent 'location' field
- Replace browser confirm() dialogs with accessible modal dialog (role=dialog, aria-modal, aria-labelledby)
- Add csrfToken() helper that validates token presence instead of silently falling back to empty string
- Fix aria-live region to remain in DOM (screen-reader friendly) rather than using x-show
- Add Backup & Restore section to docs/ConfigurationGuide.md with retention table
- Add backup env vars to .env.demo with comments
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Remove SMTP hard-requirement from /api/auth/signup: when SMTP is not
configured accounts are activated immediately (no email verification).
When SMTP is configured the existing email-verification flow is kept.
- Inject allow_signup into global template context via app/views/base.py
- Add data-allow-signup attribute to base.html body tag
- Update common.js _renderLoggedOutAuth to show Sign Up (→ /signup) when
signup is enabled, otherwise Get Started (→ /pricing)
- Add admin API endpoints before the /{user_id:path} catch-all:
GET /api/admin/users/local – list all local accounts
POST /api/admin/users/local – admin-create local account (active immediately)
DELETE /api/admin/users/local/{id} – delete local account + profile
- Add LocalUserCreate / LocalUserResponse Pydantic schemas
- Update admin_users.html with Local User Accounts section and modals
- Update .env.demo to document ALLOW_LOCAL_SIGNUP
- Update docs/BillingSetup.md: SMTP is optional, document both flows
- Update tests: test_signup_smtp_not_configured now asserts 201 + immediate
activation; add 7 new integration tests for admin local user endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add EMBEDDING_MAX_TOKENS config (default 8000) for safe text truncation
- Use conservative 3 chars/token estimate (was 4) to prevent ContextWindowExceededError
- Add compute_embedding to REAL_MAIN_STEPS in both get_file_overall_status and get_step_summary
- Fix test_near_duplicates_returned to use pre-computed embeddings
- Update .env.demo and docs with EMBEDDING_MAX_TOKENS setting
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add near_duplicate_threshold config setting (default 0.85)
- New GET /api/duplicates endpoint listing all exact-duplicate groups
- New GET /api/files/{id}/duplicates endpoint returning exact + near-duplicates
- POST /api/ui-upload now returns immediate exact-duplicate warning (respects ENABLE_DEDUPLICATION)
- New /duplicates management UI with Exact Duplicates tab and Near-Duplicate Finder tab
- Add Duplicates link in admin nav menu (desktop + mobile)
- Document new config options in ConfigurationGuide.md and .env.demo
- 20 new tests covering all acceptance criteria
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
When enabled, IMAP processing will fetch and process attachments but
will NOT modify the mailbox state (no starring, labeling, deleting,
or flag changes). This allows preprod instances to safely share a
Gmail inbox with production without interfering with production
email processing.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add CORSMiddleware (disabled by default, enabled via CORS_ENABLED=true)
- Add cors_enabled, cors_allowed_origins, cors_allow_credentials,
cors_allowed_methods, cors_allowed_headers settings to config.py
- Add parse_comma_separated_list validator for CORS list env vars
- Insert CORS middleware between SessionMiddleware and ProxyHeaders
so preflight runs before CSRF/auth but after proxy-header processing
- Document CORS env vars in .env.demo with rationale for proxy-first approach
- Mark CORS TODO as completed in SECURITY_AUDIT.md
- Add tests/test_cors.py with 12 unit and integration tests
Closes#175
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add RequestSizeLimitMiddleware that checks Content-Length header
before request body is read: non-multipart requests capped at
MAX_REQUEST_BODY_SIZE (default 1 MB), multipart uploads capped at
MAX_UPLOAD_SIZE (default 1 GB). Returns HTTP 413 on violation.
- Register middleware in app/main.py
- Add max_request_body_size setting to app/config.py
- Fix ui_upload in files.py to check Content-Length early and read
in 64 KB chunks (bounded memory usage), removing the post-write
os.path.getsize check
- Document MAX_REQUEST_BODY_SIZE in .env.demo and ConfigurationGuide.md
- Mark SECURITY_AUDIT.md item #4 as resolved
- Add 9 tests in test_request_size_limit.py
- Update test_upload_file_too_large to use patch.object instead of
the now-unused os.path.getsize mock
Closes#173
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Security headers are now disabled by default since most deployments use a reverse proxy (Traefik, Nginx) that already adds these headers. Enable with SECURITY_HEADERS_ENABLED=true for direct deployments.
Changes:
- Set security_headers_enabled default to False in app/config.py
- Update all documentation to reflect new default
- Comment out examples in .env.demo (now showing disabled state)
- Update SECURITY_AUDIT.md to reflect reverse proxy as default deployment
- Tests still pass (3 passed, 8 skipped as expected with headers disabled)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add MAX_UPLOAD_SIZE config (default 1GB) to prevent resource exhaustion
- Add MAX_SINGLE_FILE_SIZE config for optional PDF file splitting
- Implement automatic PDF splitting when files exceed single file limit
- Update upload endpoint to use configured limits instead of hardcoded 500MB
- Add comprehensive tests for upload limits and file splitting
- Document configuration in ConfigurationGuide.md and SECURITY_AUDIT.md
- Reference SECURITY_AUDIT.md in error messages for user guidance
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Christian Krakau-Louis