- PDFs are now uploaded to Mistral Files API (POST /v1/files) and
processed via a signed document_url, resolving the 422 error caused
by passing data:application/pdf;base64,... to an image endpoint
- Images (JPEG/PNG/GIF/WEBP/BMP/TIFF) use base64 image_url directly
- Unsupported MIME types raise a clear ValueError
- Magic-byte fallback detects PDFs with no file extension
- Switches from openai chat completions to requests HTTP calls
- Adds helper method _upload_pdf_and_get_document()
- Adds TestMistralOCRProvider with 9 unit tests covering all paths
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
A) Per-option Save Button
- Add per-setting Save button in settings.html (visible only when value changed)
- Button calls POST /api/settings/{key} directly; existing bulk Save retained
- Add Audit Log link in settings page header
B) Immediate Worker Sync
- New app/utils/settings_sync.py with notify_settings_updated() (Redis version key)
and register_settings_reload_signal() (Celery task_prerun handler)
- Register signal in celery_worker.py at startup
- All API write paths call notify_settings_updated() after successful saves
C) Audit Log
- Add SettingsAuditLog model (key, old_value, new_value, changed_by, changed_at, action)
- save_setting_to_db / delete_setting_from_db accept changed_by and write audit entries
- New get_audit_log() service function (masks sensitive values)
- New GET /api/settings/audit-log endpoint (admin-only)
- New GET /admin/settings/audit-log view + audit_log.html template
- Visible to all admins (per clarified requirement)
D) Config Rollback / History
- New get_setting_history() and rollback_setting() service functions
- New GET /api/settings/{key}/history endpoint
- New POST /api/settings/{key}/rollback/{history_id} endpoint
- Rollback buttons in audit_log.html with confirmation dialog
- Tests: 25 new tests covering audit log, rollback, worker sync helpers, and API endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
`GET /api/settings/{key}` was calling `validate_setting_key()` which raises
HTTP 404 for keys not in SETTING_METADATA. The test expects 200 with value=None
for unknown keys.
Added `validate_setting_key_format()` to `input_validation.py` that validates
only the key format without the SETTING_METADATA existence check. Updated
`get_setting` to use the format-only validator; POST/DELETE endpoints continue
using the full `validate_setting_key()` for write-side security.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add CORSMiddleware (disabled by default, enabled via CORS_ENABLED=true)
- Add cors_enabled, cors_allowed_origins, cors_allow_credentials,
cors_allowed_methods, cors_allowed_headers settings to config.py
- Add parse_comma_separated_list validator for CORS list env vars
- Insert CORS middleware between SessionMiddleware and ProxyHeaders
so preflight runs before CSRF/auth but after proxy-header processing
- Document CORS env vars in .env.demo with rationale for proxy-first approach
- Mark CORS TODO as completed in SECURITY_AUDIT.md
- Add tests/test_cors.py with 12 unit and integration tests
Closes#175
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add RequestSizeLimitMiddleware that checks Content-Length header
before request body is read: non-multipart requests capped at
MAX_REQUEST_BODY_SIZE (default 1 MB), multipart uploads capped at
MAX_UPLOAD_SIZE (default 1 GB). Returns HTTP 413 on violation.
- Register middleware in app/main.py
- Add max_request_body_size setting to app/config.py
- Fix ui_upload in files.py to check Content-Length early and read
in 64 KB chunks (bounded memory usage), removing the post-write
os.path.getsize check
- Document MAX_REQUEST_BODY_SIZE in .env.demo and ConfigurationGuide.md
- Mark SECURITY_AUDIT.md item #4 as resolved
- Add 9 tests in test_request_size_limit.py
- Update test_upload_file_too_large to use patch.object instead of
the now-unused os.path.getsize mock
Closes#173
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix ruff format: add blank line before nested function in app/views/filemanager.py
- Fix ruff format: use double-quote escaping in tests/test_config.py
- Remove deploy job (Portainer webhook) from ci.yml
- Add update-k8s-manifest job: after main-branch build, updates
apps/docuelevate/preprod/docuelevate-stack.yaml in christianlouis/k8s-cluster-state
with the new GHCR image tag (ghcr.io/christianlouis/docuelevate:main-<short-sha>)
using mikefarah/yq@v4.44.6 and GH_PAT secret for cross-repo write access
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add [[tool.mypy.overrides]] section for app/utils/** with disallow_untyped_defs=true
- Add type annotations to all functions in app/utils/ (12 files)
- Fix type annotations in app/config.py and app/database.py (imported by utils)
- All 85 source files now pass mypy type checking
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add comprehensive tests for imap_tasks.py covering:
- Lock acquire/release mechanisms
- pull_all_inboxes task with various scenarios
- pull_inbox for Gmail and non-Gmail with edge cases
- find_all_mail_xlist functionality
- Extended fetch_attachments tests for all MIME types
- Edge cases: invalid JSON, missing Message-ID, already processed, etc.
- Achieve 98.26% coverage for imap_tasks.py (up from 48.78%)
- Fix config_validator.py to include validate_auth_config export
- Update tests to verify all exports including validate_auth_config
- Note: config_validator.py file is shadowed by config_validator/ directory
in Python's module resolution, so it cannot be directly imported or tested.
The package's __init__.py (which has 100% coverage) is what's actually used.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add comprehensive logging to all pipeline retry branches (process_document, process_with_azure_document_intelligence, extract_metadata_with_gpt, embed_metadata_into_pdf)
- Add original_file_path as 3rd fallback for embed_metadata_into_pdf (checks: local_filename, processed_file_path, original_file_path, workdir/tmp fallback)
- Include all checked paths with existence status in 400 error responses for easier debugging
- Add enhanced logging to upload task retry path showing which processed file paths were checked
- Log successful file path when found
- Add comprehensive test suite covering new logging and fallback behavior
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]