- Implement step timeout detection to prevent files from getting stuck in 'pending' state
- Add monitor_stalled_steps periodic task running every minute (Celery Beat)
- Automatically mark in-progress steps as failed if they exceed timeout (default: 10 minutes)
- Add step_timeout configuration setting (default: 600 seconds)
- Recover stalled steps with error message indicating when timeout was triggered
- Fix duplicate check to exclude self-comparison (file not duplicate of itself)
When processing crashes or hangs:
1. Worker detects stalled steps (in_progress for >10 minutes)
2. Marks them as failed with timeout error message
3. Updates UI to show failure status
4. Allows file to be retried or handled by user
This prevents files from being indefinitely stuck in processing state and provides
visibility into what went wrong.
- Add enable_deduplication and show_deduplication_step config options
- Rename hash_file step to check_for_duplicates
- Make deduplication step conditional based on configuration
- Add is_duplicate and duplicate_of_id fields to FileRecord model
- Create database migration for new deduplication fields
- Update process_document task to log deduplication results
- Update step visualization to show/hide step based on config
- Update status calculations to include deduplication step conditionally
- Default: deduplication enabled, step displayed
- Can be configured to hide from UI while still processing
- Update generate_build_metadata.sh to sync VERSION from latest git tag
- Change BUILD_DATE format from date-only to ISO 8601 with time (YYYY-MM-DDTHH:MM:SSZ)
- Fix VERSION file from 0.5.0 to 0.9.1 (matching latest git tag v0.9.1)
- Change version fallback from hardcoded '0.5.0-dev' to 'unknown' in config.py
- Update release.yml to commit all build metadata files (not just VERSION)
- Update BuildMetadata.md documentation to reflect automated versioning
- Add tests for build_date with time format and version unknown fallback
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Security headers are now disabled by default since most deployments use a reverse proxy (Traefik, Nginx) that already adds these headers. Enable with SECURITY_HEADERS_ENABLED=true for direct deployments.
Changes:
- Set security_headers_enabled default to False in app/config.py
- Update all documentation to reflect new default
- Comment out examples in .env.demo (now showing disabled state)
- Update SECURITY_AUDIT.md to reflect reverse proxy as default deployment
- Tests still pass (3 passed, 8 skipped as expected with headers disabled)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add MAX_UPLOAD_SIZE config (default 1GB) to prevent resource exhaustion
- Add MAX_SINGLE_FILE_SIZE config for optional PDF file splitting
- Implement automatic PDF splitting when files exceed single file limit
- Update upload endpoint to use configured limits instead of hardcoded 500MB
- Add comprehensive tests for upload limits and file splitting
- Document configuration in ConfigurationGuide.md and SECURITY_AUDIT.md
- Reference SECURITY_AUDIT.md in error messages for user guidance
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Run Black formatter and isort on all app/ files
- Remove unused imports (F401) across multiple files
- Add # noqa: F401 for intentional re-exports in celery_worker.py,
tasks/__init__.py, utils.py, frontend.py, views/base.py
- Fix f-strings without placeholders (F541) in azure.py, notification.py,
check_credentials.py, upload_to_onedrive.py, settings.py
- Fix bare except (E722) in upload_to_sftp.py
- Fix block comment format (E265) in models.py
- Move imports to top of file to fix E402 in celery_app.py, celery_worker.py
- Fix line-too-long (E501) by wrapping strings in multiple files
- Remove unused variable (F841) in upload_to_nextcloud.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Settings management represents a significant new feature warranting a minor version bump:
- Database-backed configuration with 102 settings
- Fernet encryption for sensitive values
- Setup wizard for first-time installations
- Complete admin UI and REST API
- OAuth admin group support
Version updated: 0.3.3 → 0.5.0
Keep 0.3.3 release notes intact (drag-and-drop feature from main branch).
Add 0.5.0 as new current release with settings features.
Updated files:
- VERSION: 0.5.0
- CHANGELOG.md: Added 0.5.0 release, kept 0.3.3 intact
- MILESTONES.md: Added v0.5.0 section, adjusted future versions
- TODO.md: Updated current version
- ROADMAP.md: Updated current status
- app/config.py: Default version 0.5.0-dev
- docs/BuildMetadata.md: Updated reference
- ANALYSIS_SUMMARY.md: Updated version
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
VERSION:
- Update from 0.1.0-test to 0.3.3
CHANGELOG.md:
- Create comprehensive changelog for all releases
- Document v0.3.3 features: settings management, encryption, setup wizard
- List all new files, changes, fixes, and security improvements
TODO.md:
- Update version reference to v0.3.3
- Mark settings management features as complete
- Add completed items for 2026-02-08
- Update last review date
MILESTONES.md:
- Update last updated date
- Mark v0.3.3 as Released (2026-02-08)
- Add comprehensive release notes with all features
- Update version history table
- Update current release to v0.3.3 with new features listed
ROADMAP.md:
- Update to reflect v0.3.3 current status
- Add settings management features to current status
- Update last updated date
Other files:
- app/config.py: Update default version to 0.3.3-dev
- docs/BuildMetadata.md: Update default version reference
- ANALYSIS_SUMMARY.md: Update current version
All documentation now reflects v0.3.3 release with complete feature list.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Introduced new authentication settings in config.py including `auth_enabled`, `admin_username`, `admin_password`, and `session_secret`.
- Added validation for `session_secret` to ensure it meets security requirements when authentication is enabled.
- Updated main.py to conditionally mount static files and log warnings if the directory is not found.
- Removed unused email template files and added new authentication and notification setup documentation.
- Implemented authentication configuration validation in validators.py and updated settings display.
- Enhanced the user interface with a new login template and SVG assets for branding.
- Added comprehensive guides for setting up authentication and notifications in the documentation.
- Added a new license route to serve the LGPL license text.
- Introduced a new attribution page to acknowledge third-party software used in the project.
- Updated the base HTML template to include a link to the attribution page.
- Included the license router in the main application router.
- Added the license text file for LGPL to the static licenses directory.
- Updated the NOTICE file to include detailed attributions for third-party libraries.
- Added a new requirements-dev.txt for license compliance checking.
- Updated the requirements.txt to clarify the LGPL license for Paramiko.
- Implemented `google_drive_callback.html` for processing Google Drive authorization, including UI for success and error states.
- Added JavaScript functionality for exchanging authorization codes, saving settings, and handling folder selection.
- Created `google_drive_callback_error.html` to display error messages during the authorization process.
- Added a script to save the build date during Docker image build.
- Introduced a build_date property in the Settings class to retrieve the build date from the environment or file.
- Enhanced the homepage to display system statistics including processed files and active integrations.
- Updated the about page with a comprehensive list of features.
- Added new privacy, imprint, cookies, and terms pages with relevant content.
- Improved the license page with related information links.
- Refactored the base template for better mobile responsiveness and footer links.
Christian Krakau-Louis