diff --git a/.github/workflows/codeql.yaml b/.github/workflows/codeql.yaml deleted file mode 100644 index a979d86c..00000000 --- a/.github/workflows/codeql.yaml +++ /dev/null @@ -1,41 +0,0 @@ -name: "CodeQL Security Scanning" - -on: - push: - branches: [ "main", "develop" ] - pull_request: - branches: [ "main", "develop" ] - schedule: - - cron: '0 0 * * 1' # Run every Monday at midnight - -jobs: - analyze: - name: Analyze - runs-on: ubuntu-latest - permissions: - actions: read - contents: read - security-events: write - - strategy: - fail-fast: false - matrix: - language: [ 'python', 'javascript' ] - - steps: - - name: Checkout repository - uses: actions/checkout@v3 - - - name: Initialize CodeQL - uses: github/codeql-action/init@v3 - with: - languages: ${{ matrix.language }} - queries: security-and-quality - - - name: Autobuild - uses: github/codeql-action/autobuild@v3 - - - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@v3 - with: - category: "/language:${{matrix.language}}" diff --git a/SECURITY_AUDIT.md b/SECURITY_AUDIT.md index 0db64b18..8b74a97c 100644 --- a/SECURITY_AUDIT.md +++ b/SECURITY_AUDIT.md @@ -238,7 +238,6 @@ ftp = ftplib.FTP() # nosec B321 - Plaintext FTP intentional when configured 2. **Penetration testing** - Professional security assessment 3. **Bug bounty program** - Community security contributions 4. **API key rotation** - Automated credential rotation -5. **Consolidate CodeQL workflows** - Two workflows (codeql.yaml and codeql.yml) - consider keeping only one ## Security Contact diff --git a/build-setup.sh b/build-setup.sh deleted file mode 100644 index e69de29b..00000000 diff --git a/docker-build.sh b/docker-build.sh deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/archive/ANALYSIS_SUMMARY.md b/docs/archive/ANALYSIS_SUMMARY.md index 2ab0530e..e4512bae 100644 --- a/docs/archive/ANALYSIS_SUMMARY.md +++ b/docs/archive/ANALYSIS_SUMMARY.md @@ -101,7 +101,7 @@ tests/ - ✅ Added Bandit security scanning - ✅ Improved linting configuration (line length: 120) -**New codeql.yaml:** +**New codeql.yml:** - ✅ Security scanning for Python and JavaScript - ✅ Scheduled weekly scans - ✅ Runs on PRs and main branch pushes diff --git a/issues/2026-02-08-fix-failing-tests-in-ci-workflow.md b/issues/2026-02-08-fix-failing-tests-in-ci-workflow.md deleted file mode 100644 index 684b5772..00000000 --- a/issues/2026-02-08-fix-failing-tests-in-ci-workflow.md +++ /dev/null @@ -1,59 +0,0 @@ -# Fix failing tests in CI workflow - -## Problems Addressed: - -### 1. Services Not Running (Redis, RabbitMQ) -Many test failures are caused by connection errors to Redis (localhost:6379) and AMQP/RabbitMQ (127.0.0.1:5672) with `ConnectionRefusedError: [Errno 111] Connection refused`. - -**Solution:** Add service containers to `.github/workflows/tests.yaml`: -```yaml -services: - redis: - image: redis:7 - ports: - - 6379:6379 - rabbitmq: - image: rabbitmq:3-management - ports: - - 5672:5672 - - 15672:15672 -``` - -### 2. Jinja2 Template Error: 'file' is undefined -Error in `file_detail.html` template. The view rendering this template is missing the 'file' variable in the context when handling nonexistent files. - -**Solution:** Ensure the template response includes the file variable or handle the error case differently in the view. - -### 3. SQLAlchemy IntegrityError: NOT NULL constraint failed: files.local_filename -In `tests/test_bulk_operations.py::TestBulkOperations::test_bulk_reprocess_missing_files`, the test tries to insert a file record with `local_filename=None`, but the database schema requires this field. - -**Solution:** Fix test data to provide a value for local_filename or make the field nullable in the schema. - -### 4. API Response Type Mismatch -In `tests/test_api.py::TestFileEndpoints::test_list_files_empty`, the test expects a list but receives a dictionary with 'files' and 'pagination' keys. - -**Solution:** Update either the test expectations or the API response format to be consistent. - -### 5. Subtask Retry Error Message -In `tests/test_file_detail_endpoints.py::TestSubtaskRetry::test_retry_subtask_invalid_task_name`, the test expects error message 'invalid subtask name' but gets 'processed file not found. cannot retry upload.'. - -**Solution:** Fix the error handling logic to return the expected error message. - -### 6. File Reprocessing 500 Error -In `tests/test_file_detail_endpoints.py::TestFileReprocessing::test_reprocess_existing_file`, the endpoint returns 500 Internal Server Error instead of 200. - -**Solution:** Investigate the cause of this server error and fix the underlying issue. - -## Failed Tests: -- `tests/test_api.py::TestFileEndpoints::test_list_files_empty` -- `tests/test_bulk_operations.py::TestBulkOperations::test_bulk_reprocess_missing_files` -- `tests/test_file_detail_endpoints.py::TestFileReprocessing::test_reprocess_existing_file` -- `tests/test_file_detail_endpoints.py::TestSubtaskRetry::test_retry_subtask_invalid_task_name` -- `tests/test_file_detail_endpoints.py::TestFileDetailView::test_file_detail_view_nonexistent` -- All tests in `test_file_upload.py` (9 tests failing due to RabbitMQ connection) - -## CI Run: -https://github.com/christianlouis/DocuElevate/actions/runs/21794817705/job/62880458485 - -## Commit: -`93149d0c56e0e7373ad92d59fd8c575eb9095914` \ No newline at end of file diff --git a/issues/2026-02-08-fix-failing-tests.md b/issues/2026-02-08-fix-failing-tests.md deleted file mode 100644 index 923a7dcf..00000000 --- a/issues/2026-02-08-fix-failing-tests.md +++ /dev/null @@ -1,41 +0,0 @@ -# Fix failing tests in CI workflow - -## Problems Addressed: - -1. **Services Not Running (Redis, RabbitMQ)** - Many test failures are caused by connection errors to Redis (localhost:6379) and AMQP/RabbitMQ (127.0.0.1:5672) with ConnectionRefusedError: [Errno 111] Connection refused. Need to add service containers to .github/workflows/tests.yaml: - ```yaml - services: - redis: - image: redis:7 - ports: - - 6379:6379 - rabbitmq: - image: rabbitmq:3-management - ports: - - 5672:5672 - - 15672:15672 - ``` - -2. **Jinja2 Template Error: 'file' is undefined** - Error in file_detail.html template. The view rendering this template is missing the 'file' variable in the context when handling nonexistent files. Need to ensure the template response includes the file variable or handle the error case differently. - -3. **SQLAlchemy IntegrityError: NOT NULL constraint failed: files.local_filename** - In tests/test_bulk_operations.py::TestBulkOperations::test_bulk_reprocess_missing_files, the test tries to insert a file record with local_filename=None, but the database schema requires this field. Need to fix test data or make the field nullable. - -4. **API Response Type Mismatch** - In tests/test_api.py::TestFileEndpoints::test_list_files_empty, the test expects a list but receives a dictionary with 'files' and 'pagination' keys. Need to update either the test expectations or the API response format to be consistent. - -5. **Subtask Retry Error Message** - In tests/test_file_detail_endpoints.py::TestSubtaskRetry::test_retry_subtask_invalid_task_name, the test expects error message 'invalid subtask name' but gets 'processed file not found. cannot retry upload.'. Need to fix the error handling logic. - -6. **File Reprocessing 500 Error** - In tests/test_file_detail_endpoints.py::TestFileReprocessing::test_reprocess_existing_file, the endpoint returns 500 Internal Server Error instead of 200. Need to investigate the cause of this server error. - -## Failed Tests: -- tests/test_api.py::TestFileEndpoints::test_list_files_empty -- tests/test_bulk_operations.py::TestBulkOperations::test_bulk_reprocess_missing_files -- tests/test_file_detail_endpoints.py::TestFileReprocessing::test_reprocess_existing_file -- tests/test_file_detail_endpoints.py::TestSubtaskRetry::test_retry_subtask_invalid_task_name -- tests/test_file_detail_endpoints.py::TestFileDetailView::test_file_detail_view_nonexistent -- All tests in test_file_upload.py (9 tests failing due to RabbitMQ connection) - -## CI Run: -[https://github.com/christianlouis/DocuElevate/actions/runs/21794817705/job/62880458485](https://github.com/christianlouis/DocuElevate/actions/runs/21794817705/job/62880458485) - -## Commit: -`93149d0c56e0e7373ad92d59fd8c575eb9095914` \ No newline at end of file diff --git a/issues/issue_1.md b/issues/issue_1.md deleted file mode 100644 index b72c99dc..00000000 --- a/issues/issue_1.md +++ /dev/null @@ -1,56 +0,0 @@ -## Problem Description - -7 tests in `tests/test_file_upload.py` are failing in the CI pipeline due to Celery task mocking issues. The tests expect the `delay` method to be called once, but it's being called 0 times. - -### Failing Tests -- `test_upload_valid_text_file` -- `test_upload_valid_image_jpeg` -- `test_upload_valid_png_image` -- `test_upload_office_document_docx` -- `test_upload_csv_file` -- `test_upload_executable_file` -- `test_image_by_extension` - -### Error Message -``` -AssertionError: Expected 'delay' to have been called once. Called 0 times. -``` - -### Root Cause Analysis - -The test fixture `mock_celery_tasks` is patching the Celery tasks at: -- `app.api.files.process_document.delay` -- `app.api.files.convert_to_pdf.delay` - -However, looking at the actual implementation in `app/api/files.py` (lines 656, 662, 669, 674), the tasks are invoked using `.delay()` method correctly: -- Line 656: `task = process_document.delay(target_path, original_filename=safe_filename)` -- Line 662: `task = convert_to_pdf.delay(target_path, original_filename=safe_filename)` -- Line 669: `task = convert_to_pdf.delay(target_path, original_filename=safe_filename)` -- Line 674: `task = convert_to_pdf.delay(target_path, original_filename=safe_filename)` - -The issue is that the patches are applied correctly but the mocks are not being invoked. This suggests: -1. The patch path might need adjustment -2. The tasks might be imported/accessed differently at runtime -3. There might be an issue with how the tasks are being called in the test environment - -### Files to Fix -- `tests/test_file_upload.py` - Fix the mock patching in the `mock_celery_tasks` fixture (lines 20-36) - -### Expected Behavior -All tests should pass with 100% success rate. The mocked Celery tasks should be called appropriately when files are uploaded through the `/api/ui-upload` endpoint. - -### Solution Approach -1. Verify the correct patch path for the Celery tasks -2. Ensure the patch is applied before the import of the tasks -3. Consider patching at the module level where tasks are defined rather than where they're used -4. Alternative: Patch the task objects directly: `app.tasks.process_document.process_document` and `app.tasks.convert_to_pdf.convert_to_pdf` - -### Reference -- Failing job: https://github.com/christianlouis/DocuElevate/actions/runs/21801196852/job/62896908441 -- Commit: 17795620c62c0bcf744eb7e940a3262c364f9bed - -### Acceptance Criteria -- All 7 failing tests pass -- Total test suite passes with 180/180 tests passing (currently 173/180) -- CI pipeline succeeds -- No regression in other tests \ No newline at end of file