Commit Graph

40 Commits

Author SHA1 Message Date
google-labs-jules[bot] d22175310a 🛡️ Sentinel: [HIGH] Fix Server-Side Request Forgery in IMAP connections
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
 Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-23 14:45:22 +00:00
copilot-swe-agent[bot] f852ba9783 refactor(views): split file views into summary, detail, process, and annotations pages
- /files/<id> → new summary page with navigation cards
- /files/<id>/detail → document detail with metadata, preview, text
- /files/<id>/process → processing pipeline status and history
- /files/<id>/annotations → comments & annotations with EmbedPDF viewer
- /files/<id>/comments → redirects to /annotations
- Added embed-pdf-viewer as git submodule for PDF annotation viewer
- Updated all navigation links across templates
- Updated all tests to use new URL structure

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/12276514-bd3d-4e3e-84d9-5977d1f82b19
2026-03-22 11:48:57 +00:00
copilot-swe-agent[bot] 20e61db050 merge: resolve conflicts with main branch (keep both SharePoint + iCloud integrations)
Merge origin/main into copilot/add-sharepoint-integration.
All four conflicts were resolved by keeping both the SharePoint
additions (from this branch) and the iCloud additions (from main):
- app/models.py: added both SHAREPOINT and ICLOUD to IntegrationType
- app/tasks/send_to_all.py: added both to service_map and services list
- app/tasks/upload_to_user_integration.py: kept both upload handlers
- frontend/templates/files.html: added both filter options

No database migration conflicts — SharePoint does not require schema changes.
2026-03-16 22:25:51 +00:00
copilot-swe-agent[bot] afb168d3e8 feat(i18n): localize remaining image zoom controls in files view
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 18:15:35 +00:00
copilot-swe-agent[bot] 45c2cd7c6e feat(i18n): localize files view - replace all hardcoded strings with translation keys
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 18:05:59 +00:00
Christian Krakau-Louis dd7f7dfd97 Merge branch 'main' into copilot/refactor-template-localization 2026-03-12 12:44:33 +01:00
copilot-swe-agent[bot] 8c77982109 feat(i18n): replace hardcoded English text with translation keys in files, settings, and pipelines templates
Replace all hardcoded English strings in three HTML Jinja2 templates with
`{{ _("key") }}` translation function calls, adding 87 new keys to en.json.

files.html:
- Page title, drop overlay, upload modal, error messages
- All filter labels, placeholders, options (status, OCR quality, etc.)
- Saved searches and full-text search UI
- Bulk action buttons, table headers, pagination
- Delete and preview modal text

settings.html:
- Page heading, configuration priority legend
- Header buttons (Wizard, DB Wizard, Export, Audit Log)
- Search bar placeholder and aria-labels
- Sidebar categories, no-results state
- Per-setting labels: required, enable prefix, effective value
- Autocomplete and model picker hints/placeholders
- Revert/save button labels and states

pipelines.html:
- Page title and subtitle
- New Pipeline button, loading state, empty state
- Pipeline card badges (System, Default, Inactive, Disabled)
- Add/Edit step modal fields and labels
- OCR language options and hints
- Cancel/Delete buttons in all modals

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-12 00:17:23 +00:00
copilot-swe-agent[bot] 30f06e0b32 feat(storage): add Apple iCloud Drive storage provider
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 23:38:35 +00:00
copilot-swe-agent[bot] b85fc1d277 feat(storage): add SharePoint integration for document storage
Add Microsoft SharePoint Online as a storage provider using the
Microsoft Graph API. Includes OAuth2 authentication via MSAL,
site/drive resolution, chunked upload sessions, and metadata sync.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-10 00:17:42 +00:00
copilot-swe-agent[bot] af34ce88df feat(api): implement proper pagination for file list API
- Change default per_page from 50 to 25
- Rename total_items → total, total_pages → pages in pagination response
- Add next/previous URL fields to pagination response
- Update view and template to use new field names
- Update tests and API docs

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 17:31:56 +00:00
Christian Krakau-Louis b65d937a36 Merge pull request #439 from christianlouis/copilot/add-bulk-file-operations
feat: bulk operations (delete, reprocess, Cloud OCR, ZIP download) + persisted OCR quality score filter
2026-03-01 14:58:01 +01:00
copilot-swe-agent[bot] 83c3405c98 feat: persist ocr_quality_score and use it for numeric filtering
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:46:43 +00:00
copilot-swe-agent[bot] 705b970522 feat: add bulk download, cloud OCR, and basic OCR quality filter
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:38:42 +00:00
copilot-swe-agent[bot] 81c08ed9d0 fix: append new drops to upload queue instead of overwriting it
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 13:26:49 +00:00
copilot-swe-agent[bot] d5884f6d2c fix(api): address code review feedback - logging, wildcard escaping, UX
- Add exception logging in saved search error handlers
- Escape SQL LIKE wildcards (%, _) in tags filter to prevent unintended matching
- Improve UI alert message for empty filter save attempt

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 11:32:34 +00:00
copilot-swe-agent[bot] e5a4c6c64a feat(api): add advanced filtering and saved searches
- Add date range (date_from/date_to), storage provider, and tags filters to GET /api/files
- Add SavedSearch model and migration (005_add_saved_searches)
- Add CRUD API endpoints for saved searches at /api/saved-searches
- Update files.html template with new filter controls and saved searches UI
- Update files view to pass new filter parameters to template
- Add comprehensive tests for all new functionality (26 tests)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 11:22:07 +00:00
copilot-swe-agent[bot] c814af1145 fix(ui): preview modal falls back to original version when processed unavailable
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 07:48:44 +00:00
copilot-swe-agent[bot] 372e8cdff1 feat(ui): add pdf.js viewer, image zoom/pan, text preview, and preview modal
- Replace iframe in file_view.html with pdf.js for PDF files
- Add image viewer with zoom/pan controls to file_view.html
- Add text file viewer with line numbers to file_view.html
- Add preview side-panel modal to files.html (file list)
- Fix file_detail.html bottom preview section variable names
- Add aria-labels and WCAG compliance to all new elements

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 07:32:30 +00:00
copilot-swe-agent[bot] bf6bbe3512 feat(ui): fix accessibility in upload, search, files, and login templates
- upload.html: Add role/tabindex/aria-label to drop zone, aria-live to status messages, aria-label to file inputs
- search.html: Add role="search", sr-only label, aria-live for results, input type="search"
- files.html: Add table aria-label, scope="col" to headers, aria-sort on sortable columns, aria-labels to action buttons, dialog roles to modals, aria-live to status areas, pagination nav with aria-labels
- login.html: Wrap form in main landmark, aria-hidden on decorative icons

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-28 15:59:12 +00:00
copilot-swe-agent[bot] c7c5718f78 feat(queue): add queue monitoring dashboard and pending banner on files page
- Add /api/queue/stats endpoint with Redis queue lengths, Celery worker
  inspection, and DB processing summaries
- Add /api/queue/pending-count lightweight endpoint for the files page banner
- Add /admin/queue admin-only view with auto-refreshing queue dashboard
- Add queue pending banner on /files page showing queued/processing count
- Add Queue Monitor link to admin dropdown in navigation (desktop + mobile)
- Add comprehensive tests for all new endpoints and views

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-27 00:42:55 +00:00
copilot-swe-agent[bot] a05690bd5f feat(upload): adaptive 429 backoff, full Gotenberg file types, directory traversal
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-26 23:24:26 +00:00
copilot-swe-agent[bot] 09a6337c6f fix(ui): prevent search input from clearing on each keystroke and add dedicated /search page
The debounceSearch() function in files.html called clearFullTextSearch()
when the query was shorter than 2 characters. Since clearFullTextSearch()
sets input.value = '', every single keystroke was immediately erased —
users could paste text but not type.

Fix: debounceSearch now only hides the results panel for short queries
without touching the input value.

Also adds a dedicated /search page with Google-style results showing
content previews (document title, filename, type badges, tag badges,
sender, and OCR text snippets with highlighted matches).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-26 09:18:28 +00:00
copilot-swe-agent[bot] ff96340154 fix(config): default meilisearch_url to http://meilisearch:7700 for Docker/K8s service discovery
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-25 13:27:09 +00:00
copilot-swe-agent[bot] d08e175352 feat(ui): implement responsive mobile interface
- Add `relative` to nav for correct mobile dropdown positioning
- Increase hamburger button and mobile nav links to ≥44px touch targets
- Add mobile card view on files page (table hidden on small screens)
- Make bulk actions bar and pagination responsive/wrapping
- Add file type `accept` attribute and camera capture button on upload page
- Increase all interactive button/input sizes to ≥44px
- Add responsive CSS for pagination wrap and filter stacking at 480px"

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-23 19:09:49 +00:00
copilot-swe-agent[bot] b03ebab747 chore: apply ruff formatting and fix whitespace issues
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-13 09:12:11 +00:00
Christian Krakau-Louis 9a2c2d20b1 feat(duplication): implement duplicate file handling and detection across processing steps 2026-02-12 02:38:00 +01:00
copilot-swe-agent[bot] ed0bac7345 feat(ui): add auto-refresh after upload and inline file preview
- Auto-refresh files table after successful uploads using custom event
- Add inline preview support for PDFs, images, and text files
- Set Content-Disposition header to inline for preview endpoint
- Add download button as secondary action in file details view

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-08 16:12:44 +00:00
copilot-swe-agent[bot] a29af88dd4 Update documentation and version for drag-and-drop feature
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-08 06:10:04 +00:00
copilot-swe-agent[bot] a7128cecd2 Add drag-and-drop upload functionality to Files view
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-08 06:05:35 +00:00
copilot-swe-agent[bot] 74724973d8 Fix API routes to return JSON instead of HTML on errors
- Modified exception handlers in app/main.py to check if request path starts with /api/
- HTTPException handler returns JSON for API routes, HTML for frontend routes
- General exception handler (500) also checks and returns appropriate format
- Enhanced frontend deleteFile() to handle non-JSON responses gracefully
- Added content-type checking before parsing JSON
- Added comprehensive tests for API error handling

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 19:18:42 +00:00
copilot-swe-agent[bot] 77678f6368 Fix single file deletion frontend to parse JSON response properly
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 17:53:12 +00:00
copilot-swe-agent[bot] 34ba8a4548 Improve error messages for bulk operations
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 15:55:44 +00:00
copilot-swe-agent[bot] 08775459aa Fix /files view issues and add bulk operations
- Fixed status filter in /files view endpoint
- Added bulk delete and reprocess API endpoints
- Added bulk selection UI with checkboxes
- Added bulk actions bar with reprocess and delete buttons
- Updated JavaScript to handle bulk operations

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 15:52:38 +00:00
copilot-swe-agent[bot] 775ffa5082 Address code review feedback
- Fixed status filtering to occur before pagination for correct counts
- Resolved N+1 query problem by batch-fetching processing statuses
- Extracted status computation logic to shared utility function
- Changed sort indicator from ⬍ to ↕ for better browser compatibility
- Updated both API and view layers to use shared status utilities

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 15:20:59 +00:00
copilot-swe-agent[bot] 530c63ad14 Add server-side pagination, filtering, sorting, and file detail view
- Updated /api/files endpoint with pagination, filtering, and sorting support
- Added /api/files/{file_id} endpoint for detailed file information
- Updated /files view to support server-side operations
- Added /files/{file_id}/detail route for file detail page
- Created new files.html with filters, status column, and pagination
- Created file_detail.html for viewing processing history
- Status computed from ProcessingLog entries (pending, processing, completed, failed)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 15:12:49 +00:00
copilot-swe-agent[bot] 1903dc5bcd Add comprehensive processing logging system
- Added database logging to all major processing tasks
- Created API endpoints for retrieving processing logs
- Updated frontend to display processing logs per file
- Logging includes: process_document, convert_to_pdf, extract_metadata_with_gpt, embed_metadata_into_pdf, finalize_document_storage, send_to_all_destinations

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-06 22:30:18 +00:00
Christian Krakau-Louis d23a48b1c0 feat: add frontend views and file management functionality, enhance user response structure, and update configuration guide 2025-04-03 21:49:15 +02:00
Christian Krakau-Louis 2cafda11dd feat: update version to 0.4.3-dev, add file deletion feature, and refactor API structure 2025-04-03 15:55:34 +02:00
Christian Krakau-Louis e4ffa457db added grid.js 2025-03-26 03:22:41 +01:00
Christian Krakau-Louis 48fce5d661 added the /files view and auth for api and files 2025-03-26 02:51:49 +01:00