Commit Graph

240 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] ae675485af Merge remote-tracking branch 'origin/main' into copilot/fix-watch-folder-settings
# Conflicts:
#	app/views/dropbox.py
#	frontend/templates/dropbox.html
#	frontend/templates/integrations_dashboard.html
#	tests/test_api_dropbox.py
#	tests/test_views_dropbox.py
2026-03-20 23:33:58 +00:00
copilot-swe-agent[bot] 1e1e6e6280 test: add tests for folder browser APIs and system credentials toggle
Added tests for POST /api/dropbox/list-folders (7 tests) and
POST /api/onedrive/list-folders (7 tests) covering success, subfolder
navigation, empty directories, auth errors, API errors, path
normalization, and alphabetical sorting.

Added view tests for system credentials toggle visibility in Dropbox
setup wizard.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 13:40:51 +00:00
copilot-swe-agent[bot] a8423064ec feat(api): add folder browser API endpoints and UI for Dropbox and OneDrive
Added POST /api/dropbox/list-folders and POST /api/onedrive/list-folders
endpoints that accept an OAuth access_token and return folder listings.

After successful OAuth authorization in the callback pages, users now
see an interactive folder browser to select the target folder for their
integration. The selected folder is saved to the integration config.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 13:21:52 +00:00
Christian Krakau-Louis 91eecd9396 Merge pull request #753 from christianlouis/copilot/scale-worker-and-api-pods
feat(scaling): enable horizontal scaling for API and worker pods
2026-03-20 13:55:31 +01:00
Christian Krakau-Louis b5fce418fb Merge pull request #773 from christianlouis/copilot/update-deprecated-npm-packages
fix(mobile): wire i18n reactivity, translate all screens, sync language preference with server
2026-03-20 12:15:10 +01:00
copilot-swe-agent[bot] 3b5ca04ebc fix(mobile): wire i18n reactivity, translate all screens, sync language with server
- Add LocaleProvider + useLocale() hook with AsyncStorage persistence to mobile i18n
- Replace all hardcoded English strings in every screen with t() calls
- Add missing profile.settings/language keys to all 5 translation files (en/de/es/fr/it)
- Wrap app root in LocaleProvider; apply server preferred_language on login in AuthGuard
- Tab labels and header titles now re-render on language switch
- ProfileScreen: use useLocale() context, sync language to server via POST /api/i18n/language
- Backend: add preferred_language field to GET /api/mobile/whoami response
- Mobile API: add preferred_language to WhoAmIResponse type + setServerLanguage() method
- Tests: add test_whoami_returns_preferred_language and test_whoami_no_profile_preferred_language_is_null
- Docs: update MobileApp.md with language sync priority and whoami response format

Language priority: server preference > AsyncStorage > device locale > English fallback

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 09:08:08 +00:00
copilot-swe-agent[bot] 5e3e2b1999 fix(dropbox): fix Invalid redirect_uri error by adding PUBLIC_BASE_URL config and URL-encoding
- Add PUBLIC_BASE_URL optional config to override auto-detected OAuth redirect URIs
  when behind a reverse proxy that doesn't forward X-Forwarded-Proto headers
- Add _build_dropbox_redirect_uri() helper in app/api/dropbox.py
- URL-encode redirect_uri in server-side Dropbox authorization URL
- Add _get_dropbox_callback_url() helper in app/views/dropbox.py
- Pass callback_url to both setup and callback templates
- Update templates to use server-provided callback_url instead of window.location.origin
- Update settings_service.py to register new setting
- Update .env.demo, ConfigurationGuide.md, and DropboxSetup.md documentation
- Add tests for new helper functions and global-authorize-url endpoint

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 08:19:48 +00:00
Christian Krakau-Louis 60e3ea030a Merge pull request #768 from christianlouis/copilot/check-imprint-privacy-policy
feat(mobile): add pre-login legal pages, fix image sharing, multi-image selection, file details, search, settings, and i18n
2026-03-20 08:48:34 +01:00
github-actions[bot] 910fb297ba style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-19 20:22:20 +00:00
copilot-swe-agent[bot] d1f9819f4e feat(integrations): add Dropbox connection test and global-credential sharing
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-19 19:17:12 +00:00
copilot-swe-agent[bot] 67c17e7baa feat(mobile): add pre-login legal pages, multi-image selection, file detail view, search, i18n, HEIC support
- Add Privacy Policy, Terms of Service, and Imprint links to WelcomeScreen
  and LoginScreen for GDPR/Apple compliance (pre-login access)
- Enable multiple image selection in photo library picker
- Add HEIC/HEIF image support to backend (allowed_types, convert_to_pdf, upload handler)
- Create FileDetailScreen with processing status and logs
- Add search bar to FilesScreen with debounced search
- Set up i18n with expo-localization (EN, DE, ES, FR, IT)
- Add language selector to ProfileScreen settings
- Add Imprint link to ProfileScreen legal section
- Update docs and tests

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-19 19:11:44 +00:00
copilot-swe-agent[bot] d5c18ccf07 fix(upload): reject exact duplicates at upload time and prevent duplicate mobile uploads
- Move duplicate check before task enqueue in ui_upload endpoint
- Clean up temp file and return status "duplicate" for exact duplicates
- Add URI-level dedup guard in mobile UploadScreen to prevent repeated uploads
- Improve ShareContext URI normalization (collapse slashes, decode percent-encoding)
- Guard +not-found.tsx effect against re-firing for the same pathname
- Update mobile UploadResponse type and handlers for duplicate status
- Update web frontend upload.js to show duplicate status
- Update API and Configuration docs

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-19 14:07:43 +00:00
copilot-swe-agent[bot] 2941f6e177 fix: resolve merge conflict in database.py
Merge origin/main into branch, resolving conflict in app/database.py.
Combined improvements from both branches:
- Keep pool_pre_ping=True and structured variable approach from feature branch
- Add explicit QueuePool import and poolclass assignment from main
2026-03-19 10:40:43 +00:00
copilot-swe-agent[bot] e4749b4e7c feat(api): allow disabled tokens/devices to be deleted & reactivated; add token lifetime
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-18 23:43:16 +00:00
Christian Krakau-Louis 2192783737 Merge pull request #758 from christianlouis/fix-idor-vuln-8344028020677453820
🛡️ Sentinel: [CRITICAL] Fix authorization bypass in API (IDOR)
2026-03-18 23:17:58 +01:00
copilot-swe-agent[bot] a8eb6504ac fix(qr-login): render QR code server-side using segno instead of CDN JS library
The QR code on /qr-login was not rendering because it depended on loading
qrcode@1.5.4 from the jsdelivr CDN, which may be blocked in some network
environments.

- Add segno>=1.6.0 (pure-Python QR library, no Pillow needed) to requirements.txt
- Generate QR code as a base64 SVG data URI server-side in the challenge endpoint
- Add qr_code_svg field to CreateChallengeResponse Pydantic model
- Replace canvas+CDN script in qr_login.html with an <img :src="qrCodeSvg">
- Remove the $nextTick/QRCode.toCanvas() client-side rendering block
- Extract QR rendering parameters (_QR_ERROR_LEVEL, _QR_SCALE) as module constants

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-18 08:34:22 +00:00
google-labs-jules[bot] f6a2bae05e Fix authorization bypass in API (IDOR) by applying owner filter
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-18 03:44:12 +00:00
copilot-swe-agent[bot] 571cc81789 feat(api): add per-user health-aware upload rate limiting
Introduces a Redis-backed sliding-window rate limiter for upload
endpoints (/api/ui-upload, /api/process-url) that:
- Enforces per-user limits (default: 20 uploads / 60 s)
- Dynamically reduces limits under system stress (queue depth, CPU load)
- Returns 429 with Retry-After header when exceeded
- Fails open when Redis is unavailable
- Works with the existing client-side adaptive back-off

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 12:12:50 +00:00
copilot-swe-agent[bot] 0f6a1ee1ec fix(api): add ttl_seconds to QR challenge response and fix client-side countdown
The QR login page countdown timer compared the server's UTC expiration
timestamp against the client's local clock, causing the QR code to appear
immediately expired when the client clock was ahead of the server.

Changes:
- Add ttl_seconds field to CreateChallengeResponse (seconds until expiry)
- Frontend countdown now uses relative elapsed time since response was
  received, eliminating clock-skew issues
- Mobile app: replace alert-only QR button with actual camera-based
  QR code scanner using expo-camera
- Add QRScannerScreen with barcode scanning, permission handling, and
  scan area overlay
- Update camera permission description to mention QR code scanning
- Add tests for ttl_seconds computation

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 12:02:33 +00:00
Christian Krakau-Louis 8905031d16 Merge branch 'main' into copilot/implement-log-off-everywhere-functionality 2026-03-17 12:12:50 +01:00
copilot-swe-agent[bot] 03712cfb08 feat(ui): add dedicated Devices page for mobile tokens and registered devices
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 11:00:36 +00:00
copilot-swe-agent[bot] f75b125992 feat(scaling): enable horizontal scaling for API and worker pods
- Add unauthenticated /api/diagnostic/healthz/live and /healthz/ready
  probe endpoints for Kubernetes liveness/readiness checks
- Separate Celery Beat into dedicated beat service in docker-compose.yaml
- Remove container_name from api and worker services to allow scaling
- Create Helm beat-deployment.yaml for standalone Beat scheduler pod
- Remove -B flag from worker-deployment.yaml so workers can scale safely
- Add beat section and fix probe paths in Helm values.yaml
- Add tests for the new probe endpoints

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 23:52:56 +00:00
copilot-swe-agent[bot] a4588a57cb feat(auth): add server-side session management and QR code login backend
- Add UserSession and QRLoginChallenge models for session tracking
  and mobile QR authentication
- Add session_manager utility with create/validate/revoke/cleanup
  functions and QR challenge helpers
- Add /api/sessions endpoints for listing, revoking, and
  'log off everywhere' functionality
- Add /api/qr-auth endpoints for challenge creation, polling, and
  claiming with API token issuance
- Add session config fields (lifetime, custom override, QR TTL)
- Update get_current_user to validate server-side sessions
- Create server-side sessions on all login paths (local, OAuth,
  social, admin)
- Revoke server-side session on logout
- Configure SessionMiddleware max_age from session lifetime settings
- Graceful degradation: old sessions without _session_token continue
  to work

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-16 22:29:04 +00:00
copilot-swe-agent[bot] a88d790445 feat(system-reset): add system reset and factory reset feature
- Add FACTORY_RESET_ON_STARTUP and ENABLE_FACTORY_RESET config settings
- Create app/utils/system_reset.py with core reset logic (wipe DB + files, reimport)
- Create app/api/system_reset.py with admin-only API endpoints
- Create app/views/system_reset.py with admin-only UI view
- Create frontend/templates/system_reset.html with confirmation dialogs
- Auto-reset on startup when FACTORY_RESET_ON_STARTUP=true
- Re-import uses watch folder mechanism for re-ingestion
- Register routers in API and views init files
- Add i18n keys and SETTING_METADATA entries
- Add nav links in base.html (desktop + mobile)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:26:17 +00:00
copilot-swe-agent[bot] 825f3cc3a8 fix(translation): fix apply_owner_filter called with wrong arguments
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 15:25:29 +00:00
Christian Krakau-Louis ebbbd3d62e Merge pull request #731 from christianlouis/copilot/fix-mobile-app-login
fix: resolve CI failures from stale request mocks, Annotated+Depends pattern, and CodeQL sensitive data logging
2026-03-16 15:29:15 +01:00
Christian Krakau-Louis c81914f78d Merge pull request #729 from christianlouis/copilot/add-default-language-version-support
feat(translation): automatic document translation to configurable default language
2026-03-16 13:55:25 +01:00
copilot-swe-agent[bot] 7bd667e271 fix(security): clarify billing log message wording
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 12:23:07 +00:00
copilot-swe-agent[bot] 075a505085 feat(profile): expose default_document_language in profile API and UI
- Add default_document_language to ProfileResponse and ProfileUpdateRequest
- Handle validation in PATCH /api/profile endpoint
- Add dropdown in profile.html template with Alpine.js binding
- Add translation keys for profile UI labels
- Add comprehensive tests for profile default language feature

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 12:13:54 +00:00
copilot-swe-agent[bot] b7a3b301a3 fix(security): remove clear-text logging of sensitive data and fix Annotated+Depends pattern
- audit_logs.py: Remove _db_dep pattern that fails with latest FastAPI on
  Python 3.11. Use clean DbSession = Annotated[Session, Depends(get_db)]
  without default values.
- billing.py: Remove owner_id from log messages to fix CodeQL clear-text
  logging of sensitive information alerts.
- files.py: Remove owner_id from log messages to fix CodeQL clear-text
  logging of sensitive information alerts.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 12:07:04 +00:00
copilot-swe-agent[bot] 3a221a62cd feat(translation): add model, config, task, API, and UI for default document language translation
- Add detected_language, default_language_text, default_language_code columns to FileRecord
- Add default_document_language column to UserProfile
- Add DEFAULT_DOCUMENT_LANGUAGE config setting (defaults to "en")
- Create translate_to_default_language Celery task
- Integrate translation trigger into embed_metadata_into_pdf pipeline
- Add /api/files/{id}/translate and /api/files/{id}/translation/default API endpoints
- Add /files/{id}/text/default-language view endpoint
- Update file_view.html with translation sections (default language, on-the-fly)
- Add translation keys to en.json
- Create Alembic migration 036
- Update .env.demo

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 11:54:13 +00:00
Christian Krakau-Louis 3447a408db Merge pull request #714 from christianlouis/perf/async-onedrive-token-refresh-7215258063318287629
 Async OneDrive token validation using httpx
2026-03-16 12:15:39 +01:00
Christian Krakau-Louis 2a8a4b7471 Merge branch 'main' into code-health/simplify-ui-upload-12009112042086709661 2026-03-16 12:14:11 +01:00
Christian Krakau-Louis b8a1ac52b3 Merge branch 'main' into perf/async-url-upload-7099409897484162483 2026-03-16 12:12:21 +01:00
Christian Krakau-Louis e10f0bff42 Merge branch 'main' into refactor/audit-logs-annotated-b008-6104828608171783200 2026-03-16 12:10:32 +01:00
google-labs-jules[bot] 8ad90d7da9 style: resolve conflicts and use Annotated pattern in audit_logs.py
- Resolves merge conflicts with main.
- Implements Annotated pattern for FastAPI dependencies and query parameters.
- Maintains compatibility with decorators by using module-level dependency singletons.
- Fixes Ruff B008 issues.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 10:01:19 +00:00
Christian Krakau-Louis bcd49aa793 Merge pull request #695 from christianlouis/fix/rename-save-dropbox-settings-9203732237520055590
🧹 [code health improvement] Rename save_dropbox_settings to save_google_drive_settings
2026-03-16 10:58:07 +01:00
Christian Krakau-Louis 3be78708cd Merge pull request #716 from christianlouis/code-health/simplify-save-onedrive-settings-12338634662690804221
🧹 [Code Health] Simplify save_onedrive_settings logic and extract env_utils
2026-03-16 10:56:25 +01:00
Christian Krakau-Louis 992adad978 Merge pull request #718 from christianlouis/perf-upload-15584535240168516797
 Optimize file upload endpoint to use asynchronous aiofiles
2026-03-16 10:55:57 +01:00
google-labs-jules[bot] 7b21a69ceb perf: optimize url upload with async i/o
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.

Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.

Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by deriving the extension from original_filename and filtering out all non-alphanumerics.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:53:47 +00:00
Christian Krakau-Louis 00be8d7b8e Merge pull request #703 from christianlouis/perf/fix-n-plus-1-shared-links-7028382104319809262
 fix N+1 query in list_shared_links
2026-03-16 10:52:34 +01:00
Christian Krakau-Louis 6e2e54aac6 Merge pull request #705 from christianlouis/perf/fix-n-plus-1-duplicates-17979853535459179799
 Fix N+1 Query in Duplicate Grouping List
2026-03-16 10:51:59 +01:00
Christian Krakau-Louis 056292dbe6 Merge pull request #710 from christianlouis/jules/perf-opt-reorder-plans-5035457001465312501
 Optimize reorder_plans by fixing N+1 database query
2026-03-16 10:51:02 +01:00
Christian Krakau-Louis c00a35bbac Merge pull request #711 from christianlouis/performance-optimize-dropbox-token-2748375428782384195
 Optimize Dropbox token check by using async httpx
2026-03-16 10:49:34 +01:00
google-labs-jules[bot] 30718218cc perf: optimize url upload with async i/o
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.

Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.

Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by leveraging `os.path.basename` around the generated target file paths, and filtering out non-alphanumerics from the file extension.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:49:01 +00:00
google-labs-jules[bot] b8db664c2e perf: optimize url upload with async i/o
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.

Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.

Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by leveraging `os.path.basename` around the generated target file paths.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:45:06 +00:00
google-labs-jules[bot] fffb7cf357 Fix ruff linting errors resulting from aiofiles addition
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:44:59 +00:00
google-labs-jules[bot] 320a2acedd perf: optimize url upload with async i/o
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.

Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:41:17 +00:00
github-actions[bot] 80a0ddcbfc style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-16 09:36:54 +00:00
google-labs-jules[bot] b290cffb98 Performance Optimization: Replace synchronous file upload read with async aiofiles
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:36:33 +00:00