Commit Graph

604 Commits

Author SHA1 Message Date
Christian Krakau-Louis 3e541ea655 Merge pull request #733 from christianlouis/copilot/debug-login-functionality
fix(ui): replace emoji flags with cross-platform CSS flag sprites
2026-03-16 15:29:49 +01:00
Christian Krakau-Louis ebbbd3d62e Merge pull request #731 from christianlouis/copilot/fix-mobile-app-login
fix: resolve CI failures from stale request mocks, Annotated+Depends pattern, and CodeQL sensitive data logging
2026-03-16 15:29:15 +01:00
copilot-swe-agent[bot] 4619e6b8a6 fix(ui): replace emoji flags with cross-platform flag-icons CSS library
Windows does not render regional indicator emoji as graphical flags,
showing raw text (DE, GB) instead. Replace all emoji flag values in
SUPPORTED_LANGUAGES with lowercase ISO 3166-1 alpha-2 country codes,
add the flag-icons@7.3.2 CSS library via CDN, and update templates to
render <span class="fi fi-{code}"> instead of emoji text.

Special cases:
- Welsh (cy): uses flag-icons region code "gb-wls"
- Catalan (ca): falls back to "es" (no dedicated ISO flag)
- Esperanto (eo): uses "un" (UN flag for international language)

<option> elements in profile.html no longer display flags since CSS
icon classes cannot be applied inside native <option> tags.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 14:25:28 +00:00
Christian Krakau-Louis c81914f78d Merge pull request #729 from christianlouis/copilot/add-default-language-version-support
feat(translation): automatic document translation to configurable default language
2026-03-16 13:55:25 +01:00
copilot-swe-agent[bot] 0c10fcbafc fix: register translate task in celery_worker and add setting metadata
- Import translate_to_default_language in celery_worker.py
- Add default_document_language to SETTING_METADATA in settings_service.py

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 12:32:03 +00:00
copilot-swe-agent[bot] 7bd667e271 fix(security): clarify billing log message wording
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 12:23:07 +00:00
copilot-swe-agent[bot] 075a505085 feat(profile): expose default_document_language in profile API and UI
- Add default_document_language to ProfileResponse and ProfileUpdateRequest
- Handle validation in PATCH /api/profile endpoint
- Add dropdown in profile.html template with Alpine.js binding
- Add translation keys for profile UI labels
- Add comprehensive tests for profile default language feature

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 12:13:54 +00:00
copilot-swe-agent[bot] b7a3b301a3 fix(security): remove clear-text logging of sensitive data and fix Annotated+Depends pattern
- audit_logs.py: Remove _db_dep pattern that fails with latest FastAPI on
  Python 3.11. Use clean DbSession = Annotated[Session, Depends(get_db)]
  without default values.
- billing.py: Remove owner_id from log messages to fix CodeQL clear-text
  logging of sensitive information alerts.
- files.py: Remove owner_id from log messages to fix CodeQL clear-text
  logging of sensitive information alerts.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 12:07:04 +00:00
copilot-swe-agent[bot] 8cc817930d Merge remote-tracking branch 'origin/main' into copilot/fix-mobile-app-login 2026-03-16 11:58:14 +00:00
copilot-swe-agent[bot] 3a221a62cd feat(translation): add model, config, task, API, and UI for default document language translation
- Add detected_language, default_language_text, default_language_code columns to FileRecord
- Add default_document_language column to UserProfile
- Add DEFAULT_DOCUMENT_LANGUAGE config setting (defaults to "en")
- Create translate_to_default_language Celery task
- Integrate translation trigger into embed_metadata_into_pdf pipeline
- Add /api/files/{id}/translate and /api/files/{id}/translation/default API endpoints
- Add /files/{id}/text/default-language view endpoint
- Update file_view.html with translation sections (default language, on-the-fly)
- Add translation keys to en.json
- Create Alembic migration 036
- Update .env.demo

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 11:54:13 +00:00
copilot-swe-agent[bot] f2255f9a1c docs: improve docstring and comment clarity in extract_metadata_from_file
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 11:29:53 +00:00
copilot-swe-agent[bot] 9d6bfde288 feat(tasks): extract and map embedded PDF metadata in upload_to_email
Implement extraction of embedded PDF metadata using pypdf, mapping
/Title, /Author, /Subject, /Keywords to filename, absender,
document_type, and tags respectively. JSON metadata takes precedence;
PDF metadata fills missing fields only.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 11:27:34 +00:00
copilot-swe-agent[bot] 9a256741d5 fix(auth): address review comments - improve debug logging, use SimpleNamespace, fix session cleanup
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 11:26:52 +00:00
copilot-swe-agent[bot] 15a9ed9435 fix(auth): resolve Bearer token in get_current_owner_id for mobile API auth
get_current_owner_id() only checked the session for authenticated users.
When the mobile app sends Authorization: Bearer <token>, there is no
session cookie, so the Depends(_get_owner_id) dependency raised HTTP 401
before the @require_login wrapper could resolve the Bearer token.

The function now checks three sources in order:
1. Session user dict (existing behavior)
2. request.state.api_token_user (cached by require_login or prior call)
3. Direct Bearer token resolution via _resolve_bearer_user (new)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 11:25:02 +00:00
Christian Krakau-Louis 3447a408db Merge pull request #714 from christianlouis/perf/async-onedrive-token-refresh-7215258063318287629
 Async OneDrive token validation using httpx
2026-03-16 12:15:39 +01:00
Christian Krakau-Louis 2a8a4b7471 Merge branch 'main' into code-health/simplify-ui-upload-12009112042086709661 2026-03-16 12:14:11 +01:00
Christian Krakau-Louis b8a1ac52b3 Merge branch 'main' into perf/async-url-upload-7099409897484162483 2026-03-16 12:12:21 +01:00
Christian Krakau-Louis e10f0bff42 Merge branch 'main' into refactor/audit-logs-annotated-b008-6104828608171783200 2026-03-16 12:10:32 +01:00
copilot-swe-agent[bot] c7ff177e17 fix: resolve test failures and mypy errors on main
- Fix Dropbox tests: patch httpx.AsyncClient instead of non-existent requests.post
- Add SETTING_METADATA entries for 6 logging settings (log_level, log_format, log_syslog_*)
- Add types-aiofiles to dev dependencies to fix mypy import-untyped error

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 10:32:20 +00:00
google-labs-jules[bot] 8ad90d7da9 style: resolve conflicts and use Annotated pattern in audit_logs.py
- Resolves merge conflicts with main.
- Implements Annotated pattern for FastAPI dependencies and query parameters.
- Maintains compatibility with decorators by using module-level dependency singletons.
- Fixes Ruff B008 issues.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 10:01:19 +00:00
Christian Krakau-Louis 740d18555b Merge pull request #707 from christianlouis/copilot/fix-debug-logging-issue
feat: fix DEBUG logging and add LOG_LEVEL/LOG_FORMAT/LOG_SYSLOG for standard log management
2026-03-16 11:00:41 +01:00
Christian Krakau-Louis bcd49aa793 Merge pull request #695 from christianlouis/fix/rename-save-dropbox-settings-9203732237520055590
🧹 [code health improvement] Rename save_dropbox_settings to save_google_drive_settings
2026-03-16 10:58:07 +01:00
Christian Krakau-Louis 3be78708cd Merge pull request #716 from christianlouis/code-health/simplify-save-onedrive-settings-12338634662690804221
🧹 [Code Health] Simplify save_onedrive_settings logic and extract env_utils
2026-03-16 10:56:25 +01:00
Christian Krakau-Louis 992adad978 Merge pull request #718 from christianlouis/perf-upload-15584535240168516797
 Optimize file upload endpoint to use asynchronous aiofiles
2026-03-16 10:55:57 +01:00
copilot-swe-agent[bot] df1fa51800 refactor(main): move JSON formatter imports to module level per code review
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:55:30 +00:00
google-labs-jules[bot] 7b21a69ceb perf: optimize url upload with async i/o
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.

Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.

Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by deriving the extension from original_filename and filtering out all non-alphanumerics.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:53:47 +00:00
Christian Krakau-Louis b3d1824d66 Merge pull request #700 from christianlouis/extract-pdf-metadata-upload-to-email-12238364102013658937
Extract embedded PDF metadata using pypdf
2026-03-16 10:52:51 +01:00
Christian Krakau-Louis 00be8d7b8e Merge pull request #703 from christianlouis/perf/fix-n-plus-1-shared-links-7028382104319809262
 fix N+1 query in list_shared_links
2026-03-16 10:52:34 +01:00
Christian Krakau-Louis 6e2e54aac6 Merge pull request #705 from christianlouis/perf/fix-n-plus-1-duplicates-17979853535459179799
 Fix N+1 Query in Duplicate Grouping List
2026-03-16 10:51:59 +01:00
copilot-swe-agent[bot] 6da1e6fd81 feat(config): add JSON structured logging and syslog forwarding for application logs
- Add LOG_FORMAT setting (text/json) for structured JSON log output
- Add LOG_SYSLOG_* settings for direct syslog forwarding of app logs
- JSON format compatible with Grafana Loki, Splunk, ELK, Datadog
- Syslog forwarding uses Python's SysLogHandler (UDP/TCP)
- Update .env.demo and ConfigurationGuide.md with all new settings
- Add tests for JSON formatter and syslog config fields

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:51:16 +00:00
Christian Krakau-Louis 056292dbe6 Merge pull request #710 from christianlouis/jules/perf-opt-reorder-plans-5035457001465312501
 Optimize reorder_plans by fixing N+1 database query
2026-03-16 10:51:02 +01:00
Christian Krakau-Louis c00a35bbac Merge pull request #711 from christianlouis/performance-optimize-dropbox-token-2748375428782384195
 Optimize Dropbox token check by using async httpx
2026-03-16 10:49:34 +01:00
google-labs-jules[bot] 30718218cc perf: optimize url upload with async i/o
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.

Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.

Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by leveraging `os.path.basename` around the generated target file paths, and filtering out non-alphanumerics from the file extension.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:49:01 +00:00
Christian Krakau-Louis 5b3c7da644 Merge pull request #719 from christianlouis/jules-fix-database-index-quoting-4036216779366574643
🔒 fix: explicitly quote identifiers in CREATE INDEX for saved_searches
2026-03-16 10:48:47 +01:00
Christian Krakau-Louis 1d51266208 Merge pull request #693 from christianlouis/fix-rclone-command-injection-16912117938106238295
🔒 fix(tasks): prevent command injection in rclone commands
2026-03-16 10:46:40 +01:00
Christian Krakau-Louis 3088459c70 Merge pull request #713 from christianlouis/perf/fix-notifications-api-n-plus-1-17980566457111073264
 Fix N+1 Query in Notification Preferences Update
2026-03-16 10:46:12 +01:00
google-labs-jules[bot] b8db664c2e perf: optimize url upload with async i/o
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.

Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.

Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by leveraging `os.path.basename` around the generated target file paths.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:45:06 +00:00
google-labs-jules[bot] fffb7cf357 Fix ruff linting errors resulting from aiofiles addition
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:44:59 +00:00
google-labs-jules[bot] 320a2acedd perf: optimize url upload with async i/o
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.

Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:41:17 +00:00
github-actions[bot] b4118f6162 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-16 09:37:40 +00:00
google-labs-jules[bot] 71d2d4100e 🔒 Prevent SQL injection by explicitly quoting identifier in CREATE INDEX
While `_ensure_indexes` was already secured, the `CREATE INDEX` for
`ix_saved_searches_user_id` was hardcoded. This commit explicitly
quotes it to unify our security posture against SQL injection
and keep static analyzers happy.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:37:10 +00:00
github-actions[bot] 80a0ddcbfc style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-16 09:36:54 +00:00
google-labs-jules[bot] b290cffb98 Performance Optimization: Replace synchronous file upload read with async aiofiles
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:36:33 +00:00
google-labs-jules[bot] c941738644 Refactor save_onedrive_settings and test_onedrive_token to use shared env utility
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:34:10 +00:00
copilot-swe-agent[bot] 18c49c6b2d feat(config): add LOG_LEVEL setting and configure root logging at startup
- Add `log_level` setting to config.py (default: INFO, env: LOG_LEVEL)
- Configure Python root logger in main.py with standard precedence:
  LOG_LEVEL explicit > DEBUG=true implies DEBUG > default INFO
- Add timestamp to log format for production readability
- Suppress noisy third-party loggers at DEBUG level
- Add comprehensive debug logging to all auth functions
- Add LOG_LEVEL/DEBUG to .env.demo and ConfigurationGuide.md
- Add tests for logging config and auth debug output

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:33:50 +00:00
github-actions[bot] ca2d023d81 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-16 09:25:47 +00:00
google-labs-jules[bot] 7242f3c168 perf(onedrive): use async httpx for token refresh
Replaces the synchronous `requests.post` and `requests.get` calls in
`app/api/onedrive.py:test_onedrive_token` with an asynchronous
`httpx.AsyncClient` implementation. This unblocks the FastAPI event loop
when this endpoint is hit.

Tests were updated to mock `httpx.AsyncClient` and a sync wrapper using `asyncio.run` was added to integration tests to maintain test coverage without massive test refactoring.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:24:58 +00:00
google-labs-jules[bot] fe20e02f78 perf(api): fix n+1 query issue in user notification preferences update
- Added a benchmark script in tests/test_notifications_api.py that proved the N+1 issue issue.
- Replaced iterative DB lookups inside `for item in body.preferences:` with single pre-fetch query and local `prefs_dict` lookups.
- Verified test benchmark time drops from ~0.0964s to ~0.0141s for a batch of 100 items.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:20:41 +00:00
google-labs-jules[bot] 84c6e1c5dd perf: optimize dropbox token refresh by replacing blocking requests with httpx
Replaced the synchronous `requests.post` calls in `app/api/dropbox.py` with asynchronous `httpx.AsyncClient().post` calls. This ensures that the FastAPI event loop is not blocked during network I/O, allowing better concurrent performance.

Also updated the `test_api_dropbox.py` tests to use `httpx.AsyncClient.post` in mocks and properly construct `httpx.RequestError` in exception handling tests.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:19:55 +00:00
google-labs-jules[bot] d8372c6fb8 perf(api): optimize reorder_plans to prevent N+1 queries
Replaced the loop over `body.order` which generated an N+1 issue
with a single bulk query fetching all relevant `SubscriptionPlan`
records via the `.in_()` clause.

Added an in-memory dictionary map of `plan_id` to `SubscriptionPlan`
objects to allow `O(1)` lookups while updating the order.

Benchmark speedup: 14.71x faster on 500 records.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:19:50 +00:00