flutter downgrade dependencies

New translations from
[Crowdin](https://crowdin.com/project/ente-photos-app)

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -1,24 +1,21 @@
 name: Report a bug
-description: Let us know if something's not working the way you expected.
+description: For regressions only (things that were working earlier)
 labels: []
 body:
     - type: markdown
       attributes:
           value: |
-              Before opening a new bug report, please ensure
-              1. you are on the latest version (it might've already been fixed),
-              2. you've searched for existing issues (please add your observations as a comment there instead of creating a duplicate).
-
-              If you are self hosting, please create a community [Q&A](https://github.com/ente-io/ente/discussions/categories/q-a) instead.
+              Before opening a new issue, **please** ensure
+              1. You are on the latest version,
+              2. You've searched for existing issues,
+              3. It was working earlier (otherwise use [this](https://github.com/ente-io/ente/discussions/categories/enhancements))
+              4. It is not about self hosting (otherwise use [this](https://github.com/ente-io/ente/discussions/categories/q-a))
     - type: textarea
       attributes:
           label: Description
           description: >
-              Please describe the bug. If possible, also include the steps to
-              reproduce the behaviour, and the expected behaviour (sometimes
-              bugs are just expectation mismatches, in which case this would be
-              a good fit for [feature
-              requests](https://github.com/ente-io/ente/discussions/categories/feature-requests)).
+              Describe the bug and steps to reproduce the behaviour, and how it
+              differs from the previously working behaviour.
       validations:
           required: true
     - type: input
@@ -26,6 +23,17 @@ body:
           label: Version
           description: The version can be seen at the bottom of settings.
           placeholder: e.g. v1.2.3
+    - type: input
+      attributes:
+          label: Last working version
+          description: >
+              The version where things were last known to be working. It is fine
+              if you don't remember the exact version (mention roughly then),
+              but **if there just isn't a last working version, then please file
+              it as an
+              [enhancement](https://github.com/ente-io/ente/discussions/categories/enhancements))**
+              (where the community upvotes can be used to help prioritize).
+          placeholder: e.g. v1.2.3
     - type: dropdown
       attributes:
           label: What product are you using?

.github/ISSUE_TEMPLATE/config.yml

@@ -1,5 +1,5 @@
 blank_issues_enabled: true
 contact_links:
-    - name: Feature requests and questions
+    - name: Enhacements, feature requests, feedback and questions
       url: https://github.com/ente-io/ente/discussions
       about: Please use Discussions for everything apart from the above.

.github/workflows/auth-crowdin-push.yml

@@ -24,8 +24,8 @@ jobs:
             - name: Crowdin's action
               uses: crowdin/github-action@v2
               with:
-                  base_path: "auth/"
-                  config: "auth/crowdin.yml"
+                  base_path: "mobile/apps/auth/"
+                  config: "mobile/apps/auth/crowdin.yml"
                   upload_sources: true
                   upload_translations: false
                   download_translations: false

.github/workflows/auth-crowdin-sync.yml

@@ -23,8 +23,8 @@ jobs:
             - name: Crowdin's action
               uses: crowdin/github-action@v2
               with:
-                  base_path: "auth/"
-                  config: "auth/crowdin.yml"
+                  base_path: "mobile/apps/auth/"
+                  config: "mobile/apps/auth/crowdin.yml"
                   upload_sources: true
                   upload_translations: false
                   download_translations: true

.github/workflows/auth-internal-release.yml

@@ -15,7 +15,7 @@ jobs:
 
         defaults:
             run:
-                working-directory: auth
+                working-directory: mobile/apps/auth
 
         steps:
             - name: Checkout code and submodules
@@ -55,7 +55,7 @@ jobs:
               with:
                   serviceAccountJsonPlainText: ${{ secrets.SERVICE_ACCOUNT_JSON }}
                   packageName: io.ente.auth
-                  releaseFiles: auth/build/app/outputs/bundle/playstoreRelease/app-playstore-release.aab
+                  releaseFiles: mobile/apps/auth/build/app/outputs/bundle/playstoreRelease/app-playstore-release.aab
                   track: internal
 
             - name: Notify Discord

.github/workflows/auth-lint.yml

@@ -18,7 +18,7 @@ jobs:
         runs-on: ubuntu-latest
         defaults:
             run:
-                working-directory: auth
+                working-directory: mobile/apps/auth
         steps:
             - name: Checkout code and submodules
               uses: actions/checkout@v4

.github/workflows/auth-release.yml

@@ -36,11 +36,11 @@ permissions:
 
 jobs:
     build-linux-latest:
-        runs-on: ubuntu-latest
+        runs-on: ubuntu-22.04
 
         defaults:
             run:
-                working-directory: auth
+                working-directory: mobile/apps/auth
 
         steps:
             - name: Checkout code and submodules
@@ -83,7 +83,7 @@ jobs:
               # disable this step if release tag contains nightly or beta
               if: startsWith(github.ref, 'refs/tags/auth-v') && !contains(github.ref, 'nightly') && !contains(github.ref, 'beta')
               run: |
-                  flutter build appbundle --release --flavor playstore --dart-define=app.flavor=playstore
+                  flutter build appbundle --release --flavor playstore --dart-define=app.flavor=playstore --dart-define=cronetHttpNoPlay=true
               env:
                   SIGNING_KEY_PATH: "/home/runner/work/_temp/keystore/ente_auth_key.jks"
                   SIGNING_KEY_ALIAS: ${{ secrets.SIGNING_KEY_ALIAS }}
@@ -93,7 +93,7 @@ jobs:
             - name: Install dependencies for desktop build
               run: |
                   sudo apt-get update -y
-                  sudo apt-get install -y libsecret-1-dev libsodium-dev libfuse2 ninja-build libgtk-3-dev dpkg-dev pkg-config rpm patchelf libsqlite3-dev locate libayatana-appindicator3-dev libffi-dev libtiff6 xz-utils libarchive-tools libcurl4-openssl-dev
+                  sudo apt-get install -y libsecret-1-dev libsodium-dev libfuse2 ninja-build libgtk-3-dev dpkg-dev pkg-config rpm patchelf libsqlite3-dev locate libayatana-appindicator3-dev libffi-dev libtiff5 xz-utils libarchive-tools libcurl4-openssl-dev
                   sudo updatedb --localpaths='/usr/lib/x86_64-linux-gnu'
 
             - name: Install appimagetool
@@ -117,12 +117,14 @@ jobs:
                   mv dist/**/*-*-linux.deb artifacts/ente-${{ github.ref_name }}-x86_64.deb
 
             - name: Generate checksums
-              run: sha256sum artifacts/ente-* >> artifacts/sha256sum-rpm-appimage
+              run: |
+                sha256sum artifacts/ente-auth-*.apk >> artifacts/sha256sum-apk
+                sha256sum artifacts/ente-auth-*.deb artifacts/ente-auth-*.rpm artifacts/ente-auth-*.AppImage >> artifacts/sha256sum-linux
 
             - name: Create a draft GitHub release
               uses: ncipollo/release-action@v1
               with:
-                  artifacts: "auth/artifacts/*"
+                  artifacts: "mobile/apps/auth/artifacts/*"
                   draft: true
                   allowUpdates: true
                   updateOnlyUnreleased: true
@@ -134,15 +136,16 @@ jobs:
               with:
                   serviceAccountJsonPlainText: ${{ secrets.SERVICE_ACCOUNT_JSON }}
                   packageName: io.ente.auth
-                  releaseFiles: auth/build/app/outputs/bundle/playstoreRelease/app-playstore-release.aab
+                  releaseFiles: mobile/apps/auth/build/app/outputs/bundle/playstoreRelease/app-playstore-release.aab
                   track: internal
 
     build-windows:
         runs-on: windows-latest
+        environment: "auth-win-build"
 
         defaults:
             run:
-                working-directory: auth
+                working-directory: mobile/apps/auth
 
         steps:
             - name: Checkout code and submodules
@@ -172,14 +175,22 @@ jobs:
             - name: Retain Windows EXE and DLLs
               run: cp -r build/windows/x64/runner/Release ente-${{ github.ref_name }}-windows
 
-            - name: Code sign Windows installer and EXE
-              uses: dlemstra/code-sign-action@v1
+            - name: Sign files with Trusted Signing
+              uses: azure/trusted-signing-action@v0
               with:
-                  certificate: "${{ secrets.WINDOWS_CERTIFICATE }}"
-                  password: "${{ secrets.WINDOWS_CERTIFICATE_PASSWORD }}"
-                  files: |
-                      auth/artifacts/ente-${{ github.ref_name }}-installer.exe
-                      auth/ente-${{ github.ref_name }}-windows/auth.exe
+                azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
+                azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
+                azure-client-secret: ${{ secrets.AZURE_CLIENT_SECRET }}
+                endpoint: ${{ secrets.AZURE_ENDPOINT }}
+                trusted-signing-account-name: ${{ secrets.AZURE_CODE_SIGNING_NAME }}
+                certificate-profile-name: ${{ secrets.AZURE_CERT_PROFILE_NAME }}
+                files: |
+                      ${{ github.workspace }}/mobile/apps/auth/artifacts/ente-${{ github.ref_name }}-installer.exe
+                      ${{ github.workspace }}/mobile/apps/auth/ente-${{ github.ref_name }}-windows/auth.exe
+                file-digest: SHA256
+                timestamp-rfc3161: http://timestamp.acs.microsoft.com
+                timestamp-digest: SHA256  
+
 
             - name: Zip Windows EXE and DLLs
               run: tar.exe -a -c -f artifacts/ente-${{ github.ref_name }}-windows.zip ente-${{ github.ref_name }}-windows
@@ -190,7 +201,7 @@ jobs:
             - name: Create a draft GitHub release
               uses: ncipollo/release-action@v1
               with:
-                  artifacts: "auth/artifacts/*"
+                  artifacts: "mobile/apps/auth/artifacts/*"
                   draft: true
                   allowUpdates: true
                   updateOnlyUnreleased: true
@@ -200,7 +211,7 @@ jobs:
 
         defaults:
             run:
-                working-directory: auth
+                working-directory: mobile/apps/auth
 
         steps:
             - name: Checkout code and submodules
@@ -287,7 +298,7 @@ jobs:
             - name: Create a draft GitHub release
               uses: ncipollo/release-action@v1
               with:
-                  artifacts: "auth/artifacts/*"
+                  artifacts: "mobile/apps/auth/artifacts/*"
                   draft: true
                   allowUpdates: true
                   updateOnlyUnreleased: true

.github/workflows/auth-win-sign.yml

@@ -0,0 +1,70 @@
+name: "Windows build & Sign (auth)"
+
+
+on:
+    workflow_dispatch: # Allow manually running the action
+
+env:
+    FLUTTER_VERSION: "3.24.3"
+
+permissions:
+    contents: write
+
+jobs:
+    build-windows:
+        runs-on: windows-latest
+        environment: "auth-win-build"
+
+        defaults:
+            run:
+                working-directory: mobile/apps/auth
+
+        steps:
+            - name: Checkout code and submodules
+              uses: actions/checkout@v4
+              with:
+                  submodules: recursive
+
+            - name: Install Flutter ${{ env.FLUTTER_VERSION  }}
+              uses: subosito/flutter-action@v2
+              with:
+                  channel: "stable"
+                  flutter-version: ${{ env.FLUTTER_VERSION  }}
+                  cache: true
+
+            - name: Create artifacts directory
+              run: mkdir artifacts
+
+            - name: Build Windows installer
+              run: |
+                  flutter config --enable-windows-desktop
+                  # dart pub global activate flutter_distributor
+                  dart pub global activate --source git https://github.com/ente-io/flutter_distributor_fork --git-ref develop --git-path packages/flutter_distributor
+                  make innoinstall
+                  flutter_distributor package --platform=windows --targets=exe --skip-clean
+                  mv dist/**/*-windows-setup.exe artifacts/ente-${{ github.ref_name }}-installer.exe
+
+            - name: Retain Windows EXE and DLLs
+              run: cp -r build/windows/x64/runner/Release ente-${{ github.ref_name }}-windows
+
+            - name: Sign files with Trusted Signing
+              uses: azure/trusted-signing-action@v0
+              with:
+                azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
+                azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
+                azure-client-secret: ${{ secrets.AZURE_CLIENT_SECRET }}
+                endpoint: ${{ secrets.AZURE_ENDPOINT }}
+                trusted-signing-account-name: ${{ secrets.AZURE_CODE_SIGNING_NAME }}
+                certificate-profile-name: ${{ secrets.AZURE_CERT_PROFILE_NAME }}
+                files: |
+                      ${{ github.workspace }}/mobile/apps/auth/artifacts/ente-${{ github.ref_name }}-installer.exe
+                      ${{ github.workspace }}/mobile/apps/auth/ente-${{ github.ref_name }}-windows/auth.exe
+                file-digest: SHA256
+                timestamp-rfc3161: http://timestamp.acs.microsoft.com
+                timestamp-digest: SHA256  
+
+            - name: Zip Windows EXE and DLLs
+              run: tar.exe -a -c -f artifacts/ente-${{ github.ref_name }}-windows.zip ente-${{ github.ref_name }}-windows
+
+            - name: Generate checksums
+              run: sha256sum artifacts/ente-* > artifacts/sha256sum-windows

.github/workflows/docs-lint.yml

@@ -0,0 +1,32 @@
+name: "Lint (docs)"
+
+on:
+    # Run on every pull request (open or push to it) that changes docs/
+    pull_request:
+        paths:
+            - "docs/**"
+            - ".github/workflows/docs-lint.yml"
+
+permissions:
+    contents: read
+
+jobs:
+    lint:
+        runs-on: ubuntu-latest
+        defaults:
+            run:
+                working-directory: docs
+        steps:
+            - name: Checkout code
+              uses: actions/checkout@v4
+
+            - name: Setup node and enable yarn caching
+              uses: actions/setup-node@v4
+              with:
+                  node-version: 22
+                  cache: "yarn"
+                  cache-dependency-path: "web/yarn.lock"
+
+            - run: yarn install
+
+            - run: yarn pretty:check

.github/workflows/mobile-crowdin-push.yml

@@ -5,7 +5,7 @@ on:
         branches: [main]
         paths:
             # Run workflow when mobiles's intl_en.arb is changed
-            - "mobile/lib/l10n/intl_en.arb"
+            - "mobile/apps/photos/lib/l10n/intl_en.arb"
             # Or the workflow itself is changed
             - ".github/workflows/mobile-crowdin.yml"
 
@@ -24,8 +24,8 @@ jobs:
             - name: Crowdin's action
               uses: crowdin/github-action@v2
               with:
-                  base_path: "mobile/"
-                  config: "mobile/crowdin.yml"
+                  base_path: "mobile/apps/photos/"
+                  config: "mobile/apps/photos/crowdin.yml"
                   upload_sources: true
                   upload_translations: false
                   download_translations: false

.github/workflows/mobile-crowdin-sync.yml

@@ -1,4 +1,4 @@
-name: "Sync Crowdin translations (mobile)"
+name: "Sync Crowdin translations (mobile/photos)"
 
 on:
     schedule:
@@ -23,14 +23,14 @@ jobs:
             - name: Crowdin's action
               uses: crowdin/github-action@v2
               with:
-                  base_path: "mobile/"
-                  config: "mobile/crowdin.yml"
+                  base_path: "mobile/apps/photos/"
+                  config: "mobile/apps/photos/crowdin.yml"
                   upload_sources: true
                   upload_translations: false
                   download_translations: true
                   localization_branch_name: translations/mobile
                   create_pull_request: true
-                  pull_request_title: "[mobile] New translations"
+                  pull_request_title: "[mobile/photos] New translations"
                   pull_request_body: "New translations from [Crowdin](https://crowdin.com/project/ente-photos-app)"
                   pull_request_base_branch_name: "main"
                   project_id: 574741

.github/workflows/mobile-internal-release-rust.yml

@@ -0,0 +1,77 @@
+name: "Internal release (photos)"
+
+on:
+    workflow_dispatch: # Allow manually running the action
+
+env:
+    FLUTTER_VERSION: "3.24.3"
+    RUST_VERSION: "1.85.1"
+
+permissions:
+    contents: write
+
+jobs:
+    build:
+        runs-on: ubuntu-latest
+
+        defaults:
+            run:
+                working-directory: mobile/apps/photos
+
+        steps:
+            - name: Checkout code and submodules
+              uses: actions/checkout@v4
+              with:
+                  submodules: recursive
+
+            - name: Setup JDK 17
+              uses: actions/setup-java@v1
+              with:
+                  java-version: 17
+
+            - name: Install Flutter ${{ env.FLUTTER_VERSION  }}
+              uses: subosito/flutter-action@v2
+              with:
+                  channel: "stable"
+                  flutter-version: ${{ env.FLUTTER_VERSION  }}
+                  cache: true
+            
+            - name: Install Rust ${{ env.RUST_VERSION }}
+              uses: dtolnay/rust-toolchain@master
+              with:
+                toolchain: ${{ env.RUST_VERSION }}
+
+            - name: Install Flutter Rust Bridge
+              run: cargo install flutter_rust_bridge_codegen
+
+            - name: Setup keys
+              uses: timheuer/base64-to-file@v1
+              with:
+                  fileName: "keystore/ente_photos_key.jks"
+                  encodedString: ${{ secrets.SIGNING_KEY_PHOTOS }}
+
+            - name: Build PlayStore AAB
+              run: |
+                  flutter build appbundle --dart-define=cronetHttpNoPlay=true --release --flavor playstore
+              env:
+                  SIGNING_KEY_PATH: "/home/runner/work/_temp/keystore/ente_photos_key.jks"
+                  SIGNING_KEY_ALIAS: ${{ secrets.SIGNING_KEY_ALIAS_PHOTOS }}
+                  SIGNING_KEY_PASSWORD: ${{ secrets.SIGNING_KEY_PASSWORD_PHOTOS }}
+                  SIGNING_STORE_PASSWORD: ${{ secrets.SIGNING_STORE_PASSWORD_PHOTOS }}
+
+            - name: Upload AAB to PlayStore
+              uses: r0adkll/upload-google-play@v1
+              with:
+                  serviceAccountJsonPlainText: ${{ secrets.SERVICE_ACCOUNT_JSON }}
+                  packageName: io.ente.photos
+                  releaseFiles: mobile/apps/photos/build/app/outputs/bundle/playstoreRelease/app-playstore-release.aab
+                  track: internal
+
+            - name: Notify Discord
+              uses: sarisia/actions-status-discord@v1
+              with:
+                  webhook: ${{ secrets.DISCORD_INTERNAL_RELEASE_WEBHOOK }}
+                  nodetail: true
+                  title: "🏆 Internal release available for Photos"
+                  description: "[Download](https://play.google.com/store/apps/details?id=io.ente.photos)"
+                  color: 0x00ff00

.github/workflows/mobile-internal-release.yml

@@ -15,7 +15,7 @@ jobs:
 
         defaults:
             run:
-                working-directory: mobile
+                working-directory: mobile/apps/photos
 
         steps:
             - name: Checkout code and submodules
@@ -55,7 +55,7 @@ jobs:
               with:
                   serviceAccountJsonPlainText: ${{ secrets.SERVICE_ACCOUNT_JSON }}
                   packageName: io.ente.photos
-                  releaseFiles: mobile/build/app/outputs/bundle/playstoreRelease/app-playstore-release.aab
+                  releaseFiles: mobile/apps/photos/build/app/outputs/bundle/playstoreRelease/app-playstore-release.aab
                   track: internal
 
             - name: Notify Discord
@@ -63,6 +63,6 @@ jobs:
               with:
                   webhook: ${{ secrets.DISCORD_INTERNAL_RELEASE_WEBHOOK }}
                   nodetail: true
-                  title: "🏆 Internal release available for Photos"
+                  title: "🏆 Internal release Photos (Branch: ${{ github.ref_name }})"
                   description: "[Download](https://play.google.com/store/apps/details?id=io.ente.photos)"
                   color: 0x00ff00

.github/workflows/mobile-lint.yml

@@ -4,7 +4,7 @@ on:
     # Run on every pull request (open or push to it) that changes mobile/
     pull_request:
         paths:
-            - "mobile/**"
+            - "mobile/apps/photos/**"
             - ".github/workflows/mobile-lint.yml"
 
 env:
@@ -18,7 +18,7 @@ jobs:
         runs-on: ubuntu-latest
         defaults:
             run:
-                working-directory: mobile
+                working-directory: mobile/apps/photos
         steps:
             - name: Checkout code and submodules
               uses: actions/checkout@v4

.github/workflows/mobile-release.yml

@@ -20,7 +20,7 @@ jobs:
 
         defaults:
             run:
-                working-directory: mobile
+                working-directory: mobile/apps/photos
 
         steps:
             - name: Checkout code and submodules
@@ -62,5 +62,5 @@ jobs:
             - name: Create a draft GitHub release
               uses: ncipollo/release-action@v1
               with:
-                  artifacts: "mobile/build/app/outputs/flutter-apk/ente-${{ github.ref_name }}.apk,mobile/build/app/outputs/flutter-apk/sha256sum"
+                  artifacts: "mobile/apps/photos/build/app/outputs/flutter-apk/ente-${{ github.ref_name }}.apk,mobile/apps/photos/build/app/outputs/flutter-apk/sha256sum"
                   draft: true

.github/workflows/web-deploy.yml

@@ -93,3 +93,13 @@ jobs:
                   accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
                   apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
                   command: pages deploy --project-name=ente --commit-dirty=true --branch=deploy/payments web/apps/payments/dist
+
+            - name: Build locker
+              run: yarn build:locker
+
+            - name: Publish locker
+              uses: cloudflare/wrangler-action@v3
+              with:
+                  accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
+                  apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
+                  command: pages deploy --project-name=ente --commit-dirty=true --branch=deploy/locker web/apps/locker/out

.gitmodules

@@ -3,9 +3,9 @@
 	url = https://github.com/ente-io/sentry-dart.git
 	branch = sentry_flutter_ente
 [submodule "auth/flutter"]
-	path = auth/flutter
+	path = mobile/apps/auth/flutter
 	url = https://github.com/flutter/flutter.git
 	branch = stable
 [submodule "auth/assets/simple-icons"]
-	path = auth/assets/simple-icons
+	path = mobile/apps/auth/assets/simple-icons
 	url = https://github.com/simple-icons/simple-icons.git

CONTRIBUTING.md

@@ -23,7 +23,7 @@ Just hang around, enjoy the vibe. Answer someone's query on our
 [Discord](https://discord.gg/z2YVKkycX3), or pile on in the sporadic #off-topic
 rants there. Chuckle (or wince!) at our [Twitter](https://twitter.com/enteio)
 memes. Suggest a new feature in our [Github
-Discussions](https://github.com/ente-io/ente/discussions/new?category=feature-requests),
+Discussions](https://github.com/ente-io/ente/discussions/new?category=enhancements),
 or upvote the existing ones that you feel we should focus on first. Provide your
 opinion on existing threads.
 
@@ -68,8 +68,8 @@ best to start small. Consider some well-scoped changes, say like adding more
 Each of the individual product/platform specific directories in this repository
 have instructions on setting up a dev environment.
 
-For anything beyond trivial bug fixes, please use [features requests and
-discussions](https://github.com/ente-io/ente/discussions) instead of performing
+For anything beyond trivial bug fixes, please use
+[discussions](https://github.com/ente-io/ente/discussions) instead of performing
 code changes directly.
 
 > [!TIP]

auth/android/app/src/main/res/mipmap-anydpi-v26/launcher_icon.xml

@@ -1,6 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
-  <background android:drawable="@drawable/ic_launcher_background"/>
-  <foreground android:drawable="@drawable/ic_launcher_foreground"/>
-  <monochrome android:drawable="@drawable/ic_launcher_foreground"/>
-</adaptive-icon>

auth/assets/custom-icons/icons/bingx.svg

@@ -1 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?><svg id="a" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 150 150"><defs><style>.e{fill:#2a54ff;}.f{fill:url(#d);}.g{fill:none;}</style><linearGradient id="d" x1="17.68" y1="116.45" x2="132.14" y2="32.11" gradientTransform="matrix(1, 0, 0, 1, 0, 0)" gradientUnits="userSpaceOnUse"><stop offset="0" stop-color="#2a54ff"/><stop offset=".52" stop-color="#2143cb"/><stop offset="1" stop-color="#2a54ff"/></linearGradient></defs><g id="b"><path id="c" class="g" d="M0,0H150V150H0V0Z"/></g><path class="f" d="M140.2,22.33c-25.18-.09-49.79,10.83-66.63,29.47-6.06,6.27-10.1,13.95-14.96,21.06-11.64,15.93-29.81,25.14-49.5,25.13h0v28.65h0c25.17,.1,49.78-10.86,66.63-29.5,6.03-6.27,10.13-13.94,14.96-21.06,11.64-15.91,29.81-25.12,49.5-25.11V22.33h0Z"/><path class="e" d="M140.2,97.99c-19.68,0-37.86-9.2-49.5-25.11-4.81-7.12-8.92-14.78-14.94-21.06C58.95,33.18,34.3,22.24,9.13,22.35h0v28.65h0c21.8-.11,42.05,11.62,53.01,30.46,3.22,5.62,7.06,10.9,11.45,15.74,16.83,18.63,41.46,29.59,66.63,29.5l-.02-28.7h0Z"/></svg>

auth/assets/custom-icons/icons/coinspot.svg

@@ -1,130 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<!-- Generator: Adobe Illustrator 27.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
-	 viewBox="0 0 30 30" style="enable-background:new 0 0 30 30;" xml:space="preserve">
-<style type="text/css">
-	.st0{fill:#C5C8CA;}
-	.st1{fill:#9DA4A8;}
-	.st2{fill:#B7BBBD;}
-	.st3{fill:#CBCFD1;}
-	.st4{fill:#BBBFC2;}
-	.st5{fill:#CACDCE;}
-	.st6{fill:#BFC3C5;}
-	.st7{fill:#BCC0C2;}
-	.st8{fill:#BDC1C4;}
-	.st9{fill:#C7CACC;}
-	.st10{fill:url(#SVGID_1_);}
-	.st11{fill:#FFFFFF;}
-	.st12{fill:#B8BCBF;}
-	.st13{fill:#C4C7C9;}
-	.st14{fill:#C1C5C7;}
-	.st15{fill:url(#SVGID_00000003093454306001190100000011813141018663887528_);}
-	.st16{fill:url(#SVGID_00000017503418065689336600000007511615486600436881_);}
-	.st17{fill:url(#SVGID_00000057845154053127761930000017803385842445649033_);}
-	.st18{fill:url(#SVGID_00000156571711195124538550000006687723982713171592_);}
-	.st19{fill:#DF3030;}
-	.st20{fill:url(#SVGID_00000001636660173574603980000008731795684331757470_);}
-	.st21{fill:#17181C;}
-	.st22{fill:url(#SVGID_00000180343933242210086490000003762167186865041053_);}
-	.st23{fill:url(#SVGID_00000015338415700440354440000005681408021599925436_);}
-</style>
-<g>
-	<path class="st0" d="M14.4,29.5c0.1,0,0.1,0,0.2,0c0.1,0,0.2,0,0.2,0H14.4z"/>
-	<path class="st1" d="M15.3,29.5h0.1c0,0,0,0,0,0c0,0,0,0,0,0c0,0,0,0,0,0c0,0,0,0,0,0c0,0,0,0,0,0L15.3,29.5
-		C15.2,29.5,15.3,29.5,15.3,29.5z"/>
-	<path class="st2" d="M15.3,29.5L15.3,29.5l-0.2,0C15.2,29.5,15.2,29.5,15.3,29.5z"/>
-	<path class="st3" d="M15.5,29.5L15.5,29.5L15.5,29.5L15.5,29.5L15.5,29.5z"/>
-	<path class="st0" d="M14.1,29.5c0.1,0,0.1,0,0.2,0H14.1z"/>
-	<path class="st4" d="M13.9,29.5C13.9,29.5,14,29.5,13.9,29.5c0.1,0,0.1,0,0.2,0H13.9z"/>
-	<path class="st5" d="M13.6,29.5C13.6,29.5,13.6,29.5,13.6,29.5c0.1,0,0.1,0,0.1,0H13.6z"/>
-	<path class="st6" d="M13.7,29.5C13.8,29.5,13.8,29.5,13.7,29.5c0.1,0,0.1,0,0.1,0H13.7z"/>
-	<path class="st7" d="M13.3,29.4C13.3,29.4,13.3,29.4,13.3,29.4C13.4,29.4,13.4,29.4,13.3,29.4L13.3,29.4z"/>
-	<path class="st8" d="M13.4,29.5C13.4,29.4,13.5,29.4,13.4,29.5C13.5,29.4,13.5,29.4,13.4,29.5L13.4,29.5z"/>
-	<path class="st8" d="M13.1,29.4C13.1,29.4,13.1,29.4,13.1,29.4C13.1,29.4,13.1,29.4,13.1,29.4L13.1,29.4z"/>
-	<path class="st9" d="M13.2,29.4C13.2,29.4,13.2,29.4,13.2,29.4C13.2,29.4,13.2,29.4,13.2,29.4C13.2,29.4,13.2,29.4,13.2,29.4
-		C13.3,29.4,13.3,29.4,13.2,29.4L13.2,29.4z"/>
-	
-		<linearGradient id="SVGID_1_" gradientUnits="userSpaceOnUse" x1="21.8812" y1="-88.078" x2="8.2545" y2="-104.6955" gradientTransform="matrix(1 0 0 -1 0 -81.48)">
-		<stop  offset="0" style="stop-color:#020037"/>
-		<stop  offset="1" style="stop-color:#050F62"/>
-	</linearGradient>
-	<path class="st10" d="M15,0.4C11.1,0.4,7.5,2,4.7,4.7C2,7.4,0.5,11.1,0.5,15c0,1.7,0.3,3.4,0.9,5.1c0.3,0,0.5,0,0.8,0
-		c2.9,0,5.8,0.9,8.2,2.6c2.4,1.7,4.2,4.1,5.1,6.9c3.8-0.1,7.4-1.7,10-4.4c2.6-2.7,4.1-6.4,4.1-10.1c0-3.9-1.5-7.6-4.3-10.3
-		C22.6,2,18.9,0.4,15,0.4"/>
-	<path class="st11" d="M20.7,22.5C20.7,22.5,20.7,22.5,20.7,22.5L20.7,22.5c0,0.4,0.1,0.8,0.3,1c0.2,0.2,0.6,0.3,1,0.3c0,0,0,0,0,0
-		c0,0,0,0,0,0c0,0,0,0,0,0c0,0,0,0,0,0c0,0,0,0,0,0l0,0c-0.4,0-0.7,0.1-1,0.3c-0.2,0.2-0.3,0.6-0.3,1c0,0,0,0,0,0c0,0,0,0,0,0
-		c0,0,0,0,0,0c0,0,0,0,0,0h0l0,0c0-0.4-0.1-0.7-0.3-1c-0.2-0.2-0.6-0.3-1-0.3c0,0,0,0,0,0l0,0c0,0,0,0,0,0s0,0,0,0c0,0,0,0,0,0
-		c0,0,0,0,0,0c0.4,0,0.7-0.1,1-0.3S20.7,22.9,20.7,22.5C20.7,22.5,20.7,22.5,20.7,22.5C20.7,22.5,20.7,22.5,20.7,22.5z"/>
-	<path class="st11" d="M6.9,15.5C6.9,15.5,6.9,15.5,6.9,15.5L6.9,15.5c0,0.4,0.1,0.8,0.3,1c0.2,0.2,0.6,0.3,1,0.3c0,0,0,0,0,0
-		c0,0,0,0,0,0c0,0,0,0,0,0c0,0,0,0,0,0c0,0,0,0,0,0l0,0c-0.4,0-0.7,0.1-1,0.3c-0.2,0.2-0.3,0.6-0.3,1c0,0,0,0,0,0c0,0,0,0,0,0
-		c0,0,0,0,0,0c0,0,0,0,0,0h0l0,0c0-0.4-0.1-0.7-0.3-1c-0.2-0.2-0.6-0.3-1-0.3c0,0,0,0,0,0l0,0c0,0,0,0,0,0c0,0,0,0,0,0c0,0,0,0,0,0
-		c0,0,0,0,0,0c0.4,0,0.7-0.1,1-0.3C6.8,16.2,6.9,15.9,6.9,15.5C6.9,15.5,6.9,15.5,6.9,15.5C6.9,15.5,6.9,15.5,6.9,15.5z"/>
-	<path class="st11" d="M10.6,4.1L10.6,4.1C10.7,4.1,10.7,4.1,10.6,4.1c0,0.3,0.1,0.5,0.3,0.7c0.2,0.2,0.4,0.3,0.7,0.2h0v0l0,0l0,0
-		l0,0l0,0c-0.3,0-0.5,0.1-0.7,0.2c-0.2,0.2-0.3,0.4-0.2,0.7l0,0l0,0l0,0l0,0h0v0c0-0.3-0.1-0.5-0.2-0.7C10.2,5.1,10,5,9.7,5.1h0v0v0
-		h0C10,5,10.2,5,10.4,4.8C10.6,4.6,10.7,4.3,10.6,4.1C10.6,4.1,10.6,4.1,10.6,4.1C10.6,4.1,10.6,4.1,10.6,4.1z"/>
-	<path class="st12" d="M12.8,29.4C12.8,29.4,12.8,29.4,12.8,29.4C12.8,29.4,12.8,29.4,12.8,29.4C12.8,29.4,12.8,29.4,12.8,29.4
-		C12.8,29.4,12.8,29.4,12.8,29.4L12.8,29.4z"/>
-	<path class="st13" d="M13,29.4C13,29.4,13,29.4,13,29.4C13,29.4,13,29.4,13,29.4L13,29.4z"/>
-	<path class="st14" d="M12.9,29.4C12.9,29.4,12.9,29.4,12.9,29.4C12.9,29.4,12.9,29.4,12.9,29.4L12.9,29.4z"/>
-	
-		<linearGradient id="SVGID_00000173122186048074043340000017421439166240502921_" gradientUnits="userSpaceOnUse" x1="19.2457" y1="-89.3156" x2="22.9553" y2="-91.7188" gradientTransform="matrix(1 0 0 -1 0 -81.48)">
-		<stop  offset="0" style="stop-color:#E5E5E5"/>
-		<stop  offset="1" style="stop-color:#B7B8C1"/>
-	</linearGradient>
-	<path style="fill:url(#SVGID_00000173122186048074043340000017421439166240502921_);" d="M21.8,1.2c-1.4,0.7-3,1.9-4.4,4.2
-		c-2.5,3.9-3.2,7.4-3.2,7.4L16,14l0.3,0.2l1.9,1.2c0,0,2.9-2,5.4-5.9c1.5-2.3,2-4.3,2-5.8c-0.8-0.1-1.5-0.4-2.2-0.8
-		C22.8,2.5,22.2,1.9,21.8,1.2z"/>
-	
-		<linearGradient id="SVGID_00000127763695479642710240000017533313096818365313_" gradientUnits="userSpaceOnUse" x1="21.2378" y1="-99.9826" x2="19.0472" y2="-97.8815" gradientTransform="matrix(1 0 0 -1 0 -81.48)">
-		<stop  offset="0" style="stop-color:#EC4F4F"/>
-		<stop  offset="1" style="stop-color:#A91919"/>
-	</linearGradient>
-	<path style="fill:url(#SVGID_00000127763695479642710240000017533313096818365313_);" d="M20.8,16.8c0.9-1.4,0.3-3.2,0-3.8
-		c-0.7,0.8-1.5,1.5-2.3,2.1c0.1,0.4,0.3,0.8,0.3,1.2c0,0.1,0,0.2-0.1,0.3c-0.4,0.6-0.8,1.3-1.1,2c-0.1,0.1-0.1,0.2-0.1,0.3
-		c-0.1,0.2-0.1,0.3,0,0.5c0,0.3,0.2,0.5,0.3,0.8c0,0,0.1,0.1,0.1,0.1c0.1,0,0.1,0.1,0.2,0.1s0.1,0,0.2-0.1c0.1-0.1,0.3-0.2,0.4-0.4
-		C19.5,19,19.8,18.5,20.8,16.8z"/>
-	
-		<linearGradient id="SVGID_00000060717637781723915790000002744012061535479481_" gradientUnits="userSpaceOnUse" x1="11.3158" y1="-99.2586" x2="14.8122" y2="-101.5237" gradientTransform="matrix(1 0 0 -1 0 -81.48)">
-		<stop  offset="0" style="stop-color:#F2A518"/>
-		<stop  offset="1" style="stop-color:#F4E23E"/>
-	</linearGradient>
-	<path style="fill:url(#SVGID_00000060717637781723915790000002744012061535479481_);" d="M15.1,15.7l-1.7-1.1c-2,3.1-3.3,7-2.4,7.5
-		c0.9,0.6,3.9-2.2,5.9-5.3L15.1,15.7z"/>
-	
-		<linearGradient id="SVGID_00000070084874335106853820000008402293642909580433_" gradientUnits="userSpaceOnUse" x1="-4386.2534" y1="747.6443" x2="-4497.9517" y2="769.0099" gradientTransform="matrix(1 0 0 -1 0 -81.48)">
-		<stop  offset="0" style="stop-color:#EC4F4F"/>
-		<stop  offset="1" style="stop-color:#A91919"/>
-	</linearGradient>
-	<path style="fill:url(#SVGID_00000070084874335106853820000008402293642909580433_);" d="M15.2,9.5c-0.7-0.1-2.5,0.1-3.4,1.5
-		c-1.1,1.6-1.5,2.1-2,3.2c-0.1,0.2-0.1,0.3-0.2,0.5c0,0.1,0,0.1,0,0.2C9.6,15,9.7,15,9.7,15c0,0,0.1,0,0.2,0.1c0.3,0.1,0.6,0,0.8,0
-		c0.2,0,0.3-0.1,0.4-0.2c0.1-0.1,0.2-0.2,0.3-0.3c0.5-0.6,0.9-1.2,1.3-1.8c0.1-0.1,0.2-0.2,0.3-0.2c0.4-0.1,0.8-0.1,1.2-0.2l0,0
-		C14.5,11.4,14.8,10.4,15.2,9.5z"/>
-	<path class="st19" d="M25,0.6c-0.2-0.1-1.5-0.2-3.2,0.7c0.4,0.7,1,1.2,1.6,1.7c0.7,0.4,1.4,0.7,2.2,0.8C25.7,1.9,25.1,0.7,25,0.6z"
-		/>
-	<path class="st19" d="M18.4,15.5L14,12.7c-0.1,0-0.1,0-0.2,0l-0.9,1.4c0,0.1,0,0.1,0,0.2l4.4,2.8c0.1,0,0.1,0,0.2,0l0.9-1.4
-		C18.4,15.6,18.4,15.6,18.4,15.5z"/>
-	
-		<linearGradient id="SVGID_00000044894753735506851200000013592864944465274029_" gradientUnits="userSpaceOnUse" x1="14.9436" y1="-95.9217" x2="16.3716" y2="-96.8468" gradientTransform="matrix(1 0 0 -1 0 -81.48)">
-		<stop  offset="0" style="stop-color:#B71E1E"/>
-		<stop  offset="0.44" style="stop-color:#DF3030"/>
-		<stop  offset="1" style="stop-color:#C51D1D"/>
-	</linearGradient>
-	<path style="fill:url(#SVGID_00000044894753735506851200000013592864944465274029_);" d="M17.8,11.6c-0.4-0.2-2.1,1.6-3.2,3.3
-		c-0.8,1.2-1.4,3-1.1,3.2c0.4,0.2,1.7-1,2.5-2.3C17.1,14.2,18.1,11.9,17.8,11.6z"/>
-	<path class="st21" d="M21.2,8.6c1.3,0,2.3-1,2.3-2.3s-1-2.3-2.3-2.3c-1.3,0-2.3,1-2.3,2.3S20,8.6,21.2,8.6z"/>
-	
-		<linearGradient id="SVGID_00000090987122570624474440000002432161440392897685_" gradientUnits="userSpaceOnUse" x1="20.068" y1="-87.0655" x2="22.3556" y2="-88.5473" gradientTransform="matrix(1 0 0 -1 0 -81.48)">
-		<stop  offset="0" style="stop-color:#CED1EC"/>
-		<stop  offset="1" style="stop-color:#FFFFFF"/>
-	</linearGradient>
-	<path style="fill:url(#SVGID_00000090987122570624474440000002432161440392897685_);" d="M21.2,7.7c0.8,0,1.4-0.6,1.4-1.4
-		S22,5,21.2,5c-0.8,0-1.4,0.6-1.4,1.4S20.5,7.7,21.2,7.7z"/>
-	
-		<linearGradient id="SVGID_00000044151119195171880090000016489263670362291109_" gradientUnits="userSpaceOnUse" x1="14.4192" y1="-110.4727" x2="2.0973" y2="-101.7197" gradientTransform="matrix(1 0 0 -1 0 -81.48)">
-		<stop  offset="0" style="stop-color:#B7B7BD"/>
-		<stop  offset="0.68" style="stop-color:#EFEFEF"/>
-	</linearGradient>
-	<path style="fill:url(#SVGID_00000044151119195171880090000016489263670362291109_);" d="M2.1,20c-0.3,0-0.5,0-0.8,0
-		c1,2.8,2.9,5.2,5.3,6.9s5.3,2.6,8.3,2.6c0.1,0,0.3,0,0.4,0c-0.9-2.8-2.7-5.2-5.1-6.9C7.9,20.9,5.1,20,2.1,20z"/>
-</g>
-</svg>

auth/assets/custom-icons/icons/mistral.svg

@@ -1,15 +0,0 @@
-<svg width="500" height="500" viewBox="0 0 500 500" fill="none" xmlns="http://www.w3.org/2000/svg">
-<g style="mix-blend-mode:difference">
-<path fill-rule="evenodd" clip-rule="evenodd" d="M363.636 23H409.091V477.545H363.636V23ZM0 23H45.4545V477.545H0V23ZM227.273 295.727H181.818V386.636H227.273V295.727ZM272.727 113.909H318.182V204.818H272.727V113.909Z" fill="white"/>
-</g>
-<path d="M136.364 386.636H45.4545V477.545H136.364V386.636Z" fill="#EA3326"/>
-<path d="M500 386.636H409.091V477.545H500V386.636Z" fill="#EA3326"/>
-<path d="M136.364 295.727H45.4545V386.636H136.364V295.727Z" fill="#EB5829"/>
-<path d="M318.182 295.727H227.273V386.636H318.182V295.727Z" fill="#EB5829"/>
-<path d="M500 295.727H409.091V386.636H500V295.727Z" fill="#EB5829"/>
-<path d="M136.364 23H45.4545V113.909H136.364V23Z" fill="#F7D046"/>
-<path d="M500 23H409.091V113.909H500V23Z" fill="#F7D046"/>
-<path d="M227.273 113.909H45.4545V204.818H227.273V113.909Z" fill="#F2A73B"/>
-<path d="M500 113.909H318.182V204.818H500V113.909Z" fill="#F2A73B"/>
-<path d="M500 204.818H45.4545V295.727H500V204.818Z" fill="#EE792F"/>
-</svg>

auth/lib/l10n/arb/app_be.arb

@@ -1,97 +0,0 @@
-{
-  "account": "Уліковы запіс",
-  "unlock": "Разблакіраваць",
-  "recoveryKey": "Ключ аднаўлення",
-  "counterAppBarTitle": "Лічыльнік",
-  "@counterAppBarTitle": {
-    "description": "Text shown in the AppBar of the Counter Page"
-  },
-  "onBoardingBody": "Бяспечна зрабіць рэзервовую копію кодаў 2ФА",
-  "onBoardingGetStarted": "Пачаць",
-  "setupFirstAccount": "Наладзіць ваш першы ўліковы запіс",
-  "importScanQrCode": "Сканіраваць код QR-код",
-  "qrCode": "QR-код",
-  "importEnterSetupKey": "Увесці ключ наладжвання",
-  "importAccountPageTitle": "Увесці падрабязнасці ўліковага запісу",
-  "secretCanNotBeEmpty": "Сакрэт не можа быць пустым",
-  "bothIssuerAndAccountCanNotBeEmpty": "І выдавец, і ўліковы запіс не могуць быць пустымі",
-  "incorrectDetails": "Няправільныя падрабязнасці",
-  "pleaseVerifyDetails": "Праверце падрабязнасці і паспрабуйце яшчэ раз",
-  "codeIssuerHint": "Выдавец",
-  "codeSecretKeyHint": "Сакрэтны ключ",
-  "secret": "Сакрэт",
-  "all": "Усе",
-  "notes": "Нататкі",
-  "notesLengthLimit": "Максімальная колькасць сімвалаў у нататках не больш за {count}",
-  "@notesLengthLimit": {
-    "description": "Text to indicate the maximum number of characters allowed for notes",
-    "placeholders": {
-      "count": {
-        "description": "The maximum number of characters allowed for notes",
-        "type": "int",
-        "example": "100"
-      }
-    }
-  },
-  "codeAccountHint": "Уліковы запіс (vy@damen.com)",
-  "codeTagHint": "Тэг",
-  "accountKeyType": "Тып ключа",
-  "sessionExpired": "Сеанс завяршыўся",
-  "@sessionExpired": {
-    "description": "Title of the dialog when the users current session is invalid/expired"
-  },
-  "pleaseLoginAgain": "Аўтарызуйцеся яшчэ раз",
-  "loggingOut": "Выхад...",
-  "timeBasedKeyType": "Заснаваныя на часе (TOTP)",
-  "counterBasedKeyType": "Заснаваныя на лічыльніку (HOTP)",
-  "saveAction": "Захаваць",
-  "nextTotpTitle": "наступны",
-  "deleteCodeTitle": "Выдаліць код?",
-  "deleteCodeMessage": "Вы сапраўды хочаце выдаліць гэты код? Гэта дзеянне з'яўляецца незваротным.",
-  "trashCode": "Выдаліць код?",
-  "trashCodeMessage": "Вы сапраўды хочаце выдаліць код для {account}?",
-  "trash": "Сметніца",
-  "viewLogsAction": "Паглядзець журнал",
-  "preparingLogsTitle": "Падрыхтоўка журнала...",
-  "emailLogsTitle": "Адправіць журнал па электроннай пошце",
-  "exportLogsAction": "Экспартаваць журнал",
-  "reportABug": "Паведаміць пра памылку",
-  "reportBug": "Паведаміць пра памылку",
-  "contactSupport": "Звярнуцца ў службу падтрымкі",
-  "blog": "Блог",
-  "verifyPassword": "Праверыць пароль",
-  "pleaseWait": "Пачакайце...",
-  "generatingEncryptionKeysTitle": "Генерацыя ключоў шыфравання...",
-  "recreatePassword": "Стварыць пароль паўторна",
-  "useRecoveryKey": "Выкарыстоўваць ключ аднаўлення",
-  "incorrectPasswordTitle": "Няправільны пароль",
-  "welcomeBack": "З вяртаннем!",
-  "changePassword": "Змяніць пароль",
-  "data": "Даныя",
-  "importCodes": "Імпартаваць коды",
-  "passwordEmptyError": "Пароль не можа быць пустым",
-  "importFromApp": "Імпартаваць коды з {appName}",
-  "exportCodes": "Экспартаваць коды",
-  "importLabel": "Імпарт",
-  "selectFile": "Выбраць файл",
-  "yes": "Так",
-  "no": "Не",
-  "email": "Электронная пошта",
-  "support": "Падтрымка",
-  "settings": "Налады",
-  "pleaseTryAgain": "Калі ласка, паспрабуйце яшчэ раз",
-  "delete": "Выдаліць",
-  "enterYourPasswordHint": "Увядзіце ваш пароль",
-  "faq": "Частыя пытанні",
-  "deleteAccount": "Выдаліць уліковы запіс",
-  "noDeleteAccountAction": "Не, выдаліць уліковы запіс",
-  "sendEmail": "Адправіць ліст",
-  "createNewAccount": "Стварыць новы ўліковы запіс",
-  "confirmPassword": "Пацвердзіць пароль",
-  "close": "Закрыць",
-  "oopsSomethingWentWrong": "Штосьці пайшло не так.",
-  "language": "Мова",
-  "security": "Бяспека",
-  "searchHint": "Пошук...",
-  "search": "Пошук"
-}

auth/lib/l10n/arb/app_et.arb

@@ -1,4 +0,0 @@
-{
-  "loggingOut": "Väljalogimine...",
-  "useRecoveryKey": "Kasuta taastevõtit"
-}

auth/lib/services/user_remote_flag_service.dart

@@ -1,141 +0,0 @@
-import 'dart:async';
-import 'dart:io';
-
-import 'package:collection/collection.dart';
-import 'package:dio/dio.dart';
-import 'package:ente_auth/core/configuration.dart';
-import 'package:ente_auth/core/event_bus.dart';
-import 'package:ente_auth/core/network.dart';
-import 'package:ente_auth/events/notification_event.dart';
-import 'package:ente_auth/services/user_service.dart';
-import 'package:logging/logging.dart';
-import 'package:shared_preferences/shared_preferences.dart';
-
-class UserRemoteFlagService {
-  final _dio = Network.instance.getDio();
-  final _logger = Logger((UserRemoteFlagService).toString());
-  final _config = Configuration.instance;
-  late SharedPreferences _prefs;
-
-  UserRemoteFlagService._privateConstructor();
-
-  static final UserRemoteFlagService instance =
-      UserRemoteFlagService._privateConstructor();
-
-  static const String recoveryVerificationFlag = "recoveryKeyVerified";
-  static const String needRecoveryKeyVerification =
-      "needRecoveryKeyVerification";
-
-  Future<void> init() async {
-    _prefs = await SharedPreferences.getInstance();
-  }
-
-  bool shouldShowRecoveryVerification() {
-    if (!_prefs.containsKey(needRecoveryKeyVerification)) {
-      // fetch the status from remote
-      unawaited(_refreshRecoveryVerificationFlag());
-      return false;
-    } else {
-      final bool shouldShow = _prefs.getBool(needRecoveryKeyVerification)!;
-      if (shouldShow) {
-        // refresh the status to check if user marked it as done on another device
-        unawaited(_refreshRecoveryVerificationFlag());
-      }
-      return shouldShow;
-    }
-  }
-
-  // markRecoveryVerificationAsDone is used to track if user has verified their
-  // recovery key in the past or not. This helps in avoid showing the same
-  // prompt to the user on re-install or signing into a different device
-  Future<void> markRecoveryVerificationAsDone() async {
-    await _updateKeyValue(recoveryVerificationFlag, true.toString());
-    await _prefs.setBool(needRecoveryKeyVerification, false);
-  }
-
-  Future<void> _refreshRecoveryVerificationFlag() async {
-    _logger.finest('refresh recovery key verification flag');
-    final remoteStatusValue =
-        await _getValue(recoveryVerificationFlag, "false");
-    final bool isNeedVerificationFlagSet =
-        _prefs.containsKey(needRecoveryKeyVerification);
-    if (remoteStatusValue.toLowerCase() == "true") {
-      await _prefs.setBool(needRecoveryKeyVerification, false);
-      // If the user verified on different device, then we should refresh
-      // the UI to dismiss the Notification.
-      if (isNeedVerificationFlagSet) {
-        Bus.instance.fire(NotificationEvent());
-      }
-    } else if (!isNeedVerificationFlagSet) {
-      // Verification is not done yet as remoteStatus is false and local flag to
-      // show notification isn't set. Set the flag to true if any active
-      // session is older than 1 day.
-      final activeSessions = await UserService.instance.getActiveSessions();
-      final int microSecondsInADay = const Duration(days: 1).inMicroseconds;
-      final bool anyActiveSessionOlderThanADay =
-          activeSessions.sessions.firstWhereOrNull(
-                (e) =>
-                    (e.creationTime + microSecondsInADay) <
-                    DateTime.now().microsecondsSinceEpoch,
-              ) !=
-              null;
-      if (anyActiveSessionOlderThanADay) {
-        await _prefs.setBool(needRecoveryKeyVerification, true);
-        Bus.instance.fire(NotificationEvent());
-      } else {
-        // continue defaulting to no verification prompt
-        _logger.finest('No active session older than 1 day');
-      }
-    }
-  }
-
-  Future<String> _getValue(String key, String? defaultValue) async {
-    try {
-      final Map<String, dynamic> queryParams = {"key": key};
-      if (defaultValue != null) {
-        queryParams["defaultValue"] = defaultValue;
-      }
-      final response = await _dio.get(
-        "${_config.getHttpEndpoint()}/remote-store",
-        queryParameters: queryParams,
-        options: Options(
-          headers: {
-            "X-Auth-Token": _config.getToken(),
-          },
-        ),
-      );
-      if (response.statusCode != HttpStatus.ok) {
-        throw Exception("Unexpected status code ${response.statusCode}");
-      }
-      return response.data["value"];
-    } catch (e) {
-      _logger.info("Error while fetching bool status for $key", e);
-      rethrow;
-    }
-  }
-
-  // _setBooleanFlag sets the corresponding flag on remote
-  // to mark recovery as completed
-  Future<void> _updateKeyValue(String key, String value) async {
-    try {
-      final response = await _dio.post(
-        "${_config.getHttpEndpoint()}/remote-store/update",
-        data: {
-          "key": key,
-          "value": value,
-        },
-        options: Options(
-          headers: {
-            "X-Auth-Token": _config.getToken(),
-          },
-        ),
-      );
-      if (response.statusCode != HttpStatus.ok) {
-        throw Exception("Unexpected state");
-      }
-    } catch (e) {
-      _logger.warning("Failed to set flag for $key", e);
-      rethrow;
-    }
-  }
-}

auth/lib/ui/account/verify_recovery_page.dart

@@ -1,219 +0,0 @@
-import 'package:bip39/bip39.dart' as bip39;
-import 'package:dio/dio.dart';
-import 'package:ente_auth/core/configuration.dart';
-import 'package:ente_auth/ente_theme_data.dart';
-import 'package:ente_auth/l10n/l10n.dart';
-import 'package:ente_auth/services/local_authentication_service.dart';
-import 'package:ente_auth/services/user_remote_flag_service.dart';
-import 'package:ente_auth/ui/account/recovery_key_page.dart';
-import 'package:ente_auth/ui/common/gradient_button.dart';
-import 'package:ente_auth/ui/components/buttons/button_widget.dart';
-import 'package:ente_auth/utils/dialog_util.dart';
-import 'package:ente_auth/utils/navigation_util.dart';
-import 'package:ente_auth/utils/platform_util.dart';
-import 'package:ente_crypto_dart/ente_crypto_dart.dart';
-import 'package:flutter/material.dart';
-import 'package:logging/logging.dart';
-
-class VerifyRecoveryPage extends StatefulWidget {
-  const VerifyRecoveryPage({super.key});
-
-  @override
-  State<VerifyRecoveryPage> createState() => _VerifyRecoveryPageState();
-}
-
-class _VerifyRecoveryPageState extends State<VerifyRecoveryPage> {
-  final _recoveryKey = TextEditingController();
-  final Logger _logger = Logger((_VerifyRecoveryPageState).toString());
-
-  void _verifyRecoveryKey() async {
-    final dialog =
-        createProgressDialog(context, context.l10n.verifyingRecoveryKey);
-    await dialog.show();
-    try {
-      final String inputKey = _recoveryKey.text.trim();
-      final String recoveryKey =
-          CryptoUtil.bin2hex(Configuration.instance.getRecoveryKey());
-      final String recoveryKeyWords = bip39.entropyToMnemonic(recoveryKey);
-      if (inputKey == recoveryKey || inputKey == recoveryKeyWords) {
-        try {
-          await UserRemoteFlagService.instance.markRecoveryVerificationAsDone();
-        } catch (e) {
-          await dialog.hide();
-          if (e is DioException && e.type == DioExceptionType.unknown) {
-            await showErrorDialog(
-              context,
-              "No internet connection",
-              "Please check your internet connection and try again.",
-            );
-          } else {
-            await showGenericErrorDialog(
-              context: context,
-              error: e,
-            );
-          }
-          return;
-        }
-
-        await dialog.hide();
-        // todo: change this as per figma once the component is ready
-        await showErrorDialog(
-          context,
-          context.l10n.recoveryKeyVerified,
-          context.l10n.recoveryKeySuccessBody,
-        );
-        Navigator.of(context).pop();
-      } else {
-        throw Exception("recovery key didn't match");
-      }
-    } catch (e, s) {
-      _logger.severe("failed to verify recovery key", e, s);
-      await dialog.hide();
-      final String errMessage = context.l10n.invalidRecoveryKey;
-      final result = await showChoiceDialog(
-        context,
-        title: context.l10n.invalidKey,
-        body: errMessage,
-        firstButtonLabel: context.l10n.tryAgain,
-        secondButtonLabel: context.l10n.viewRecoveryKey,
-        secondButtonAction: ButtonAction.second,
-      );
-      if (result!.action == ButtonAction.second) {
-        await _onViewRecoveryKeyClick();
-      }
-    }
-  }
-
-  Future<void> _onViewRecoveryKeyClick() async {
-    final hasAuthenticated =
-        await LocalAuthenticationService.instance.requestLocalAuthentication(
-      context,
-      "Please authenticate to view your recovery key",
-    );
-    await PlatformUtil.refocusWindows();
-
-    if (hasAuthenticated) {
-      String recoveryKey;
-      try {
-        recoveryKey =
-            CryptoUtil.bin2hex(Configuration.instance.getRecoveryKey());
-        await routeToPage(
-          context,
-          RecoveryKeyPage(
-            recoveryKey,
-            context.l10n.ok,
-            showAppBar: true,
-            onDone: () {
-              Navigator.of(context).pop();
-            },
-          ),
-        );
-      } catch (e) {
-        // ignore: unawaited_futures
-        showGenericErrorDialog(
-          context: context,
-          error: e,
-        );
-        return;
-      }
-    }
-  }
-
-  @override
-  Widget build(BuildContext context) {
-    final enteTheme = Theme.of(context).colorScheme.enteTheme;
-    return Scaffold(
-      appBar: AppBar(
-        elevation: 0,
-        leading: IconButton(
-          icon: const Icon(Icons.arrow_back),
-          color: Theme.of(context).iconTheme.color,
-          onPressed: () {
-            Navigator.of(context).pop();
-          },
-        ),
-      ),
-      body: Padding(
-        padding: const EdgeInsets.symmetric(horizontal: 16.0),
-        child: LayoutBuilder(
-          builder: (context, constraints) {
-            return SingleChildScrollView(
-              child: ConstrainedBox(
-                constraints: BoxConstraints(
-                  minWidth: constraints.maxWidth,
-                  minHeight: constraints.maxHeight,
-                ),
-                child: IntrinsicHeight(
-                  child: Column(
-                    children: [
-                      const SizedBox(height: 12),
-                      SizedBox(
-                        width: double.infinity,
-                        child: Text(
-                          context.l10n.confirmRecoveryKey,
-                          style: enteTheme.textTheme.h3Bold,
-                          textAlign: TextAlign.left,
-                        ),
-                      ),
-                      const SizedBox(height: 18),
-                      Text(
-                        context.l10n.recoveryKeyVerifyReason,
-                        style: enteTheme.textTheme.small
-                            .copyWith(color: enteTheme.colorScheme.textMuted),
-                      ),
-                      const SizedBox(height: 12),
-                      TextFormField(
-                        decoration: InputDecoration(
-                          filled: true,
-                          hintText: context.l10n.enterYourRecoveryKey,
-                          contentPadding: const EdgeInsets.all(20),
-                          border: UnderlineInputBorder(
-                            borderSide: BorderSide.none,
-                            borderRadius: BorderRadius.circular(6),
-                          ),
-                        ),
-                        style: const TextStyle(
-                          fontSize: 14,
-                          fontFeatures: [FontFeature.tabularFigures()],
-                        ),
-                        controller: _recoveryKey,
-                        autofocus: false,
-                        autocorrect: false,
-                        keyboardType: TextInputType.multiline,
-                        minLines: 4,
-                        maxLines: null,
-                        onChanged: (_) {
-                          setState(() {});
-                        },
-                      ),
-                      const SizedBox(height: 12),
-                      Expanded(
-                        child: Container(
-                          alignment: Alignment.bottomCenter,
-                          width: double.infinity,
-                          padding: const EdgeInsets.fromLTRB(0, 12, 0, 40),
-                          child: Column(
-                            mainAxisAlignment: MainAxisAlignment.end,
-                            crossAxisAlignment: CrossAxisAlignment.stretch,
-                            children: [
-                              GradientButton(
-                                onTap: _verifyRecoveryKey,
-                                text: context.l10n.confirm,
-                              ),
-                              const SizedBox(height: 8),
-                            ],
-                          ),
-                        ),
-                      ),
-                      const SizedBox(height: 20),
-                    ],
-                  ),
-                ),
-              ),
-            );
-          },
-        ),
-      ),
-    );
-  }
-}

cli/config.yaml.example

@@ -5,8 +5,6 @@
 
 endpoint:
   api: "http://localhost:8080"
-  # Endpoint for the account service for passkey
-  accounts: "http://localhost:3001"
 
 log:
  http: false # log status code & time taken by requests

cli/internal/api/login.go

@@ -83,13 +83,14 @@ func (c *Client) VerifySRPSession(
 	return &res, nil
 }
 
-func (c *Client) SendEmailOTP(
+func (c *Client) SendLoginOTP(
 	ctx context.Context,
 	email string,
 ) error {
 	var res AuthorizationResponse
 	payload := map[string]interface{}{
-		"email": email,
+		"email":   email,
+		"purpose": "login",
 	}
 	r, err := c.restClient.R().
 		SetContext(ctx).

cli/internal/api/login_type.go

@@ -35,6 +35,7 @@ type AuthorizationResponse struct {
 	ID                 int64          `json:"id"`
 	KeyAttributes      *KeyAttributes `json:"keyAttributes,omitempty"`
 	EncryptedToken     string         `json:"encryptedToken,omitempty"`
+	AccountsUrl        string         `json:"accountsUrl"`
 	Token              string         `json:"token,omitempty"`
 	TwoFactorSessionID string         `json:"twoFactorSessionID"`
 	PassKeySessionID   string         `json:"passkeySessionID"`

cli/main.go

@@ -110,7 +110,6 @@ func initConfig(cliConfigDir string) {
 	viper.AddConfigPath(".")                // optionally look for config in the working directory
 
 	viper.SetDefault("endpoint.api", constants.EnteApiUrl)
-	viper.SetDefault("endpoint.accounts", constants.EnteAccountUrl)
 	viper.SetDefault("log.http", false)
 	if err := viper.ReadInConfig(); err != nil {
 		if _, ok := err.(viper.ConfigFileNotFoundError); ok {

cli/pkg/sign_in.go

@@ -8,8 +8,8 @@ import (
 	eCrypto "github.com/ente-io/cli/internal/crypto"
 	"github.com/ente-io/cli/pkg/model"
 	"github.com/ente-io/cli/utils/browser"
+	"github.com/ente-io/cli/utils/constants"
 	"github.com/ente-io/cli/utils/encoding"
-	"github.com/spf13/viper"
 	"log"
 
 	"github.com/kong/go-srp"
@@ -145,7 +145,10 @@ func (c *ClICtrl) verifyPassKey(ctx context.Context, authResp *api.Authorization
 	if !authResp.IsPasskeyRequired() {
 		return authResp, nil
 	}
-	baseAccountUrl := viper.GetString("endpoint.accounts")
+	baseAccountUrl := constants.EnteAccountUrl
+	if authResp.AccountsUrl != "" {
+		baseAccountUrl = authResp.AccountsUrl
+	}
 	passkeyAuthUrl := fmt.Sprintf("%s/passkeys/verify?passkeySessionID=%s&redirect=ente-cli://passkey&clientPackage=%s", baseAccountUrl, authResp.PassKeySessionID, app.ClientPkg())
 	fmt.Printf("Open this url in browser to verify passkey: %s\n", passkeyAuthUrl)
 	err := browser.OpenURL(passkeyAuthUrl)
@@ -167,7 +170,7 @@ func (c *ClICtrl) verifyPassKey(ctx context.Context, authResp *api.Authorization
 }
 
 func (c *ClICtrl) validateEmail(ctx context.Context, email string) (*api.AuthorizationResponse, error) {
-	err := c.Client.SendEmailOTP(ctx, email)
+	err := c.Client.SendLoginOTP(ctx, email)
 	if err != nil {
 		return nil, err
 	}

desktop/.github/workflows/desktop-release.yml

@@ -113,6 +113,11 @@ jobs:
                   APPLE_APP_SPECIFIC_PASSWORD:
                       ${{ secrets.APPLE_APP_SPECIFIC_PASSWORD }}
                   APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
+                  # Windows Azure Trusted Signing related values
+                  # https://www.electron.build/code-signing-win#using-azure-trusted-signing-beta
+                  AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
+                  AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
+                  AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
                   # Default is "draft", but since our nightly builds update
                   # existing pre-releases, set this to "prerelease".
                   EP_PRE_RELEASE: true

desktop/CHANGELOG.md

@@ -1,9 +1,27 @@
 # CHANGELOG
 
-## v1.7.12 (Unreleased)
+## v1.7.15 (Unreleased)
 
 - .
 
+## v1.7.14
+
+- Increase file size limit to 10 GB.
+
+## v1.7.13
+
+- Generate streams for videos (beta)
+
+    > Streamable videos can be enabled in Preferences. For more details, see the
+    > [video streaming FAQ](https://help.ente.io/photos/faq/video-streaming).
+
+- Support Turkish translations.
+
+## v1.7.12
+
+- Improved video player with streaming support (for already processed videos).
+- Support Arabic translations.
+
 ## v1.7.11
 
 - Improved file viewer.

desktop/docs/dependencies.md

@@ -3,6 +3,7 @@
 - [Electron](#electron)
 - [Dev dependencies](#dev)
 - [Functionality](#functionality)
+- [Pinned](#pinned)
 
 ## Electron
 
@@ -140,3 +141,13 @@ handles to avoid reopening them for every operation.
 
 [chokidar](https://github.com/paulmillr/chokidar) is used as a file system
 watcher for the watch folders functionality.
+
+## Pinned
+
+- `electron-builder` is pinned to 26.0.14 because of
+  https://github.com/electron-userland/electron-builder/issues/9161
+
+    To reproduce this locally, add `x64ArchFiles: "ffmpeg"` to
+    `electron-builder.yml`, then run `node_modules/.bin/electron-builder --mac`.
+
+- `electron-store` is pinned to 8.2.0 because subsequent versions are ESM only.

desktop/electron-builder.yml

@@ -14,6 +14,11 @@ win:
     target:
         - target: nsis
           arch: [x64, arm64]
+    azureSignOptions:
+        publisherName: ENTE TECHNOLOGIES, INC.
+        endpoint: https://eus.codesigning.azure.net/
+        certificateProfileName: EnteTrustCertProfile
+        codeSigningAccountName: EnteTechnologiesInc
 nsis:
     deleteAppDataOnUninstall: true
 linux:

desktop/eslint.config.mjs

@@ -39,6 +39,15 @@ export default ts.config(
                 "error",
                 { allowTernary: true },
             ],
+            // Allow force unwrapping potentially optional values.
+            //
+            // See: [Note: non-null-assertions have better stack trace]
+            "@typescript-eslint/no-non-null-assertion": "off",
+            // Allow `while(true)` etc.
+            "@typescript-eslint/no-unnecessary-condition": [
+                "error",
+                { allowConstantLoopConditions: true },
+            ],
         },
     },
 );

desktop/package.json

@@ -1,6 +1,6 @@
 {
     "name": "ente",
-    "version": "1.7.12-beta",
+    "version": "1.7.15-beta",
     "private": true,
     "description": "Desktop client for Ente Photos",
     "repository": "github:ente-io/photos-desktop",
@@ -21,9 +21,6 @@
         "lint-fix": "yarn prettier --write --log-level warn . && yarn eslint && yarn tsc",
         "prepare": "node scripts/vips.js"
     },
-    "resolutions": {
-        "jackspeak": "2.1.1"
-    },
     "dependencies": {
         "any-shell-escape": "^0.1.1",
         "auto-launch": "^5.0.6",
@@ -31,32 +28,32 @@
         "clip-bpe-js": "^0.0.6",
         "comlink": "^4.4.2",
         "compare-versions": "^6.1.1",
-        "electron-log": "^5.3.3",
+        "electron-log": "^5.4.1",
         "electron-store": "^8.2.0",
-        "electron-updater": "^6.6.2",
+        "electron-updater": "^6.6.5",
         "ffmpeg-static": "^5.2.0",
         "lru-cache": "^11.1.0",
         "next-electron-server": "^1.0.0",
         "node-stream-zip": "^1.15.0",
-        "onnxruntime-node": "^1.20.1"
+        "onnxruntime-node": "1.22.0-rev",
+        "zod": "^3.25.67"
     },
     "devDependencies": {
-        "@eslint/js": "^9.24.0",
-        "@tsconfig/node22": "^22.0.1",
+        "@eslint/js": "^9.30.1",
+        "@tsconfig/node22": "^22.0.2",
         "@types/auto-launch": "^5.0.5",
-        "@types/ffmpeg-static": "^3.0.3",
         "ajv": "^8.17.1",
-        "concurrently": "^9.1.2",
+        "concurrently": "^9.2.0",
         "cross-env": "^7.0.3",
-        "electron": "^35.1.4",
-        "electron-builder": "^26.0.12",
+        "electron": "^37.1.0",
+        "electron-builder": "26.0.14",
         "eslint": "^9",
-        "prettier": "3.5.3",
+        "prettier": "3.6.2",
         "prettier-plugin-organize-imports": "^4.1.0",
-        "prettier-plugin-packagejson": "^2.5.10",
-        "shx": "^0.3.4",
+        "prettier-plugin-packagejson": "^2.5.17",
+        "shx": "^0.4.0",
         "typescript": "^5.8.3",
-        "typescript-eslint": "^8.29.1"
+        "typescript-eslint": "^8.35.1"
     },
     "packageManager": "yarn@1.22.22",
     "productName": "ente"

desktop/src/main.ts

@@ -423,7 +423,14 @@ const createMainWindow = () => {
     window.on("hide", () => {
         // On macOS, when hiding the window also hide the app's icon in the dock
         // unless the user has unchecked the Settings > Hide dock icon checkbox.
-        if (shouldHideDockIcon()) app.dock?.hide();
+        if (shouldHideDockIcon()) {
+            // macOS emits a window "hide" event when going fullscreen, and if
+            // we hide the dock icon there then the window disappears. So ignore
+            // this scenario.
+            if (!window.isFullScreen()) {
+                app.dock?.hide();
+            }
+        }
     });
 
     window.on("show", () => void app.dock?.show());

desktop/src/main/ipc.ts

@@ -13,8 +13,10 @@ import type { BrowserWindow } from "electron";
 import { ipcMain } from "electron/main";
 import type {
     CollectionMapping,
+    FFmpegCommand,
     FolderWatch,
     PendingUploads,
+    UtilityProcessType,
     ZipItem,
 } from "../types/ipc";
 import { logToDisk } from "./log";
@@ -30,7 +32,7 @@ import {
     openLogDirectory,
     selectDirectory,
 } from "./services/dir";
-import { ffmpegExec } from "./services/ffmpeg";
+import { ffmpegDetermineVideoDuration, ffmpegExec } from "./services/ffmpeg";
 import {
     fsExists,
     fsFindFiles,
@@ -40,23 +42,23 @@ import {
     fsRename,
     fsRm,
     fsRmdir,
+    fsStatMtime,
     fsWriteFile,
     fsWriteFileViaBackup,
 } from "./services/fs";
 import { convertToJPEG, generateImageThumbnail } from "./services/image";
 import { logout } from "./services/logout";
-import { createMLWorker } from "./services/ml";
 import {
     lastShownChangelogVersion,
-    masterKeyB64,
-    saveMasterKeyB64,
+    masterKeyFromSafeStorage,
+    saveMasterKeyInSafeStorage,
     setLastShownChangelogVersion,
 } from "./services/store";
 import {
     clearPendingUploads,
     listZipItems,
-    markUploadedFiles,
-    markUploadedZipItems,
+    markUploadedFile,
+    markUploadedZipItem,
     pathOrZipItemSize,
     pendingUploads,
     setPendingUploads,
@@ -68,6 +70,7 @@ import {
     watchUpdateIgnoredFiles,
     watchUpdateSyncedFiles,
 } from "./services/watch";
+import { triggerCreateUtilityProcess } from "./services/workers";
 
 /**
  * Listen for IPC events sent/invoked by the renderer process, and route them to
@@ -105,10 +108,12 @@ export const attachIPCHandlers = () => {
 
     ipcMain.handle("selectDirectory", () => selectDirectory());
 
-    ipcMain.handle("masterKeyB64", () => masterKeyB64());
+    ipcMain.handle("masterKeyFromSafeStorage", () =>
+        masterKeyFromSafeStorage(),
+    );
 
-    ipcMain.handle("saveMasterKeyB64", (_, masterKeyB64: string) =>
-        saveMasterKeyB64(masterKeyB64),
+    ipcMain.handle("saveMasterKeyInSafeStorage", (_, masterKey: string) =>
+        saveMasterKeyInSafeStorage(masterKey),
     );
 
     ipcMain.handle("lastShownChangelogVersion", () =>
@@ -163,6 +168,8 @@ export const attachIPCHandlers = () => {
 
     ipcMain.handle("fsIsDir", (_, dirPath: string) => fsIsDir(dirPath));
 
+    ipcMain.handle("fsStatMtime", (_, path: string) => fsStatMtime(path));
+
     ipcMain.handle("fsFindFiles", (_, folderPath: string) =>
         fsFindFiles(folderPath),
     );
@@ -177,20 +184,26 @@ export const attachIPCHandlers = () => {
         "generateImageThumbnail",
         (
             _,
-            dataOrPathOrZipItem: Uint8Array | string | ZipItem,
+            pathOrZipItem: string | ZipItem,
             maxDimension: number,
             maxSize: number,
-        ) => generateImageThumbnail(dataOrPathOrZipItem, maxDimension, maxSize),
+        ) => generateImageThumbnail(pathOrZipItem, maxDimension, maxSize),
     );
 
     ipcMain.handle(
         "ffmpegExec",
         (
             _,
-            command: string[],
-            dataOrPathOrZipItem: Uint8Array | string | ZipItem,
+            command: FFmpegCommand,
+            pathOrZipItem: string | ZipItem,
             outputFileExtension: string,
-        ) => ffmpegExec(command, dataOrPathOrZipItem, outputFileExtension),
+        ) => ffmpegExec(command, pathOrZipItem, outputFileExtension),
+    );
+
+    ipcMain.handle(
+        "ffmpegDetermineVideoDuration",
+        (_, pathOrZipItem: string | ZipItem) =>
+            ffmpegDetermineVideoDuration(pathOrZipItem),
     );
 
     // - Upload
@@ -210,13 +223,15 @@ export const attachIPCHandlers = () => {
     );
 
     ipcMain.handle(
-        "markUploadedFiles",
-        (_, paths: PendingUploads["filePaths"]) => markUploadedFiles(paths),
+        "markUploadedFile",
+        (_, path: string, associatedPath: string | undefined) =>
+            markUploadedFile(path, associatedPath),
     );
 
     ipcMain.handle(
-        "markUploadedZipItems",
-        (_, items: PendingUploads["zipItems"]) => markUploadedZipItems(items),
+        "markUploadedZipItem",
+        (_, item: ZipItem, associatedItem: ZipItem | undefined) =>
+            markUploadedZipItem(item, associatedItem),
     );
 
     ipcMain.handle("clearPendingUploads", () => clearPendingUploads());
@@ -227,9 +242,11 @@ export const attachIPCHandlers = () => {
  * the main window to do their thing.
  */
 export const attachMainWindowIPCHandlers = (mainWindow: BrowserWindow) => {
-    // - ML
+    // - Utility processes
 
-    ipcMain.on("createMLWorker", () => createMLWorker(mainWindow));
+    ipcMain.on("triggerCreateUtilityProcess", (_, type: UtilityProcessType) =>
+        triggerCreateUtilityProcess(type, mainWindow),
+    );
 };
 
 /**

desktop/src/main/log-worker.ts

@@ -0,0 +1,59 @@
+/**
+ * A object that behaves similar to the default export of "./log", except this
+ * can be used from within a utility process.
+ *
+ * ---
+ *
+ * We cannot directly do
+ *
+ *     import log from "../log";
+ *
+ * because that requires the Electron APIs that are not available to a utility
+ * process (See: [Note: Using Electron APIs in UtilityProcess]).
+ *
+ * But even if that were to work, logging will still be problematic since we'd
+ * try opening the log file from two different Node.js processes (this one, and
+ * the main one), and I didn't find any indication in the electron-log
+ * repository that the log file's integrity would be maintained in such cases.
+ *
+ * So instead we provide this proxy log object that uses the
+ * `process.parentPort` to transport the logs over to the main process, where
+ * the {@link processUtilityProcessLogMessage} function in the main process is
+ * expected to handle these (sending them to the actual log).
+ */
+export default {
+    error: (s: string, e?: unknown) =>
+        mainProcess("log.errorString", messageWithError(s, e)),
+    warn: (s: string, e?: unknown) =>
+        mainProcess("log.warnString", messageWithError(s, e)),
+    info: (...ms: unknown[]) => mainProcess("log.info", ms),
+    /**
+     * Unlike the real {@link log.debug}, this is (a) eagerly evaluated, and (b)
+     * accepts only strings.
+     */
+    debugString: (s: string) => mainProcess("log.debugString", s),
+};
+
+/**
+ * Send a message to the main process using a barebones RPC protocol.
+ */
+const mainProcess = (method: string, param: unknown) =>
+    process.parentPort.postMessage({ method, p: param });
+
+// Duplicated verbatim from ./log.ts
+const messageWithError = (message: string, e?: unknown) => {
+    if (!e) return message;
+
+    let es: string;
+    if (e instanceof Error) {
+        // In practice, we expect ourselves to be called with Error objects, so
+        // this is the happy path so to say.
+        es = [`${e.name}: ${e.message}`, e.stack].filter((x) => x).join("\n");
+    } else {
+        // For the rest rare cases, use the default string serialization of e.
+        // eslint-disable-next-line @typescript-eslint/no-base-to-string
+        es = String(e);
+    }
+
+    return `${message}: ${es}`;
+};

desktop/src/main/log.ts

@@ -83,6 +83,56 @@ const logDebug = (param: () => unknown) => {
     }
 };
 
+/**
+ * Handle log messages posted from the utility process in the main process.
+ *
+ * See: [Note: Using Electron APIs in UtilityProcess]
+ *
+ * @param message The arbitrary message that was received as an argument to the
+ * "message" event invoked on a {@link UtilityProcess}.
+ *
+ * @returns true if the message was recognized and handled, and false otherwise.
+ */
+export const processUtilityProcessLogMessage = (
+    logTag: string,
+    message: unknown,
+) => {
+    const m = message; /* shorter alias */
+    if (m && typeof m == "object" && "method" in m && "p" in m) {
+        const p = m.p;
+        switch (m.method) {
+            case "log.errorString":
+                if (typeof p == "string") {
+                    logError(`${logTag} ${p}`);
+                    return true;
+                }
+                break;
+            case "log.warnString":
+                if (typeof p == "string") {
+                    logWarn(`${logTag} ${p}`);
+                    return true;
+                }
+                break;
+            case "log.info":
+                if (Array.isArray(p)) {
+                    // Need to cast from any[] to unknown[]
+                    logInfo(logTag, ...(p as unknown[]));
+                    return true;
+                }
+                break;
+            case "log.debugString":
+                if (typeof p == "string") {
+                    logDebug(() => `${logTag} ${p}`);
+                    return true;
+                }
+                break;
+            default:
+                break;
+        }
+    }
+    return false;
+};
+
 /**
  * Ente's logger.
  *

desktop/src/main/services/ffmpeg.ts

@@ -1,132 +1,77 @@
-import pathToFfmpeg from "ffmpeg-static";
+/**
+ * @file A bridge to the ffmpeg utility process. This code runs in the main
+ * process.
+ */
+
+import { wrap } from "comlink";
 import fs from "node:fs/promises";
-import type { ZipItem } from "../../types/ipc";
-import { ensure } from "../utils/common";
-import { execAsync } from "../utils/electron";
+import type { FFmpegCommand, ZipItem } from "../../types/ipc";
 import {
     deleteTempFileIgnoringErrors,
-    makeFileForDataOrPathOrZipItem,
+    makeFileForStreamOrPathOrZipItem,
     makeTempFilePath,
 } from "../utils/temp";
-
-/* Ditto in the web app's code (used by the Wasm FFmpeg invocation). */
-const ffmpegPathPlaceholder = "FFMPEG";
-const inputPathPlaceholder = "INPUT";
-const outputPathPlaceholder = "OUTPUT";
+import type { FFmpegUtilityProcess } from "./ffmpeg-worker";
+import { ffmpegUtilityProcessEndpoint } from "./workers";
 
 /**
- * Run a FFmpeg command
- *
- * [Note: FFmpeg in Electron]
- *
- * There is a Wasm build of FFmpeg, but that is currently 10-20 times slower
- * that the native build. That is slow enough to be unusable for our purposes.
- * https://ffmpegwasm.netlify.app/docs/performance
- *
- * So the alternative is to bundle a FFmpeg executable binary with our app. e.g.
- *
- *     yarn add fluent-ffmpeg ffmpeg-static ffprobe-static
- *
- * (we only use ffmpeg-static, the rest are mentioned for completeness' sake).
- *
- * Interestingly, Electron already bundles an binary FFmpeg library (it comes
- * from the ffmpeg fork maintained by Chromium).
- * https://chromium.googlesource.com/chromium/third_party/ffmpeg
- * https://stackoverflow.com/questions/53963672/what-version-of-ffmpeg-is-bundled-inside-electron
- *
- * This can be found in (e.g. on macOS) at
- *
- *     $ file ente.app/Contents/Frameworks/Electron\ Framework.framework/Versions/Current/Libraries/libffmpeg.dylib
- *     .../libffmpeg.dylib: Mach-O 64-bit dynamically linked shared library arm64
- *
- * But I'm not sure if our code is supposed to be able to use it, and how.
+ * Return a handle to the ffmpeg utility process, starting it if needed.
+ */
+export const ffmpegUtilityProcess = () =>
+    ffmpegUtilityProcessEndpoint().then((port) =>
+        wrap<FFmpegUtilityProcess>(port),
+    );
+
+/**
+ * Implement the IPC "ffmpegExec" contract, writing the input and output to
+ * temporary files as needed, and then forward to the {@link ffmpegExec} running
+ * in the utility process.
  */
 export const ffmpegExec = async (
-    command: string[],
-    dataOrPathOrZipItem: Uint8Array | string | ZipItem,
+    command: FFmpegCommand,
+    pathOrZipItem: string | ZipItem,
     outputFileExtension: string,
-): Promise<Uint8Array> => {
+): Promise<Uint8Array> =>
+    withInputFile(pathOrZipItem, async (worker, inputFilePath) => {
+        const outputFilePath = await makeTempFilePath(outputFileExtension);
+        try {
+            await worker.ffmpegExec(command, inputFilePath, outputFilePath);
+            return await fs.readFile(outputFilePath);
+        } finally {
+            await deleteTempFileIgnoringErrors(outputFilePath);
+        }
+    });
+
+export const withInputFile = async <T>(
+    pathOrZipItem: string | ZipItem,
+    f: (worker: FFmpegUtilityProcess, inputFilePath: string) => Promise<T>,
+): Promise<T> => {
+    const worker = await ffmpegUtilityProcess();
+
     const {
         path: inputFilePath,
         isFileTemporary: isInputFileTemporary,
         writeToTemporaryFile: writeToTemporaryInputFile,
-    } = await makeFileForDataOrPathOrZipItem(dataOrPathOrZipItem);
+    } = await makeFileForStreamOrPathOrZipItem(pathOrZipItem);
 
-    const outputFilePath = await makeTempFilePath(outputFileExtension);
     try {
         await writeToTemporaryInputFile();
 
-        const cmd = substitutePlaceholders(
-            command,
-            inputFilePath,
-            outputFilePath,
-        );
-
-        await execAsync(cmd);
-
-        return await fs.readFile(outputFilePath);
+        return await f(worker, inputFilePath);
     } finally {
         if (isInputFileTemporary)
             await deleteTempFileIgnoringErrors(inputFilePath);
-        await deleteTempFileIgnoringErrors(outputFilePath);
     }
 };
 
-const substitutePlaceholders = (
-    command: string[],
-    inputFilePath: string,
-    outputFilePath: string,
-) =>
-    command.map((segment) => {
-        if (segment == ffmpegPathPlaceholder) {
-            return ffmpegBinaryPath();
-        } else if (segment == inputPathPlaceholder) {
-            return inputFilePath;
-        } else if (segment == outputPathPlaceholder) {
-            return outputFilePath;
-        } else {
-            return segment;
-        }
-    });
-
 /**
- * Return the path to the `ffmpeg` binary.
- *
- * At runtime, the FFmpeg binary is present in a path like (macOS example):
- * `ente.app/Contents/Resources/app.asar.unpacked/node_modules/ffmpeg-static/ffmpeg`
+ * Implement the IPC "ffmpegDetermineVideoDuration" contract, writing the input
+ * to temporary files as needed, and then forward to the
+ * {@link ffmpegDetermineVideoDuration} running in the utility process.
  */
-const ffmpegBinaryPath = () => {
-    // This substitution of app.asar by app.asar.unpacked is suggested by the
-    // ffmpeg-static library author themselves:
-    // https://github.com/eugeneware/ffmpeg-static/issues/16
-    return ensure(pathToFfmpeg).replace("app.asar", "app.asar.unpacked");
-};
-
-/**
- * A variant of {@link ffmpegExec} adapted to work with streams so that it can
- * handle the MP4 conversion of large video files.
- *
- * See: [Note: Convert to MP4]
-
- * @param inputFilePath The path to a file on the user's local file system. This
- * is the video we want to convert.
- * @param inputFilePath The path to a file on the user's local file system where
- * we should write the converted MP4 video.
- */
-export const ffmpegConvertToMP4 = async (
-    inputFilePath: string,
-    outputFilePath: string,
-): Promise<void> => {
-    const command = [
-        ffmpegPathPlaceholder,
-        "-i",
-        inputPathPlaceholder,
-        "-preset",
-        "ultrafast",
-        outputPathPlaceholder,
-    ];
-
-    const cmd = substitutePlaceholders(command, inputFilePath, outputFilePath);
-
-    await execAsync(cmd);
-};
+export const ffmpegDetermineVideoDuration = async (
+    pathOrZipItem: string | ZipItem,
+): Promise<number> =>
+    withInputFile(pathOrZipItem, async (worker, inputFilePath) =>
+        worker.ffmpegDetermineVideoDuration(inputFilePath),
+    );

desktop/src/main/services/fs.ts

@@ -36,6 +36,17 @@ export const fsIsDir = async (dirPath: string) => {
     return stat.isDirectory();
 };
 
+export const fsStatMtime = (path: string) =>
+    // [Note: Integral last modified time]
+    //
+    // Whenever we need to find the modified time of a file, use the
+    // `mtime.getTime()` instead of `mtimeMs` of the stat; this way, it is
+    // guaranteed that the times are integral (we persist these values to remote
+    // in some cases, and the contract is for them to be integral; mtimeMs is a
+    // float with sub-millisecond precision), and that all places use the same
+    // value so that they're comparable.
+    fs.stat(path).then((st) => st.mtime.getTime());
+
 export const fsFindFiles = async (dirPath: string) => {
     const items = await fs.readdir(dirPath, { withFileTypes: true });
     let paths: string[] = [];

desktop/src/main/services/image.ts

@@ -6,7 +6,7 @@ import { type ZipItem } from "../../types/ipc";
 import { execAsync, isDev } from "../utils/electron";
 import {
     deleteTempFileIgnoringErrors,
-    makeFileForDataOrPathOrZipItem,
+    makeFileForStreamOrPathOrZipItem,
     makeTempFilePath,
 } from "../utils/temp";
 
@@ -61,7 +61,7 @@ const vipsPath = () =>
     );
 
 export const generateImageThumbnail = async (
-    dataOrPathOrZipItem: Uint8Array | string | ZipItem,
+    pathOrZipItem: string | ZipItem,
     maxDimension: number,
     maxSize: number,
 ): Promise<Uint8Array> => {
@@ -69,7 +69,7 @@ export const generateImageThumbnail = async (
         path: inputFilePath,
         isFileTemporary: isInputFileTemporary,
         writeToTemporaryFile: writeToTemporaryInputFile,
-    } = await makeFileForDataOrPathOrZipItem(dataOrPathOrZipItem);
+    } = await makeFileForStreamOrPathOrZipItem(pathOrZipItem);
 
     const outputFilePath = await makeTempFilePath("jpeg");
 

desktop/src/main/services/logout.ts

@@ -1,8 +1,9 @@
 import type { FSWatcher } from "chokidar";
 import log from "../log";
-import { clearConvertToMP4Results } from "../stream";
+import { clearPendingVideoResults } from "../stream";
 import { clearStores } from "./store";
 import { watchReset } from "./watch";
+import { terminateUtilityProcesses } from "./workers";
 import { clearOpenZipCache } from "./zip";
 
 /**
@@ -22,9 +23,9 @@ export const logout = (watcher: FSWatcher) => {
         ignoreError("FS watch", e);
     }
     try {
-        clearConvertToMP4Results();
+        clearPendingVideoResults();
     } catch (e) {
-        ignoreError("convert-to-mp4", e);
+        ignoreError("video", e);
     }
     try {
         clearStores();
@@ -36,4 +37,9 @@ export const logout = (watcher: FSWatcher) => {
     } catch (e) {
         ignoreError("zip cache", e);
     }
+    try {
+        terminateUtilityProcesses();
+    } catch (e) {
+        ignoreError("utility processes", e);
+    }
 };

desktop/src/main/services/ml-worker.ts

@@ -15,46 +15,16 @@ import { existsSync } from "fs";
 import fs from "node:fs/promises";
 import path from "node:path";
 import * as ort from "onnxruntime-node";
+import { z } from "zod/v4";
+import log from "../log-worker";
 import { messagePortMainEndpoint } from "../utils/comlink";
-import { ensure, wait } from "../utils/common";
+import { wait } from "../utils/common";
 import { writeStream } from "../utils/stream";
+import { fsStatMtime } from "./fs";
 
-/**
- * We cannot do
- *
- *     import log from "../log";
- *
- * because that requires the Electron APIs that are not available to a utility
- * process (See: [Note: Using Electron APIs in UtilityProcess]). But even if
- * that were to work, logging will still be problematic since we'd try opening
- * the log file from two different Node.js processes (this one, and the main
- * one), and I didn't find any indication in the electron-log repository that
- * the log file's integrity would be maintained in such cases.
- *
- * So instead we create this proxy log object that uses `process.parentPort` to
- * transport the logs over to the main process.
- */
-const log = {
-    /**
-     * Unlike the real {@link log.error}, this accepts only the first string
-     * argument, not the second optional error one.
-     */
-    errorString: (s: string) => mainProcess("log.errorString", s),
-    info: (...ms: unknown[]) => mainProcess("log.info", ms),
-    /**
-     * Unlike the real {@link log.debug}, this is (a) eagerly evaluated, and (b)
-     * accepts only strings.
-     */
-    debugString: (s: string) => mainProcess("log.debugString", s),
-};
+log.debugString("Started ML utility process");
 
-/**
- * Send a message to the main process using a barebones RPC protocol.
- */
-const mainProcess = (method: string, param: unknown) =>
-    process.parentPort.postMessage({ method, p: param });
-
-log.debugString(`Started ML worker process`);
+process.on("uncaughtException", (e, origin) => log.error(origin, e));
 
 process.parentPort.once("message", (e) => {
     // Initialize ourselves with the data we got from our parent.
@@ -63,12 +33,13 @@ process.parentPort.once("message", (e) => {
     // parent.
     expose(
         {
+            fsStatMtime,
             computeCLIPImageEmbedding,
             computeCLIPTextEmbeddingIfAvailable,
             detectFaces,
             computeFaceEmbeddings,
         },
-        messagePortMainEndpoint(ensure(e.ports[0])),
+        messagePortMainEndpoint(e.ports[0]!),
     );
 });
 
@@ -80,19 +51,12 @@ process.parentPort.once("message", (e) => {
 let _userDataPath: string | undefined;
 
 /** Equivalent to app.getPath("userData") */
-const userDataPath = () => ensure(_userDataPath);
+const userDataPath = () => _userDataPath!;
+
+const MLWorkerInitData = z.object({ userDataPath: z.string() });
 
 const parseInitData = (data: unknown) => {
-    if (
-        data &&
-        typeof data == "object" &&
-        "userDataPath" in data &&
-        typeof data.userDataPath == "string"
-    ) {
-        _userDataPath = data.userDataPath;
-    } else {
-        log.errorString("Unparseable initialization data");
-    }
+    _userDataPath = MLWorkerInitData.parse(data).userDataPath;
 };
 
 /**
@@ -159,7 +123,7 @@ const modelPathDownloadingIfNeeded = async (
     } else {
         const size = (await fs.stat(modelPath)).size;
         if (size !== expectedByteSize) {
-            log.errorString(
+            log.error(
                 `The size ${size} of model ${modelName} does not match the expected size, downloading again`,
             );
             await downloadModel(modelPath, modelName);
@@ -220,14 +184,13 @@ const downloadModel = async (saveLocation: string, name: string) => {
 /**
  * Create an ONNX {@link InferenceSession} with some defaults.
  */
-const createInferenceSession = async (modelPath: string) => {
-    return await ort.InferenceSession.create(modelPath, {
+const createInferenceSession = (modelPath: string) =>
+    ort.InferenceSession.create(modelPath, {
         // Restrict the number of threads to 1.
         intraOpNumThreads: 1,
         // Be more conservative with RAM usage.
         enableCpuMemArena: false,
     });
-};
 
 const cachedCLIPImageSession = makeCachedInferenceSession(
     "mobileclip_s2_image_opset18_rgba_opt.onnx",
@@ -250,7 +213,7 @@ export const computeCLIPImageEmbedding = async (
     const results = await session.run(feeds);
     log.debugString(`ONNX/CLIP image embedding took ${Date.now() - t} ms`);
     /* Need these model specific casts to type the result */
-    return ensure(results.output).data as Float32Array;
+    return results.output!.data as Float32Array;
 };
 
 const cachedCLIPTextSession = makeCachedInferenceSession(
@@ -269,9 +232,11 @@ const getTokenizer = () => (_tokenizer ??= new Tokenizer());
 export const computeCLIPTextEmbeddingIfAvailable = async (text: string) => {
     const sessionOrSkip = await Promise.race([
         cachedCLIPTextSession(),
-        // Wait for a tick to get the session promise to resolved the first time
-        // this code runs on each app start (and the model has been downloaded).
-        wait(0).then(() => 1),
+        // Wait a bit to get the session promise to resolved the first time this
+        // code runs on each app start (in these cases the model will already be
+        // downloaded, so session creation should take only a 1 or 2 ticks: file
+        // system stat, and ort.InferenceSession.create).
+        wait(50).then(() => 1),
     ]);
 
     // Don't wait for the download to complete.
@@ -290,7 +255,7 @@ export const computeCLIPTextEmbeddingIfAvailable = async (text: string) => {
     const t = Date.now();
     const results = await session.run(feeds);
     log.debugString(`ONNX/CLIP text embedding took ${Date.now() - t} ms`);
-    return ensure(results.output).data as Float32Array;
+    return results.output!.data as Float32Array;
 };
 
 const cachedFaceDetectionSession = makeCachedInferenceSession(
@@ -311,7 +276,7 @@ export const detectFaces = async (
     const t = Date.now();
     const results = await session.run(feeds);
     log.debugString(`ONNX/YOLO face detection took ${Date.now() - t} ms`);
-    return ensure(results.output).data;
+    return results.output!.data;
 };
 
 const cachedFaceEmbeddingSession = makeCachedInferenceSession(

desktop/src/main/services/ml.ts

@@ -1,147 +0,0 @@
-/**
- * @file ML related functionality. This code runs in the main process.
- */
-
-import {
-    MessageChannelMain,
-    type BrowserWindow,
-    type UtilityProcess,
-} from "electron";
-import { app, utilityProcess } from "electron/main";
-import path from "node:path";
-import log from "../log";
-
-/** The active ML worker (utility) process, if any. */
-let _child: UtilityProcess | undefined;
-
-/**
- * Create a new ML worker process, terminating the older ones (if any).
- *
- * [Note: ML IPC]
- *
- * The primary reason for doing ML tasks in the Node.js layer is so that we can
- * use the binary ONNX runtime, which is 10-20x faster than the Wasm one that
- * can be used directly on the web layer.
- *
- * For this to work, the main and renderer process need to communicate with each
- * other. Further, in the web layer the ML indexing runs in a web worker (so as
- * to not get in the way of the main thread). So the communication has 2 hops:
- *
- *     Node.js main <-> Renderer main <-> Renderer web worker
- *
- * This naive way works, but has a problem. The Node.js main process is in the
- * code path for delivering user events to the renderer process. The ML tasks we
- * do take in the order of 100-300 ms (possibly more) for each individual
- * inference. Thus, the Node.js main process is busy for those 100-300 ms, and
- * does not forward events to the renderer, causing the UI to jitter.
- *
- * The solution for this is to spawn an Electron UtilityProcess, which we can
- * think of a regular Node.js child process.  This frees up the Node.js main
- * process, and would remove the jitter.
- * https://www.electronjs.org/docs/latest/tutorial/process-model
- *
- * It would seem that this introduces another hop in our IPC
- *
- *     Node.js utility process <-> Node.js main <-> ...
- *
- * but here we can use the special bit about Electron utility processes that
- * separates them from regular Node.js child processes: their support for
- * message ports. https://www.electronjs.org/docs/latest/tutorial/message-ports
- *
- * As a brief summary, a MessagePort is a web feature that allows two contexts
- * to communicate. A pair of message ports is called a message channel. The cool
- * thing about these is that we can pass these ports themselves over IPC.
- *
- * > One caveat here is that the message ports can only be passed using the
- * > `postMessage` APIs, not the usual send/invoke APIs.
- *
- * So we
- *
- * 1. In the utility process create a message channel.
- * 2. Spawn a utility process, and send one port of the pair to it.
- * 3. Send the other port of the pair to the renderer.
- *
- * The renderer will forward that port to the web worker that is coordinating
- * the ML indexing on the web layer. Thereafter, the utility process and web
- * worker can directly talk to each other!
- *
- *     Node.js utility process <-> Renderer web worker
- *
- * The RPC protocol is handled using comlink on both ends. The port itself needs
- * to be relayed using `postMessage`.
- */
-export const createMLWorker = (window: BrowserWindow) => {
-    if (_child) {
-        log.debug(() => "Terminating previous ML worker process");
-        _child.kill();
-        _child = undefined;
-    }
-
-    const { port1, port2 } = new MessageChannelMain();
-
-    const child = utilityProcess.fork(path.join(__dirname, "ml-worker.js"));
-    const userDataPath = app.getPath("userData");
-    child.postMessage({ userDataPath }, [port1]);
-
-    window.webContents.postMessage("createMLWorker/port", undefined, [port2]);
-
-    handleMessagesFromUtilityProcess(child);
-
-    _child = child;
-};
-
-/**
- * Handle messages posted from the utility process.
- *
- * [Note: Using Electron APIs in UtilityProcess]
- *
- * Only a small subset of the Electron APIs are available to a UtilityProcess.
- * As of writing (Jul 2024, Electron 30), only the following are available:
- *
- * - net
- * - systemPreferences
- *
- * In particular, `app` is not available.
- *
- * We structure our code so that it doesn't need anything apart from `net`.
- *
- * For the other cases,
- *
- * -  Additional parameters to the utility process are passed alongwith the
- *    initial message where we provide it the message port.
- *
- * -  When we need to communicate from the utility process to the main process,
- *    we use the `parentPort` in the utility process.
- */
-const handleMessagesFromUtilityProcess = (child: UtilityProcess) => {
-    const logTag = "[ml-worker]";
-    child.on("message", (m: unknown) => {
-        if (m && typeof m == "object" && "method" in m && "p" in m) {
-            const p = m.p;
-            switch (m.method) {
-                case "log.errorString":
-                    if (typeof p == "string") {
-                        log.error(`${logTag} ${p}`);
-                        return;
-                    }
-                    break;
-                case "log.info":
-                    if (Array.isArray(p)) {
-                        // Need to cast from any[] to unknown[]
-                        log.info(logTag, ...(p as unknown[]));
-                        return;
-                    }
-                    break;
-                case "log.debugString":
-                    if (typeof p == "string") {
-                        log.debug(() => `${logTag} ${p}`);
-                        return;
-                    }
-                    break;
-                default:
-                    break;
-            }
-        }
-        log.info("Ignoring unknown message from ML worker", m);
-    });
-};

desktop/src/main/services/store.ts

@@ -24,17 +24,17 @@ export const clearStores = () => {
  * On macOS, `safeStorage` stores our data under a Keychain entry named
  * "<app-name> Safe Storage". In our case, "ente Safe Storage".
  */
-export const saveMasterKeyB64 = (masterKeyB64: string) => {
-    const encryptedKey = safeStorage.encryptString(masterKeyB64);
-    const b64EncryptedKey = Buffer.from(encryptedKey).toString("base64");
-    safeStorageStore.set("encryptionKey", b64EncryptedKey);
+export const saveMasterKeyInSafeStorage = (masterKey: string) => {
+    const encryptedKeyBuffer = safeStorage.encryptString(masterKey);
+    const encryptedKey = Buffer.from(encryptedKeyBuffer).toString("base64");
+    safeStorageStore.set("encryptionKey", encryptedKey);
 };
 
-export const masterKeyB64 = (): string | undefined => {
-    const b64EncryptedKey = safeStorageStore.get("encryptionKey");
-    if (!b64EncryptedKey) return undefined;
-    const keyBuffer = Buffer.from(b64EncryptedKey, "base64");
-    return safeStorage.decryptString(keyBuffer);
+export const masterKeyFromSafeStorage = (): string | undefined => {
+    const encryptedKey = safeStorageStore.get("encryptionKey");
+    if (!encryptedKey) return undefined;
+    const encryptedKeyBuffer = Buffer.from(encryptedKey, "base64");
+    return safeStorage.decryptString(encryptedKeyBuffer);
 };
 
 export const lastShownChangelogVersion = (): number | undefined =>

desktop/src/main/services/upload.ts

@@ -135,22 +135,35 @@ export const setPendingUploads = ({
     });
 };
 
-export const markUploadedFiles = (paths: string[]) => {
+export const markUploadedFile = (
+    path: string,
+    associatedPath: string | undefined,
+) => {
     const existing = uploadStatusStore.get("filePaths") ?? [];
-    const updated = existing.filter((p) => !paths.includes(p));
+    const updated = existing.filter((p) => p != path && p != associatedPath);
     uploadStatusStore.set("filePaths", updated);
+    // See: [Note: Integral last modified time]
+    return fs.stat(path).then((st) => st.mtime.getTime());
 };
 
-export const markUploadedZipItems = (
-    items: [zipPath: string, entryName: string][],
+export const markUploadedZipItem = (
+    item: ZipItem,
+    associatedItem: ZipItem | undefined,
 ) => {
     const existing = uploadStatusStore.get("zipItems") ?? [];
-    const updated = existing.filter(
-        (z) => !items.some((e) => z[0] == e[0] && z[1] == e[1]),
+    const updated = exceptZipItem(
+        exceptZipItem(existing, item),
+        associatedItem,
     );
     uploadStatusStore.set("zipItems", updated);
+    return fs.stat(item[0]).then((st) => st.mtime.getTime());
 };
 
+const exceptZipItem = (items: ZipItem[], item: ZipItem | undefined) =>
+    item
+        ? items.filter((zi) => !(zi[0] == item[0] && zi[1] == item[1]))
+        : items;
+
 export const clearPendingUploads = () => {
     uploadStatusStore.clear();
     clearOpenZipCache();

desktop/src/main/services/watch.ts

@@ -28,6 +28,13 @@ export const createWatcher = (mainWindow: BrowserWindow) => {
         // Ask the watcher to wait for a the file size to stabilize before
         // telling us about a new file. By default, it waits for 2 seconds.
         awaitWriteFinish: true,
+        // On macOS we start getting "EMFILE: too many open files" when watching
+        // large folders. This is a known regression in Chokidar v4:
+        // https://github.com/paulmillr/chokidar/issues/1385
+        //
+        // The recommended workaround for now is to enable usePolling. Since it
+        // comes at a performance cost, we only do it where needed (macOS).
+        ...(process.platform == "darwin" ? { usePolling: true } : {}),
     });
 
     watcher

desktop/src/main/services/workers.ts

@@ -0,0 +1,241 @@
+/**
+ * @file This main process code and interface for dealing with the various
+ * utility processes that we create.
+ */
+
+import type { Endpoint } from "comlink";
+import {
+    MessageChannelMain,
+    type BrowserWindow,
+    type UtilityProcess,
+} from "electron";
+import { app, utilityProcess } from "electron/main";
+import path from "node:path";
+import type { UtilityProcessType } from "../../types/ipc";
+import log, { processUtilityProcessLogMessage } from "../log";
+import { messagePortMainEndpoint } from "../utils/comlink";
+
+/**
+ * Terminate any existing utility processes if they're running.
+ *
+ * This function is called during the logout sequence.
+ */
+export const terminateUtilityProcesses = () => {
+    terminateMLProcessIfRunning();
+    terminateFFmpegProcessIfRunning();
+};
+
+/** The active ML utility process, if any. */
+let _utilityProcessML: UtilityProcess | undefined;
+
+/** The active FFmpeg utility process, if any. */
+let _utilityProcessFFmpeg: UtilityProcess | undefined;
+
+/**
+ * A promise to a comlink {@link Endpoint} that can be used to communicate with
+ * the active ffmpeg utility process (if any).
+ */
+let _utilityProcessFFmpegEndpoint: Promise<Endpoint> | undefined;
+
+/**
+ * Create a new utility process of the given {@link type}, terminating the older
+ * ones (if any).
+ *
+ * Currently the only type is "ml". The following note explains the reasoning
+ * why utility processes were used for the first workload (ML) that was handled
+ * this way. Similar reasoning applies to subsequent workloads (ffmpeg) that
+ * have been offloaded to utility processes in a slightly different manner to
+ * avoid stutter in the UI.
+ *
+ * [Note: ML IPC]
+ *
+ * The primary reason for doing ML tasks in the Node.js layer is so that we can
+ * use the binary ONNX runtime, which is 10-20x faster than the Wasm one that
+ * can be used directly on the web layer.
+ *
+ * For this to work, the main and renderer process need to communicate with each
+ * other. Further, in the web layer the ML indexing runs in a web worker (so as
+ * to not get in the way of the main thread). So the communication has 2 hops:
+ *
+ *     Node.js main <-> Renderer main <-> Renderer web worker
+ *
+ * This naive way works, but has a problem. The Node.js main process is in the
+ * code path for delivering user events to the renderer process. The ML tasks we
+ * do take in the order of 100-300 ms (possibly more) for each individual
+ * inference. Thus, the Node.js main process is busy for those 100-300 ms, and
+ * does not forward events to the renderer, causing the UI to jitter.
+ *
+ * The solution for this is to spawn an Electron UtilityProcess, which we can
+ * think of a regular Node.js child process. This frees up the Node.js main
+ * process, and would remove the jitter.
+ * https://www.electronjs.org/docs/latest/tutorial/process-model
+ *
+ * It would seem that this introduces another hop in our IPC
+ *
+ *     Node.js utility process <-> Node.js main <-> ...
+ *
+ * but here we can use the special bit about Electron utility processes that
+ * separates them from regular Node.js child processes: their support for
+ * message ports. https://www.electronjs.org/docs/latest/tutorial/message-ports
+ *
+ * As a brief summary, a MessagePort is a web feature that allows two contexts
+ * to communicate. A pair of message ports is called a message channel. The cool
+ * thing about these is that we can pass these ports themselves over IPC.
+ *
+ * > One caveat here is that the message ports can only be passed using the
+ * > `postMessage` APIs, not the usual send/invoke APIs.
+ *
+ * So we
+ *
+ * 1. In the utility process create a message channel.
+ * 2. Spawn a utility process, and send one port of the pair to it.
+ * 3. Send the other port of the pair to the renderer.
+ *
+ * The renderer will forward that port to the web worker that is coordinating
+ * the ML indexing on the web layer. Thereafter, the utility process and web
+ * worker can directly talk to each other!
+ *
+ *     Node.js utility process <-> Renderer web worker
+ *
+ * The RPC protocol is handled using comlink on both ends. The port itself needs
+ * to be relayed using `postMessage`.
+ */
+export const triggerCreateUtilityProcess = (
+    type: UtilityProcessType,
+    window: BrowserWindow,
+) => triggerCreateMLUtilityProcess(window);
+
+const terminateMLProcessIfRunning = () => {
+    if (_utilityProcessML) {
+        log.debug(() => "Terminating running ML utility process");
+        _utilityProcessML.kill();
+        _utilityProcessML = undefined;
+    }
+};
+
+export const triggerCreateMLUtilityProcess = (window: BrowserWindow) => {
+    terminateMLProcessIfRunning();
+
+    const { port1, port2 } = new MessageChannelMain();
+
+    const child = utilityProcess.fork(path.join(__dirname, "ml-worker.js"));
+    const userDataPath = app.getPath("userData");
+    child.postMessage(/* MLWorkerInitData */ { userDataPath }, [port1]);
+
+    window.webContents.postMessage("utilityProcessPort/ml", undefined, [port2]);
+
+    handleMessagesFromMLUtilityProcess(child);
+
+    _utilityProcessML = child;
+};
+
+/**
+ * Handle messages posted from the utility process.
+ *
+ * [Note: Using Electron APIs in UtilityProcess]
+ *
+ * Only a small subset of the Electron APIs are available to a UtilityProcess.
+ * As of writing (Jul 2024, Electron 30), only the following are available:
+ *
+ * - net
+ * - systemPreferences
+ *
+ * In particular, `app` is not available.
+ *
+ * We structure our code so that it doesn't need anything apart from `net`.
+ *
+ * For the other cases,
+ *
+ * -  Additional parameters to the utility process are passed alongwith the
+ *    initial message where we provide it the message port.
+ *
+ * -  When we need to communicate from the utility process to the main process,
+ *    we use the `parentPort` in the utility process.
+ */
+const handleMessagesFromMLUtilityProcess = (child: UtilityProcess) => {
+    child.on("message", (m: unknown) => {
+        if (processUtilityProcessLogMessage("[ml-worker]", m)) {
+            return;
+        }
+        log.info("Ignoring unknown message from ML utility process", m);
+    });
+};
+
+/**
+ * A comlink endpoint that can be used to communicate with the ffmpeg utility
+ * process. If there is no ffmpeg utility process, a new one is created on
+ * demand.
+ *
+ * See [Note: ML IPC] for a general outline of why utility processes are needed
+ * (tl;dr; to avoid stutter on the UI).
+ *
+ * In the case of ffmpeg, the IPC flow is a bit different: the utility process
+ * is not exposed to the web layer, and is internal to the node layer. The
+ * reason for this difference is that we need to create temporary files etc, and
+ * doing it a utility process requires access to the `app` module which are not
+ * accessible (See: [Note: Using Electron APIs in UtilityProcess]).
+ *
+ * There could've been possible reasonable workarounds, but the architecture
+ * we've adopted of three layers:
+ *
+ *     Renderer (web) <-> Node.js main <-> Node.js ffmpeg utility process
+ *
+ * The temporary file creation etc is handled in the Node.js main process, and
+ * paths to the files are forwarded to the ffmpeg utility process to act on.
+ *
+ * @returns an endpoint that can be used to communicate with the utility
+ * process. The utility process is expected to expose an object that conforms to
+ * the {@link ElectronFFmpegWorkerNode} interface on this endpoint.
+ */
+export const ffmpegUtilityProcessEndpoint = () =>
+    (_utilityProcessFFmpegEndpoint ??= createFFmpegUtilityProcessEndpoint());
+
+const terminateFFmpegProcessIfRunning = () => {
+    if (_utilityProcessFFmpeg) {
+        log.debug(() => "Terminating running FFmpeg utility process");
+        _utilityProcessFFmpeg.kill();
+        _utilityProcessFFmpeg = undefined;
+        _utilityProcessFFmpegEndpoint = undefined;
+    }
+};
+
+const createFFmpegUtilityProcessEndpoint = () => {
+    if (_utilityProcessFFmpeg) {
+        throw new Error("FFmpeg utility process is already running");
+    }
+
+    // Promise.withResolvers is currently in the node available to us.
+    let resolve: ((endpoint: Endpoint) => void) | undefined;
+    const promise = new Promise<Endpoint>((r) => (resolve = r));
+
+    const { port1, port2 } = new MessageChannelMain();
+
+    const child = utilityProcess.fork(path.join(__dirname, "ffmpeg-worker.js"));
+    // Send a handle to the port (one end of the message channel) to the utility
+    // process (alongwith any other init data). The utility process will reply
+    // with an "ack" when it get it.
+    const appVersion = app.getVersion();
+    child.postMessage(/* FFmpegWorkerInitData */ { appVersion }, [port1]);
+
+    child.on("message", (m: unknown) => {
+        if (m && typeof m == "object" && "method" in m) {
+            switch (m.method) {
+                case "ack":
+                    resolve!(messagePortMainEndpoint(port2));
+                    return;
+            }
+        }
+
+        if (processUtilityProcessLogMessage("[ffmpeg-worker]", m)) {
+            return;
+        }
+
+        log.info("Ignoring unknown message from ffmpeg utility process", m);
+    });
+
+    _utilityProcessFFmpeg = child;
+
+    // Resolve with the other end of the message channel (once we get an "ack"
+    // from the utility process).
+    return promise;
+};

desktop/src/main/stream.ts

@@ -7,13 +7,14 @@ import fs from "node:fs/promises";
 import { Writable } from "node:stream";
 import { pathToFileURL } from "node:url";
 import log from "./log";
-import { ffmpegConvertToMP4 } from "./services/ffmpeg";
+import { ffmpegUtilityProcess } from "./services/ffmpeg";
+import { type FFmpegGenerateHLSPlaylistAndSegmentsResult } from "./services/ffmpeg-worker";
 import { markClosableZip, openZip } from "./services/zip";
-import { ensure } from "./utils/common";
 import { writeStream } from "./utils/stream";
 import {
     deleteTempFile,
     deleteTempFileIgnoringErrors,
+    makeFileForStreamOrPathOrZipItem,
     makeTempFilePath,
 } from "./utils/temp";
 
@@ -57,25 +58,39 @@ const handleStreamRequest = async (request: Request): Promise<Response> => {
     const { host, searchParams } = new URL(url);
     switch (host) {
         case "read":
-            return handleRead(ensure(searchParams.get("path")));
+            return handleRead(searchParams.get("path")!);
 
         case "read-zip":
             return handleReadZip(
-                ensure(searchParams.get("zipPath")),
-                ensure(searchParams.get("entryName")),
+                searchParams.get("zipPath")!,
+                searchParams.get("entryName")!,
             );
 
         case "write":
-            return handleWrite(ensure(searchParams.get("path")), request);
+            return handleWrite(searchParams.get("path")!, request);
+
+        case "video": {
+            const op = searchParams.get("op");
+            if (op) {
+                switch (op) {
+                    case "convert-to-mp4":
+                        return handleConvertToMP4Write(request);
+                    case "generate-hls":
+                        return handleGenerateHLSWrite(request, searchParams);
+                    default:
+                        return new Response(`Unknown op ${op}`, {
+                            status: 404,
+                        });
+                }
+            }
 
-        case "convert-to-mp4": {
             const token = searchParams.get("token");
             const done = searchParams.get("done") !== null;
-            return token
-                ? done
-                    ? handleConvertToMP4ReadDone(token)
-                    : handleConvertToMP4Read(token)
-                : handleConvertToMP4Write(request);
+            if (!token) {
+                return new Response("Missing token", { status: 404 });
+            }
+
+            return done ? handleVideoDone(token) : handleVideoRead(token);
         }
 
         default:
@@ -105,6 +120,7 @@ const handleRead = async (path: string) => {
         res.headers.set("Content-Length", `${fileSize}`);
 
         // Add the file's last modified time (as epoch milliseconds).
+        // See: [Note: Integral last modified time]
         const mtimeMs = stat.mtime.getTime();
         res.headers.set("X-Last-Modified-Ms", `${mtimeMs}`);
     }
@@ -166,21 +182,21 @@ const handleReadZip = async (zipPath: string, entryName: string) => {
 };
 
 const handleWrite = async (path: string, request: Request) => {
-    await writeStream(path, ensure(request.body));
+    await writeStream(path, request.body!);
     return new Response("", { status: 200 });
 };
 
 /**
- * A map from token to file paths for convert-to-mp4 requests that we have
- * received.
+ * A map from token to file paths generated as a result of stream://video
+ * requests we have received.
  */
-const convertToMP4Results = new Map<string, string>();
+const pendingVideoResults = new Map<string, string>();
 
 /**
- * Clear any in-memory state for in-flight convert-to-mp4 requests. Meant to be
- * called during logout.
+ * Clear any in-memory state for in-flight streamed video processing requests.
+ * Meant to be called during logout.
  */
-export const clearConvertToMP4Results = () => convertToMP4Results.clear();
+export const clearPendingVideoResults = () => pendingVideoResults.clear();
 
 /**
  * [Note: Convert to MP4]
@@ -195,30 +211,32 @@ export const clearConvertToMP4Results = () => convertToMP4Results.clear();
  * mode for the Web fetch API). So we need to simulate that using two different
  * streaming requests.
  *
- *     renderer → main  stream://convert-to-mp4
+ *     renderer → main  stream://video?op=convert-to-mp4
  *                      → request.body is the original video
- *                      ← response is a token
+ *                      ← response is [token]
  *
- *     renderer → main  stream://convert-to-mp4?token=<token>
+ *     renderer → main  stream://video?token=<token>
  *                      ← response.body is the converted video
  *
- *     renderer → main  stream://convert-to-mp4?token=<token>&done
+ *     renderer → main  stream://video?token=<token>&done
  *                      ← 200 OK
  *
  * Note that the conversion itself is not streaming. The conversion still
- * happens in a single shot, we are just streaming the data across the IPC
- * boundary to allow us to pass large amounts of data without running out of
- * memory.
+ * happens in a single invocation of ffmpeg, we are just streaming the data
+ * across the IPC boundary to allow us to pass large amounts of data without
+ * running out of memory.
  *
  * See also: [Note: IPC streams]
  */
 const handleConvertToMP4Write = async (request: Request) => {
+    const worker = await ffmpegUtilityProcess();
+
     const inputTempFilePath = await makeTempFilePath();
-    await writeStream(inputTempFilePath, ensure(request.body));
+    await writeStream(inputTempFilePath, request.body!);
 
     const outputTempFilePath = await makeTempFilePath("mp4");
     try {
-        await ffmpegConvertToMP4(inputTempFilePath, outputTempFilePath);
+        await worker.ffmpegConvertToMP4(inputTempFilePath, outputTempFilePath);
     } catch (e) {
         log.error("Conversion to MP4 failed", e);
         await deleteTempFileIgnoringErrors(outputTempFilePath);
@@ -228,25 +246,112 @@ const handleConvertToMP4Write = async (request: Request) => {
     }
 
     const token = randomUUID();
-    convertToMP4Results.set(token, outputTempFilePath);
+    pendingVideoResults.set(token, outputTempFilePath);
     return new Response(token, { status: 200 });
 };
 
-const handleConvertToMP4Read = async (token: string) => {
-    const filePath = convertToMP4Results.get(token);
+const handleVideoRead = async (token: string) => {
+    const filePath = pendingVideoResults.get(token);
     if (!filePath)
         return new Response(`Unknown token ${token}`, { status: 404 });
 
     return net.fetch(pathToFileURL(filePath).toString());
 };
 
-const handleConvertToMP4ReadDone = async (token: string) => {
-    const filePath = convertToMP4Results.get(token);
+const handleVideoDone = async (token: string) => {
+    const filePath = pendingVideoResults.get(token);
     if (!filePath)
         return new Response(`Unknown token ${token}`, { status: 404 });
 
     await deleteTempFile(filePath);
 
-    convertToMP4Results.delete(token);
+    pendingVideoResults.delete(token);
     return new Response("", { status: 200 });
 };
+
+/**
+ * Generate a HLS playlist for the given video.
+ *
+ * See: [Note: Convert to MP4] for the general architecture of commands that do
+ * renderer <-> main I/O using streams.
+ *
+ * The difference here is that we the conversion generates two streams^ - one
+ * for the HLS playlist itself, and one for the file containing the encrypted
+ * and transcoded video chunks. The video stream we write to the pre-signed
+ * object upload URL(s), and then we return a JSON object containing the token
+ * for the playlist, and other metadata for use by the renderer.
+ *
+ * ^ if the video doesn't require a stream to be generated (e.g. it is very
+ *   small and already uses a compatible codec) then a HTT 204 is returned and
+ *   no stream is generated.
+ */
+const handleGenerateHLSWrite = async (
+    request: Request,
+    params: URLSearchParams,
+) => {
+    const fileID = parseInt(params.get("fileID") ?? "", 10);
+    const fetchURL = params.get("fetchURL");
+    const authToken = params.get("authToken");
+    if (!fileID || !fetchURL || !authToken) throw new Error("Missing params");
+
+    let inputItem: Parameters<typeof makeFileForStreamOrPathOrZipItem>[0];
+    const path = params.get("path");
+    if (path) {
+        inputItem = path;
+    } else {
+        const zipPath = params.get("zipPath");
+        const entryName = params.get("entryName");
+        if (zipPath && entryName) {
+            inputItem = [zipPath, entryName];
+        } else {
+            const body = request.body;
+            if (!body) throw new Error("Missing body");
+            inputItem = body;
+        }
+    }
+
+    const worker = await ffmpegUtilityProcess();
+
+    const {
+        path: inputFilePath,
+        isFileTemporary: isInputFileTemporary,
+        writeToTemporaryFile: writeToTemporaryInputFile,
+    } = await makeFileForStreamOrPathOrZipItem(inputItem);
+
+    const outputFilePathPrefix = await makeTempFilePath();
+    let result: FFmpegGenerateHLSPlaylistAndSegmentsResult | undefined;
+    try {
+        await writeToTemporaryInputFile();
+
+        result = await worker.ffmpegGenerateHLSPlaylistAndSegments(
+            inputFilePath,
+            outputFilePathPrefix,
+            fileID,
+            fetchURL,
+            authToken,
+        );
+
+        if (!result) {
+            // This video doesn't require stream generation.
+            return new Response(null, { status: 204 });
+        }
+
+        const { playlistPath, dimensions, videoSize, videoObjectID } = result;
+
+        const playlistToken = randomUUID();
+        pendingVideoResults.set(playlistToken, playlistPath);
+
+        return new Response(
+            JSON.stringify({
+                playlistToken,
+                dimensions,
+                videoSize,
+                videoObjectID,
+            }),
+            { status: 200 },
+        );
+    } finally {
+        if (isInputFileTemporary)
+            await deleteTempFileIgnoringErrors(inputFilePath);
+    }
+};

desktop/src/main/utils/comlink.ts

@@ -19,7 +19,7 @@ export const messagePortMainEndpoint = (mp: MessagePortMain): Endpoint => {
     const listeners = new WeakMap<NL, EL>();
     return {
         postMessage: (message, transfer) => {
-            mp.postMessage(message, transfer as unknown as MessagePortMain[]);
+            mp.postMessage(message, (transfer ?? []) as MessagePortMain[]);
         },
         addEventListener: (_, eh) => {
             const l: EL = (data) =>

desktop/src/main/utils/common.ts

@@ -5,20 +5,21 @@
  * currently a common package that both of them share.
  */
 
-/**
- * Throw an exception if the given value is `null` or `undefined`.
- */
-export const ensure = <T>(v: T | null | undefined): T => {
-    if (v === null) throw new Error("Required value was null");
-    if (v === undefined) throw new Error("Required value was not found");
-    return v;
-};
-
 /**
  * Wait for {@link ms} milliseconds
  *
  * This function is a promisified `setTimeout`. It returns a promise that
  * resolves after {@link ms} milliseconds.
+ *
+ * Duplicated from `web/packages/utils/promise.ts`.
  */
 export const wait = (ms: number) =>
     new Promise((resolve) => setTimeout(resolve, ms));
+
+/**
+ * Convert `null` to `undefined`, passthrough everything else unchanged.
+ *
+ * Duplicated from `web/packages/utils/transform.ts`.
+ */
+export const nullToUndefined = <T>(v: T | null | undefined): T | undefined =>
+    v === null ? undefined : v;

desktop/src/main/utils/exec-worker.ts

@@ -0,0 +1,23 @@
+import shellescape from "any-shell-escape";
+import { exec } from "node:child_process";
+import { promisify } from "node:util";
+import log from "../log-worker";
+
+/**
+ * Run a shell command asynchronously (utility process edition).
+ *
+ * This is an almost verbatim copy of {@link execAsync} from `electron.ts`,
+ * except it is meant to be usable from a utility process where only a subset of
+ * imports are available. See [Note: Using Electron APIs in UtilityProcess].
+ */
+export const execAsyncWorker = async (command: string | string[]) => {
+    const escapedCommand = Array.isArray(command)
+        ? shellescape(command)
+        : command;
+    const startTime = Date.now();
+    const result = await execAsync_(escapedCommand);
+    log.debugString(`${escapedCommand} (${Date.now() - startTime} ms)`);
+    return result;
+};
+
+const execAsync_ = promisify(exec);

desktop/src/main/utils/http.ts

@@ -0,0 +1,31 @@
+export const clientPackageName = "io.ente.photos.desktop";
+
+/**
+ * Reimplementation of {@link publicRequestHeaders} from the web source.
+ *
+ * @param desktopAppVersion The desktop app's version. This will get passed on
+ * as the "X-Client-Version" header.
+ *
+ * We cannot directly use `app.getVersion()` to obtain this value since the
+ * {@link app} module is not accessible to Electron utility processes which also
+ * calls this function.
+ */
+export const publicRequestHeaders = (desktopAppVersion: string) => ({
+    "X-Client-Package": clientPackageName,
+    "X-Client-Version": desktopAppVersion,
+});
+
+/**
+ * Reimplementation of {@link authenticatedRequestHeaders} from the web source.
+ *
+ * This builds on top of {@link publicRequestHeaders} and takes the same
+ * parameters, and additionally also requires the {@link authToken} that will be
+ * passed as the "X-Auth-Token" header.
+ */
+export const authenticatedRequestHeaders = (
+    desktopAppVersion: string,
+    authToken: string,
+) => ({
+    ...publicRequestHeaders(desktopAppVersion),
+    "X-Auth-Token": authToken,
+});

desktop/src/main/utils/temp.ts

@@ -5,7 +5,7 @@ import path from "node:path";
 import type { ZipItem } from "../../types/ipc";
 import log from "../log";
 import { markClosableZip, openZip } from "../services/zip";
-import { ensure } from "./common";
+import { writeStream } from "./stream";
 
 /**
  * Our very own directory within the system temp directory. Go crazy, but
@@ -20,17 +20,21 @@ const enteTempDirPath = async () => {
 /** Generate a random string suitable for being used as a file name prefix */
 const randomPrefix = () => {
     const ch = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
-    const randomChar = () => ensure(ch[Math.floor(Math.random() * ch.length)]);
+    const randomChar = () => ch[Math.floor(Math.random() * ch.length)]!;
 
     return Array(10).fill("").map(randomChar).join("");
 };
 
 /**
- * Return the path to a temporary file with the given {@link suffix}.
+ * Return the path to a temporary file with an optional {@link extension}.
  *
  * The function returns the path to a file in the system temp directory (in an
  * Ente specific folder therin) with a random prefix and an (optional)
- * {@link extension}.
+ * {@link extension}. The parent directory is guaranteed to exist.
+ *
+ * @param extension A string, if provided, is used as the extension for the
+ * generated path. It will be automatically prefixed by a dot, so don't include
+ * the dot in the provided string.
  *
  * It ensures that there is no existing item with the same name already.
  *
@@ -76,8 +80,8 @@ export const deleteTempFileIgnoringErrors = async (tempFilePath: string) => {
     }
 };
 
-/** The result of {@link makeFileForDataOrPathOrZipItem}. */
-interface FileForDataOrPathOrZipItem {
+/** The result of {@link makeFileForStreamOrPathOrZipItem}. */
+interface FileForStreamOrPathOrZipItem {
     /**
      * The path to the file (possibly temporary).
      */
@@ -101,33 +105,32 @@ interface FileForDataOrPathOrZipItem {
 /**
  * Return the path to a file, a boolean indicating if this is a temporary path
  * that needs to be deleted after processing, and a function to write the given
- * {@link dataOrPathOrZipItem} into that temporary file if needed.
+ * {@link item} into that temporary file if needed.
  *
- * @param dataOrPathOrZipItem The contents of the file, or the path to an
- * existing file, or a (path to a zip file, name of an entry within that zip
- * file) tuple.
+ * @param item A {@link ReadableStream} with the contents of the file, or the
+ * path to an existing file, or a (path to a zip file, name of an entry within
+ * that zip file) tuple.
  */
-export const makeFileForDataOrPathOrZipItem = async (
-    dataOrPathOrZipItem: Uint8Array | string | ZipItem,
-): Promise<FileForDataOrPathOrZipItem> => {
+export const makeFileForStreamOrPathOrZipItem = async (
+    item: ReadableStream | string | ZipItem,
+): Promise<FileForStreamOrPathOrZipItem> => {
     let path: string;
     let isFileTemporary: boolean;
     let writeToTemporaryFile = async () => {
         /* no-op */
     };
 
-    if (typeof dataOrPathOrZipItem == "string") {
-        path = dataOrPathOrZipItem;
+    if (typeof item == "string") {
+        path = item;
         isFileTemporary = false;
     } else {
         path = await makeTempFilePath();
         isFileTemporary = true;
-        if (dataOrPathOrZipItem instanceof Uint8Array) {
-            writeToTemporaryFile = () =>
-                fs.writeFile(path, dataOrPathOrZipItem);
+        if (item instanceof ReadableStream) {
+            writeToTemporaryFile = () => writeStream(path, item);
         } else {
             writeToTemporaryFile = async () => {
-                const [zipPath, entryName] = dataOrPathOrZipItem;
+                const [zipPath, entryName] = item;
                 const zip = openZip(zipPath);
                 try {
                     await zip.extract(entryName, path);

desktop/src/preload.ts

@@ -66,8 +66,10 @@ import type { IpcRendererEvent } from "electron";
 import type {
     AppUpdate,
     CollectionMapping,
+    FFmpegCommand,
     FolderWatch,
     PendingUploads,
+    UtilityProcessType,
     ZipItem,
 } from "./types/ipc";
 
@@ -111,10 +113,11 @@ const logout = () => {
     return ipcRenderer.invoke("logout");
 };
 
-const masterKeyB64 = () => ipcRenderer.invoke("masterKeyB64");
+const masterKeyFromSafeStorage = () =>
+    ipcRenderer.invoke("masterKeyFromSafeStorage");
 
-const saveMasterKeyB64 = (masterKeyB64: string) =>
-    ipcRenderer.invoke("saveMasterKeyB64", masterKeyB64);
+const saveMasterKeyInSafeStorage = (masterKey: string) =>
+    ipcRenderer.invoke("saveMasterKeyInSafeStorage", masterKey);
 
 const lastShownChangelogVersion = () =>
     ipcRenderer.invoke("lastShownChangelogVersion");
@@ -183,47 +186,53 @@ const fsWriteFileViaBackup = (path: string, contents: string) =>
 
 const fsIsDir = (dirPath: string) => ipcRenderer.invoke("fsIsDir", dirPath);
 
+const fsStatMtime = (path: string) => ipcRenderer.invoke("fsStatMtime", path);
+
 // - Conversion
 
 const convertToJPEG = (imageData: Uint8Array) =>
     ipcRenderer.invoke("convertToJPEG", imageData);
 
 const generateImageThumbnail = (
-    dataOrPathOrZipItem: Uint8Array | string | ZipItem,
+    pathOrZipItem: string | ZipItem,
     maxDimension: number,
     maxSize: number,
 ) =>
     ipcRenderer.invoke(
         "generateImageThumbnail",
-        dataOrPathOrZipItem,
+        pathOrZipItem,
         maxDimension,
         maxSize,
     );
 
 const ffmpegExec = (
-    command: string[],
-    dataOrPathOrZipItem: Uint8Array | string | ZipItem,
+    command: FFmpegCommand,
+    pathOrZipItem: string | ZipItem,
     outputFileExtension: string,
 ) =>
     ipcRenderer.invoke(
         "ffmpegExec",
         command,
-        dataOrPathOrZipItem,
+        pathOrZipItem,
         outputFileExtension,
     );
 
-// - ML
+const ffmpegDetermineVideoDuration = (pathOrZipItem: string | ZipItem) =>
+    ipcRenderer.invoke("ffmpegDetermineVideoDuration", pathOrZipItem);
 
-const createMLWorker = () => {
+// - Utility processes
+
+const triggerCreateUtilityProcess = (type: UtilityProcessType) => {
+    const portEvent = `utilityProcessPort/${type}`;
     const l = (event: IpcRendererEvent) => {
         void windowLoaded.then(() => {
             // "*"" is the origin to send to.
-            window.postMessage("createMLWorker/port", "*", event.ports);
-            ipcRenderer.off("createMLWorker/port", l);
+            window.postMessage(portEvent, "*", event.ports);
+            ipcRenderer.off(portEvent, l);
         });
     };
-    ipcRenderer.on("createMLWorker/port", l);
-    ipcRenderer.send("createMLWorker");
+    ipcRenderer.on(portEvent, l);
+    ipcRenderer.send("triggerCreateUtilityProcess", type);
 };
 
 // - Watch
@@ -289,11 +298,11 @@ const pendingUploads = () => ipcRenderer.invoke("pendingUploads");
 const setPendingUploads = (pendingUploads: PendingUploads) =>
     ipcRenderer.invoke("setPendingUploads", pendingUploads);
 
-const markUploadedFiles = (paths: PendingUploads["filePaths"]) =>
-    ipcRenderer.invoke("markUploadedFiles", paths);
+const markUploadedFile = (path: string, associatedPath?: string) =>
+    ipcRenderer.invoke("markUploadedFile", path, associatedPath);
 
-const markUploadedZipItems = (items: PendingUploads["zipItems"]) =>
-    ipcRenderer.invoke("markUploadedZipItems", items);
+const markUploadedZipItem = (item: ZipItem, associatedItem?: ZipItem) =>
+    ipcRenderer.invoke("markUploadedZipItem", item, associatedItem);
 
 const clearPendingUploads = () => ipcRenderer.invoke("clearPendingUploads");
 
@@ -350,8 +359,8 @@ contextBridge.exposeInMainWorld("electron", {
     selectDirectory,
     pathForFile,
     logout,
-    masterKeyB64,
-    saveMasterKeyB64,
+    masterKeyFromSafeStorage,
+    saveMasterKeyInSafeStorage,
     lastShownChangelogVersion,
     setLastShownChangelogVersion,
     isAutoLaunchEnabled,
@@ -378,6 +387,7 @@ contextBridge.exposeInMainWorld("electron", {
         writeFile: fsWriteFile,
         writeFileViaBackup: fsWriteFileViaBackup,
         isDir: fsIsDir,
+        statMtime: fsStatMtime,
         findFiles: fsFindFiles,
     },
 
@@ -386,10 +396,11 @@ contextBridge.exposeInMainWorld("electron", {
     convertToJPEG,
     generateImageThumbnail,
     ffmpegExec,
+    ffmpegDetermineVideoDuration,
 
     // - ML
 
-    createMLWorker,
+    triggerCreateUtilityProcess,
 
     // - Watch
 
@@ -410,7 +421,7 @@ contextBridge.exposeInMainWorld("electron", {
     pathOrZipItemSize,
     pendingUploads,
     setPendingUploads,
-    markUploadedFiles,
-    markUploadedZipItems,
+    markUploadedFile,
+    markUploadedZipItem,
     clearPendingUploads,
 });

desktop/src/types/ipc.ts

@@ -5,6 +5,8 @@
  * See [Note: types.ts <-> preload.ts <-> ipc.ts]
  */
 
+export type UtilityProcessType = "ml";
+
 export interface AppUpdate {
     autoUpdatable: boolean;
     version: string;
@@ -32,3 +34,5 @@ export interface PendingUploads {
     filePaths: string[];
     zipItems: ZipItem[];
 }
+
+export type FFmpegCommand = string[] | { default: string[]; hdr: string[] };

docs/docs/.vitepress/config.ts

@@ -8,13 +8,6 @@ export default defineConfig({
     head: [["link", { rel: "icon", type: "image/png", href: "/favicon.png" }]],
     cleanUrls: true,
     ignoreDeadLinks: "localhostLinks",
-    vite: {
-      build: {
-        rollupOptions: {
-          external: ['client-museum-s3.png'] // Added to handle static asset import
-        }
-      }
-    },
     themeConfig: {
         // We use the default theme (with some CSS color overrides). This
         // themeConfig block can be used to further customize the default theme.

docs/docs/.vitepress/sidebar.ts

@@ -52,6 +52,10 @@ export const sidebar = [
                         link: "/photos/features/machine-learning",
                     },
                     { text: "Map", link: "/photos/features/map" },
+                    {
+                        text: "Notifications",
+                        link: "/photos/features/notifications",
+                    },
                     {
                         text: "Passkeys",
                         link: "/photos/features/passkeys",
@@ -182,6 +186,7 @@ export const sidebar = [
         text: "Auth",
         items: [
             { text: "Introduction", link: "/auth/" },
+            { text: "Features", link: "/auth/features/" },
             {
                 text: "FAQ",
                 collapsed: true,
@@ -223,6 +228,7 @@ export const sidebar = [
             },
             {
                 text: "Troubleshooting",
+                collapsed: true,
                 items: [
                     {
                         text: "Windows login",
@@ -238,58 +244,91 @@ export const sidebar = [
         items: [
             { text: "Getting started", link: "/self-hosting/" },
             {
-                text: "System requirements",
-                link: "/self-hosting/guides/system-requirements",
+                text: "Connecting to custom server",
+                link: "/self-hosting/guides/custom-server/",
+            },
+            {
+                text: "Creating accounts",
+                link: "/self-hosting/creating-accounts",
+            },
+            {
+                text: "Configuring your server",
+                link: "/self-hosting/museum",
+            },
+            {
+                text: "Configuring S3",
+                link: "/self-hosting/guides/configuring-s3",
+            },
+            {
+                text: "Reverse proxy",
+                link: "/self-hosting/reverse-proxy",
             },
             {
                 text: "Guides",
+                collapsed: true,
                 items: [
                     { text: "Introduction", link: "/self-hosting/guides/" },
-                    {
-                        text: "Connect to custom server",
-                        link: "/self-hosting/guides/custom-server/",
-                    },
-                    {
-                        text: "Hosting the web app",
-                        link: "/self-hosting/guides/web-app",
-                    },
-                    {
-                        text: "Configuring S3",
-                        link: "/self-hosting/guides/configuring-s3",
-                    },
-                    {
-                        text: "Hosting Ente with external S3 (Community)",
-                        link: "/self-hosting/guides/external-s3",
-                    },
-                    {
-                        text: "DB migration",
-                        link: "/self-hosting/guides/db-migration",
-                    },
-                    {
-                        text: "Hosting Ente without Docker",
-                        link: "/self-hosting/guides/standalone-ente",
-                    },
-                    {
-                        text: "Ente via Tailscale (Community)",
-                        link: "/self-hosting/guides/Tailscale.md",
-                    },
-                    {
-                        text: "Configure CLI for Self Hosted Instance",
-                        link: "/self-hosting/guides/selfhost-cli",
-                    },
                     {
                         text: "Administering your server",
                         link: "/self-hosting/guides/admin",
                     },
-
                     {
-                        text: "Mobile build",
-                        link: "/self-hosting/guides/mobile-build",
+                        text: "Configuring CLI for your instance",
+                        link: "/self-hosting/guides/selfhost-cli",
+                    },
+                    {
+                        text: "Running Ente from source",
+                        link: "/self-hosting/guides/from-source",
+                    },
+                    {
+                        text: "Running Ente without Docker",
+                        link: "/self-hosting/guides/standalone-ente",
+                    },
+                ],
+            },
+            {
+                text: "Troubleshooting",
+                collapsed: true,
+                items: [
+                    {
+                        text: "General",
+                        link: "/self-hosting/troubleshooting/misc",
+                    },
+                    {
+                        text: "Bucket CORS",
+                        link: "/self-hosting/troubleshooting/bucket-cors",
+                    },
+                    {
+                        text: "Uploads",
+                        link: "/self-hosting/troubleshooting/uploads",
+                    },
+                    {
+                        text: "Docker / quickstart",
+                        link: "/self-hosting/troubleshooting/docker",
+                    },
+                    {
+                        text: "Ente CLI secrets",
+                        link: "/self-hosting/troubleshooting/keyring",
+                    },
+                ],
+            },
+            {
+                text: "Community Guides",
+                collapsed: true,
+                items: [
+                    {
+                        text: "Ente via Tailscale",
+                        link: "/self-hosting/guides/tailscale",
+                    },
+                    {
+                        text: "Ente with External S3",
+                        link: "/self-hosting/guides/external-s3",
                     },
                 ],
             },
             {
                 text: "FAQ",
+                collapsed: true,
                 items: [
                     { text: "General", link: "/self-hosting/faq/" },
                     {
@@ -304,41 +343,12 @@ export const sidebar = [
                         text: "Backups",
                         link: "/self-hosting/faq/backup",
                     },
-                ],
-            },
-            {
-                text: "Troubleshooting",
-                items: [
                     {
-                        text: "General",
-                        link: "/self-hosting/troubleshooting/misc",
-                    },
-                    {
-                        text: "Uploads",
-                        link: "/self-hosting/troubleshooting/uploads",
-                    },
-                    {
-                        text: "Docker / quickstart",
-                        link: "/self-hosting/troubleshooting/docker",
-                    },
-                    {
-                        text: "Yarn",
-                        link: "/self-hosting/troubleshooting/yarn",
-                    },
-                    {
-                        text: "Ente CLI Secrets",
-                        link: "/self-hosting/troubleshooting/keyring",
+                        text: "Environment variables",
+                        link: "/self-hosting/faq/environment",
                     },
                 ],
             },
         ],
     },
-    {
-        text: "About",
-        link: "/about/",
-    },
-    {
-        text: "Contribute",
-        link: "/about/contribute",
-    },
 ];

docs/docs/about/contribute.md

@@ -1,15 +0,0 @@
----
-title: Contribute
-description: Details about how to contribute to Ente's docs
----
-
-# Contributing
-
-To contribute to these docs, you can use the "Edit this page" button at the
-bottom of each page. This will allow you to directly edit the markdown file that
-is used to generate this documentation and open a quick pull request directly
-from GitHub's UI.
-
-If you're more comfortable in contributing with your text editor, see the
-`docs/` folder of our GitHub repository,
-[github.com/ente-io/ente](https://github.com/ente-io/ente).

docs/docs/about/index.md

@@ -1,72 +0,0 @@
----
-title: About Ente
-description: >
-    An overview of Ente: the company, and the people behind it, and the products
-    that we make.
----
-
-# About
-
-Ente is a end-to-end encrypted platform for privately, reliably, and securely
-storing your data on the cloud. On top of this platform, Ente offers two
-products:
-
-- **Ente Photos** - An alternative to Google Photos and Apple Photos
-
-- **Ente Auth** - A free 2FA alternative to Authy
-
-Both these apps are available for all desktop (Linux, Mac, Windows) and mobile
-(Android, iOS and F-Droid) platforms. They also work directly in your web
-browser without you needing to install anything.
-
-More products are in the pipeline.
-
-## History
-
-Ente was the founded by Vishnu Mohandas (he's also Ente's CEO) in response to
-privacy concerns with major tech companies. The underlying motivation was the
-understanding that big tech had no incentive to fix their act, but with
-end-to-end encrypted cross platform apps, there was a way for people to take
-back control over their own data without sacrificing on features.
-
-### What does Ente mean?
-
-In Malayalam, Vishnu's native language, "ente" means "mine". Thus "Ente Photos"
-has the literal meaning "my photos".
-
-This was a good name, but still Vishnu looked around for better ones. But one
-day, he discovered that "ente" means "duck" in German. This unexpected
-connection sealed the deal. We should ask him why he likes ducks so much, but
-apparently he does, so this dual meaning ("mine" / "duck") led him to finalize
-the name, and also led to the adoption of "Ducky", Ente's mascot:
-
-<div align="center">
-
-![Ente's mascot, Ducky](ducky.png){width=200px}
-
-</div>
-
-For the full origin story of Ducky you can check out
-[this blog post](https://ente.io/blog/ducky/).
-
-### How do I pronounce Ente?
-
-en-_tay_. Like ca<i>fe</i>.
-
-## Get in touch
-
-If you have a support query that is not answered by these docs, please reach out
-to our Customer Support by sending an email to support@ente.io
-
-To stay up to date with new product launches, and behind the scenes details of
-how we're building Ente, you can read our [blog](https://ente.io/blog) (or
-subscribe to it via [RSS](https://ente.io/blog/rss.xml))
-
-To suggest new features and/or offer your perspective on how we should design
-planned and upcoming features, use our
-[GitHub discussions](https://github.com/ente-io/ente/discussions)
-
-Or if you'd just like to hang out, join our
-[Discord](https://discord.gg/z2YVKkycX3), follow us on
-[Twitter](https://twitter.com/enteio) or give us a shout out on
-[Mastodon](https://mstdn.social/@ente)

docs/docs/auth/faq/index.md

@@ -41,6 +41,16 @@ Usually, this discrepancy occurs because the time in your browser might be
 incorrect. In particular, multiple users have reported that Firefox provides
 incorrect time when certain privacy settings are enabled.
 
+> [!TIP]
+>
+> Newer Ente Auth clients (upcoming 4.4.0+) will automatically try to correct
+> for incorrect system time, so you should be seeing correct codes even if your
+> system time is out of sync. However, this automatic correction will not work
+> if you're using Ente Auth in offline mode.
+>
+> If you've recently changed your system time and the codes are still incorrect,
+> try to refresh / restart the app if needed.
+
 ### Can I access my codes on web?
 
 You can access your codes on the web at [auth.ente.io](https://auth.ente.io).

docs/docs/auth/features/index.md

@@ -0,0 +1,139 @@
+---
+title: Features - Auth
+description: Features available in Ente Auth
+---
+
+# Features
+
+This page outlines the key features available in Ente Auth.
+
+### Icons
+
+Ente Auth supports the icon pack provided by
+[simple-icons](https://github.com/simple-icons/simple-icons). If an icon you
+need is missing, please refer to the
+[docs/adding-icons](https://github.com/ente-io/ente/blob/main/mobile/apps/auth/docs/adding-icons.md)
+guide for instructions on how to contribute.
+
+### Search
+
+Quickly find your codes by searching based on issuer or account name. You can
+also configure the app to focus the search bar automatically on app start by
+going to **Settings → General → Focus search on app start**.
+
+### Tags
+
+Organize and filter your codes with ease using tags.
+
+- **Creating a Tag:** When adding or editing a code, tap the orange (+) icon to
+  create a new tag.
+- **Adding an existing Tag:** When adding or editing a code, select the desired
+  tag from the list.
+
+### Pinning
+
+Highlight your frequently used services by pinning them to the top of your code
+list. To pin a code, long-press (mobile) or right-click (desktop) the code and
+select "Pin".
+
+### Notes
+
+Add additional information to your codes using notes. Notes can be added during
+the process of creating or modifying a code.
+
+### Sharing
+
+Securely share codes temporarily with others.
+
+- Long-press (mobile) or right-click (desktop) on a code and choose "Share".
+- Select a duration for the shared link: 2 minutes, 5 minutes, or 10 minutes.
+- This generates a unique, time-limited link. Recipients can view the codes for
+  the specified duration without gaining access to the underlying secret key.
+  After the link expires, the recipients will no longer be able to view new
+  codes.
+
+### Custom sorting
+
+Customize the order in which your codes are displayed. Ente Auth provides
+several sorting options:
+
+- Issuer name
+- Account name
+- Frequently used
+- Recently used
+- Manual (custom drag-and-drop order)
+
+Access the sort menu in the top-right corner (next to the search icon) to change
+your sorting preference.
+
+### Offline mode
+
+Ente Auth can be used entirely offline. Choose "Use without backups" on the
+login screen. In this mode, your codes are stored locally on your device.
+
+Unlike when using an account, data is not synced or backed up to the cloud. You
+are responsible for manually backing up your codes.
+
+### Display options
+
+Customize how your codes are displayed for optimal usability.
+
+- **Show large icons:** Display codes with larger icons for enhanced visibility.
+- **Compact mode:** Switch to a more compact layout to view more codes on the
+  screen simultaneously.
+- **Hide codes:** Hide the actual code values for extra privacy. Double-tap a
+  code to reveal it when needed.
+
+### App lock
+
+Add an additional layer of protection using the app lock. Choose from the
+following lock methods:
+
+- **Device lock:** Use the existing lock configured on your device (e.g., Face
+  ID, Touch ID, system password).
+- **PIN lock:** Set up a 4-digit PIN code to unlock the app.
+- **Password lock:** Set up a password to unlock the app.
+
+Additionally, configure **Auto lock** to automatically lock the app after a
+specified period of time (options: Immediately, 5s, 15s, 1m, 5m, 30m).
+
+### Import / Export
+
+Ente Auth offers various import and export options for your codes.
+
+- **Export:** Export your codes in plain text, as an encrypted file, or
+  automatically via the CLI.
+- **Import:** Import codes from various other authentication apps.
+
+For detailed instructions, refer to the
+[migration guides](../migration-guides/).
+
+### Deduplicate codes
+
+If you import codes and end up with duplicates, you can easily remove them. Go
+to **Settings → Data → Duplicate codes** to find and remove duplicate codes.
+
+### Trash
+
+Manage unwanted codes by moving them to the Trash. The Trash is not cleared
+automatically, giving you the flexibility to restore or permanently delete codes
+at any time.
+
+- **Trashing a code:** Long-press (mobile) or right-click (desktop) on a code
+  and select "Trash" to move it to the Trash.
+- **Viewing trashed codes:** If you have trashed codes, you can view them by
+  selecting the Trash tag.
+- **Managing trashed codes:** In the Trash view, you can either permanently
+  delete codes or restore them back to your main list.
+
+### Scan QR
+
+Easily add or share entries using QR codes:
+
+- **Add by scanning (mobile):** On mobile, you can add a new entry by scanning
+  the QR code provided by the service. This quickly adds the entry to Ente Auth.
+- **Show entry as QR code:** On all apps, you can long-press (mobile) or
+  right-click (desktop) a code and select "QR". This allows you to easily share
+  the complete entry (including the secret) with others by letting them scan the
+  displayed QR code. This can also be used to easily add the same entry to
+  another authenticator app or service.

docs/docs/auth/migration-guides/authy/index.md

@@ -10,8 +10,9 @@ A guide written by Green, an ente.io lover
 > [!WARNING]
 >
 > Authy has dropped all support for its desktop apps. It is no longer possible
-> to export data from Authy using methods 1 and 2. You will need either an iOS device
-> and computer (method 4) or a rooted Android phone (method 3) to follow this guide.
+> to export data from Authy using methods 1 and 2. You will need either an iOS
+> device and computer (method 4) or a rooted Android phone (method 3) to follow
+> this guide.
 
 ---
 
@@ -204,11 +205,24 @@ This uses the tool [Aegis Authenticator](https://getaegis.app/) from
 
 ## Method 4: Authy-iOS-MiTM
 
-**Who should use this?** Technical iOS users of Authy that cannot export their tokens with methods 1 or 2 (due to those methods being patched) or method 3 (due to that method requiring a rooted Android device).
+**Who should use this?** Technical iOS users of Authy that cannot export their
+tokens with methods 1 or 2 (due to those methods being patched) or method 3 (due
+to that method requiring a rooted Android device).
 
-This method works by intercepting the data the Authy app receives while logging in for the first time, which contains your encrypted authenticator tokens. After the encrypted authenticator tokens are dumped, you can decrypt them using your backup password and convert them to an Ente token file.
+This method works by intercepting the data the Authy app receives while logging
+in for the first time, which contains your encrypted authenticator tokens. After
+the encrypted authenticator tokens are dumped, you can decrypt them using your
+backup password and convert them to an Ente token file.
 
-For an up-to-date guide of how to retrieve the encrypted authenticator tokens and decrypt them, please see [Authy-iOS-MiTM](https://github.com/AlexTech01/Authy-iOS-MiTM). To convert the `decrypted_tokens.json` file from that guide into a format Ente Authenticator can recognize, use [this](https://gist.github.com/gboudreau/94bb0c11a6209c82418d01a59d958c93?permalink_comment_id=5317087#gistcomment-5317087) Python script. Once you have the `ente_auth_import.plain` file from that script, transfer it to your device and follow the instructions below to import it into Ente Authenticator.
+For an up-to-date guide of how to retrieve the encrypted authenticator tokens
+and decrypt them, please see
+[Authy-iOS-MiTM](https://github.com/AlexTech01/Authy-iOS-MiTM). To convert the
+`decrypted_tokens.json` file from that guide into a format Ente Authenticator
+can recognize, use
+[this](https://gist.github.com/gboudreau/94bb0c11a6209c82418d01a59d958c93?permalink_comment_id=5317087#gistcomment-5317087)
+Python script. Once you have the `ente_auth_import.plain` file from that script,
+transfer it to your device and follow the instructions below to import it into
+Ente Authenticator.
 
 ## Importing to Ente Authenticator (Method 1, method 2.1, method 4)
 

docs/docs/de/auth/index.md

@@ -10,4 +10,4 @@ Ende-zu-Ende-verschlüsselte Authenticator-App für jedermann. Wir sind froh, da
 du hier bist!
 
 **Please note that this German translation is currently just a placeholder.**
-Know German? [Help us fill this in!](/about/contribute).
+Know German? [Help us fill this in!](/#contribute).

docs/docs/index.md

@@ -1,15 +1,82 @@
 ---
 title: Home
+description: >
+    Introduction to Ente: Products, Community and Support
 ---
 
 # Welcome!
 
-This site contains documentation and help for Ente Photos and Ente Auth. It
-describes various features, and also offers various troubleshooting suggestions.
+![Ducky: Ente's Mascot](/public/ducky.png){width=50% style="margin: 0 auto"}
 
-Use the **sidebar** menu to navigate to information about the product (Photos or
-Auth) you'd like to know more about. Or use the **search** at the top to try and
-jump directly to page that might contain the information you need.
+## Introduction
 
-To know more about Ente, see [about](/about/) or visit our website
-[ente.io](https://ente.io).
+Ente (pronounced en-_tay_) is a end-to-end encrypted platform for privately,
+reliably, and securely storing your data on the cloud, over which 2 applications
+have been developed and made available for mobile, web and desktop, namely:
+
+- **Ente Photos** - An alternative to Google Photos and Apple Photos
+- **Ente Auth** - A free 2FA alternative to Authy
+
+## History
+
+Ente was the founded by Vishnu Mohandas (he's also Ente's CEO) in response to
+privacy concerns with major tech companies. The underlying motivation was the
+understanding that big tech had no incentive to fix their act, but with
+end-to-end encrypted cross platform apps, there was a way for people to take
+back control over their own data without sacrificing on features.
+
+### Origin of the name
+
+In Malayalam, Vishnu's native language, "ente" means "mine", thus "Ente Photos"
+literally means "my photos".
+
+But one day, he discovered that "ente" means "duck" in German. This unexpected
+connection sealed the deal after looking for alternative names and led to the
+adoption of ["Ducky"](https://ente.io/blog/ducky/), representing the playfulness
+and friendly nature of the community and team.
+
+## Getting Started
+
+We recommend reading the documentation for [Ente Photos](/photos/) or
+[Ente Auth](/auth/) to get started with installation on the desired platform,
+explore available features and usage.
+
+If you are looking to self-host Ente, we recommend you to read the
+[official documentation](/self-hosting/) for updated information on getting
+started, installation, administration and maintenance.
+
+## Contributing
+
+There are many ways to support Ente and you don't have to be a programmer for
+that. You can spread the word, give feedback, report bugs, help us with
+translations, contribute documentation and community guides and more.
+
+To suggest new features and/or offer your perspective on how we should design
+(planned and upcoming features), use our
+[GitHub discussions](https://github.com/ente-io/ente/discussions)
+
+You can find our contribution guidelines
+[here](https://github.com/ente-io/ente/blob/main/CONTRIBUTING.md).
+
+You can always engage with our community and team to hang out, answer queries
+and stay updated:
+
+- Chat: [Discord](https://ente.io/discord)
+- Discussions: [GitHub](https://github.com/ente-io/ente/discussions)
+- Socials:
+    - Twitter: [enteio](https://twitter.com/enteio)
+    - Mastodon: [@ente@fosstodon.org](https://fosstodon.org/@ente)
+    - Bluesky: [ente.io](https://bsky.app/profile/ente.io)
+    - Instagram: [ente.app](https://www.instagram.com/ente.app)
+- Website:
+    - [Blog](https://ente.io/blog)
+    - [RSS](https://ente.io/blog/rss.xml)
+
+## Getting Help
+
+If you encounter any issues with any of the products that's not answered by our
+documentation, please reach out to our team by sending an email to
+[support@ente.io](mailto:support@ente.io)
+
+For community support, please post your queries on
+[Discord](https://discord.gg/z2YVKkycX3)

docs/docs/photos/faq/desktop.md

@@ -1,6 +1,7 @@
 ---
 title: Desktop app FAQ
-description: An assortment of frequently asked questions about Ente Photos desktop app
+description:
+    An assortment of frequently asked questions about Ente Photos desktop app
 ---
 
 # Desktop app FAQ
@@ -15,7 +16,8 @@ to manually update the software.
 
 ### Upload errors
 
-**How do I identify which files experienced upload issues within the desktop app?**
+**How do I identify which files experienced upload issues within the desktop
+app?**
 
 Check the sections within the upload progress bar for "Failed Uploads," "Ignored
 Uploads," and "Unsuccessful Uploads."
@@ -33,6 +35,5 @@ be specific to your distro (e.g. `xdg-desktop-menu forceupdate`).
 
 > [!NOTE]
 >
-> If you're using an AppImage and not seeing the icon, you'll need to [enable
-> AppImage desktop
-> integration](/photos/troubleshooting/desktop-install/#appimage-desktop-integration).
+> If you're using an AppImage and not seeing the icon, you'll need to
+> [enable AppImage desktop integration](/photos/troubleshooting/desktop-install/#appimage-desktop-integration).

docs/docs/photos/faq/export.md

@@ -7,9 +7,10 @@ description: Frequently asked questions about keeping extra backups of your data
 
 ## How can I backup my data in a local drive outside Ente?
 
-You can use our CLI tool or our desktop app to set up exports of your data
-to your local drive. This way, you can use Ente in your day to day use, with an additional guarantee that a copy of your original photos and videos are
-always available on your machine.
+You can use our CLI tool or our desktop app to set up exports of your data to
+your local drive. This way, you can use Ente in your day to day use, with an
+additional guarantee that a copy of your original photos and videos are always
+available on your machine.
 
 - You can use [Ente's CLI](https://github.com/ente-io/ente/tree/main/cli#export)
   to export your data in a cron job to a location of your choice. The exports
@@ -21,7 +22,7 @@ always available on your machine.
   background without you needing to run any other cron jobs. See
   [migration/export](/photos/migration/export/) for more details.
 
-## Does the exported data preserve folder structure?
+## Does the exported data preserve album structure?
 
 Yes. When you export your data for local backup, it will maintain the exact
 album structure how you have set up within Ente.

docs/docs/photos/faq/face-recognition.md

@@ -1,7 +1,6 @@
 ---
 title: Face recognition
-description:
-    Frequently asked questions about Ente's face recognition
+description: Frequently asked questions about Ente's face recognition
 ---
 
 # Face recognition

docs/docs/photos/faq/general.md

@@ -26,10 +26,6 @@ unsupported file format and we will do our best to help you out.
 
 Yes, we currently do not support files larger than 4 GB.
 
-If this constraint is a concern for you, please write to
-[support@ente.io](mailto:support@ente.io) with your use case and we will do our
-best to help you.
-
 ## Does Ente support videos?
 
 Ente supports backing up and downloading of videos in their original format and
@@ -104,29 +100,53 @@ clicking on "Your map" under "Locations" on the search screen.
 
 ## How to reset my password if I lost it?
 
-On the login page, enter your email and click on Forgot Password. Then, enter your recovery key and create a new password.
+On the login page, enter your email and click on Forgot Password. Then, enter
+your recovery key and create a new password.
 
- # iOS Album Backup and Organization in Ente
+## Can I search for photos using the descriptions I’ve added?
 
- ### How does Ente handle photos that are part of multiple iOS albums?
-When you select multiple albums for backup, Ente prioritizes uploading each photo to the album with the fewest photos. This means a photo will only be uploaded once, even if it exists in multiple albums on your device. If you create new albums on your device after the initial backup, those photos may not appear in the corresponding Ente album if they were already uploaded to a different album.
+Yes, descriptions are searchable, making it easier to find specific photos
+later. To do this, open the photo, tap the (i) button, and enter your
+description.
 
+## How does the deduplication feature work on the desktop app?
 
-### Why don’t all photos from a new iOS album appear in the corresponding Ente album?
-If you create a new album on your device after the initial backup, the photos in that album may have already been uploaded to another album in Ente. To fix this, go to the "On Device" album in Ente, select all photos, and manually add them to the corresponding album in Ente.
+If the app finds exact duplicates, it will show them in the deduplication. When
+you delete a duplicate, the app keeps one copy and creates a symlink for the
+other duplicate. This helps save storage space.
 
-### What happens if I reorganize my photos in the iOS Photos app after backing up?
-Reorganizing photos in the iOS Photos app (e.g., moving photos to new albums) won’t automatically reflect in Ente. You’ll need to manually add those photos to the corresponding albums in Ente to maintain consistency.
+## What happens if I lose access to my email address? Can I use my recovery key to bypass email verification?
 
-### Can I search for photos using the descriptions I’ve added?
-Yes, descriptions are searchable, making it easier to find specific photos later.
-To do this, open the photo, tap the (i) button, and enter your description.
-
-### How does the deduplication feature work on the desktop app?
-If the app finds exact duplicates, it will show them in the deduplication. When you delete a duplicate, the app keeps one copy and creates a symlink for the other duplicate. This helps save storage space.
-
-### What happens if I lose access to my email address? Can I use my recovery key to bypass email verification?
-No, the recovery key does not bypass email verification. For security reasons, we do not disable or bypass email verification unless the account owner reaches out to us and successfully verifies their identity by providing details about their account.
+No, the recovery key does not bypass email verification. For security reasons,
+we do not disable or bypass email verification unless the account owner reaches
+out to us and successfully verifies their identity by providing details about
+their account.
 
 If you lose access to your email, please contact our support team at
 support@ente.io
+
+---
+
+# iOS Album Backup and Organization in Ente
+
+## How does Ente handle photos that are part of multiple iOS albums?
+
+When you select multiple albums for backup, Ente prioritizes uploading each
+photo to the album with the fewest photos. This means a photo will only be
+uploaded once, even if it exists in multiple albums on your device. If you
+create new albums on your device after the initial backup, those photos may not
+appear in the corresponding Ente album if they were already uploaded to a
+different album.
+
+## Why don’t all photos from a new iOS album appear in the corresponding Ente album?
+
+If you create a new album on your device after the initial backup, the photos in
+that album may have already been uploaded to another album in Ente. To fix this,
+go to the "On Device" album in Ente, select all photos, and manually add them to
+the corresponding album in Ente.
+
+## What happens if I reorganize my photos in the iOS Photos app after backing up?
+
+Reorganizing photos in the iOS Photos app (e.g., moving photos to new albums)
+won’t automatically reflect in Ente. You’ll need to manually add those photos to
+the corresponding albums in Ente to maintain consistency.

docs/docs/photos/faq/metadata.md

@@ -62,6 +62,7 @@ the upload time as the photo's creation time.
 ## Modifications
 
 Ente supports modifications to the following metadata:
+
 - File name
 - Date & time
 - Location

docs/docs/photos/faq/video-streaming.md

@@ -1,29 +1,49 @@
 ---
 title: Video streaming FAQ
-description:
-    Frequently asked questions about Ente's video streaming feature
+description: Frequently asked questions about Ente's video streaming feature
 ---
 
 # Video streaming
 
 > [!NOTE]
 >
-> Video streaming is available in beta on mobile apps starting v0.9.98.
+> Video streaming is available in beta on mobile apps starting v0.9.98 and on
+> desktop starting v1.7.13.
 
 ### How to enable video streaming?
 
+#### On mobile
+
 - Open Settings -> General -> Advanced
-- Switch on the toggle for `Video streaming`
+- Enable the toggle for `Streamable videos`
+
+#### On desktop
+
+- Open Settings -> Preferences
+- Enable the toggle for `Streamable videos`
 
 ### What happens when I enable video streaming?
 
+#### On mobile
+
 Enabling video streaming will start processing videos captured in the last 30
 days, generating streams for each. Both local and remote videos will be
 processed, so this may consume bandwidth for downloading of remote files and
 uploading of the generated streams.
 
+#### On desktop
+
+When enabled, the desktop app will generate streams both for new uploads, and
+also for all existing videos that were previously uploaded.
+
+Stream generation is CPU intensive and can take time so the app will continue
+processing them in the background. Clicking on search bar will show "Processing
+videos..." when stream generation is happening.
+
 ### How can I view video streams?
 
+### On mobile
+
 Settings -> Backup > Backup status will show details regarding the processing
 status for videos. Processed videos will have a green play button next to them.
 You can open these videos by tapping on them.
@@ -34,6 +54,12 @@ play the stream.
 Clicking on the `Info` icon within the original video will show details about
 the generated stream.
 
+### On desktop and web
+
+Desktop and web app will automatically play the streaming version of a video if
+it has been already generated. The quality selector will show "Auto" when
+playing the stream.
+
 ### What is a stream?
 
 Stream is an encrypted HLS file with an `.m3u8` playlist that helps play a video
@@ -51,6 +77,7 @@ generated stream.
 While this feature is in beta, we will not count the storage consumed by your
 streams against your storage quota. This may change in the future. If it does,
 we will provide an option to opt-in to one of the following:
+
 1. Original videos only
 2. Compressed streams only
 3. Both

docs/docs/photos/features/background.md

@@ -43,8 +43,8 @@ need to disable this "Optimize battery usage" mode in the system settings for
 Ente if you wish for Ente to automatically back up your photos in the
 background.
 
-On Android versions 15 and later, if an app is in private space and the private 
-space is locked, Android doesn’t allow the app to run any background processes. 
+On Android versions 15 and later, if an app is in private space and the private
+space is locked, Android doesn’t allow the app to run any background processes.
 As a result, background sync will not work.
 
 ### Desktop

docs/docs/photos/features/deduplicate.md

@@ -52,6 +52,11 @@ Ente also provides a tool for manual de-duplication in _Settings → Backup →
 Remove duplicates_. This is useful if you have an existing library with
 duplicates across different albums, but wish to keep only one copy.
 
+During this operation, Ente will discard duplicates across all albums, retain a
+single copy, and add symlinks to this copy within all existing albums. So your
+existing album structure remains unchanged, while the space consumed by the
+duplicate data is freed up.
+
 ## Adding to Ente album creates symlinks
 
 Note that once a file is in Ente, adding it to another Ente album will create a

docs/docs/photos/features/family-plans.md

@@ -24,19 +24,19 @@ In brief,
 
 ## Storage Limits
 
-If you're an admin of a family, you will be able to set storage limits for the 
+If you're an admin of a family, you will be able to set storage limits for the
 members in your family plan.
 
-In brief, 
+In brief,
 
-- For example, once you set a limit of 10GB for a member, their Storage
-  quota for uploading photos will be limited to 10GB.
+- For example, once you set a limit of 10GB for a member, their Storage quota
+  for uploading photos will be limited to 10GB.
 
-- Once the invited member accepts the Family invite, you will be able to see 
-  an edit icon in the Members List. Click on it to setup a family limit.
+- Once the invited member accepts the Family invite, you will be able to see an
+  edit icon in the Members List. Click on it to setup a family limit.
 
 - If the admin has set a limit for any user, that limit value will be prefilled
-  in the input box. 
+  in the input box.
 
-- If you want to remove any storage limit from a members account, you 
-  can click on the "Remove Limit" and they can upload photos without any limit. 
+- If you want to remove any storage limit from a members account, you can click
+  on the "Remove Limit" and they can upload photos without any limit.

docs/docs/photos/features/machine-learning.md

@@ -47,8 +47,20 @@ device.
 The indexes are synced across all your devices automatically using the same
 end-to-end encrypted security that we use for syncing your photos.
 
-Note that the desktop app does not currently support modifying the face
-groupings, that is only supported by the mobile app.
+---
+
+#### Local indexing on mobile
+
+In general the machine learning is optimized to work well on most mobile device.
+However, devices with low RAM (4-6GB) and large photo libraries might struggle
+to complete the indexing without affecting performance of the app. In such case,
+you might want to disable local indexing and let the desktop run it instead.
+
+You can disable local indexing from the settings, under
+`General > Advanced > Machine learning > Configuration`. This way, you can
+continue to use the ML features without your phone performance taking any hit.
+
+---
 
 For more information on how to use Machine Learning for face recognition please
 check out [the FAQ](../faq/face-recognition).

docs/docs/photos/features/notifications.md

@@ -0,0 +1,33 @@
+---
+title: Notifications
+description: Details about notifications in Ente
+---
+
+# Notifications
+
+The Ente app can send notifications to notify you of an update, or just to
+remind you of some sweet or helpful memory at the right time.
+
+## New shared photos
+
+Receive notifications when someone adds a photo to a shared album that you're a
+part of.
+
+## "On this day" memories
+
+Receive reminders about memories from this day in previous years. These
+reminders will only be shown if there are enough photos taken across previous
+years of the specific day.
+
+## Birthday notifications
+
+Receive reminders when it's someone's birthday. Tapping on the notification will
+take you to photos of the birthday person. This requires you to first add a
+birthday to a person, and will only be shown if there are enough photos of that
+person.
+
+## Notification permission
+
+By default all notification categories are enabled if you give notification
+permission. You can disable all of the above notification categories from
+`Settings > Notifications`. Notifications currently only work on mobile.

docs/docs/photos/features/trash.md

@@ -6,7 +6,7 @@ description: Deleting items and trash
 # Trash
 
 Whenever you delete an item from Ente, it is moved to Trash. These items will be
-automatically deleted from Trash after 30 days. You can manaully select photos
+automatically deleted from Trash after 30 days. You can manually select photos
 to permanently delete or completely empty the trash if you wish.
 
 Items in trash are included in your used storage calculation.

docs/docs/photos/migration/export/index.md

@@ -66,5 +66,4 @@ If you run into any issues during your data export, please reach out to
 
 Note that we also provide a
 [CLI tool](https://github.com/ente-io/ente/tree/main/cli#export) to export your
-data. You can find more information about the export in the
-[export FAQ](/photos/faq/export).
+data.

docs/docs/photos/troubleshooting/nas.md

@@ -14,9 +14,21 @@ directly stream chunks of Google Takeout zips that are stored on network drives.
 
 In particular, the folder watch functionality suffers a lot since the app needs
 access to file system events to detect changes to the users files so that they
-can be uploaded whenever there are changes.
+can be uploaded whenever there are changes. Network drives are less reliable in
+providing these file change events correctly.
 
 Since are high chances of the user having a subpar experience, we request
 customers to avoid using the desktop app directly with network attached storage
 and instead temporarily copy the files to their local storage for uploads, and
 avoid watching folders that live on a network drive.
+
+## Exporting to UNC paths
+
+Generally, exports are likely to work better than imports, since the interaction
+with the file system is relatively simpler (Note that the app still needs to
+scan the folder to find existing files, esp. if the continuous export option is
+enabled).
+
+A special case is when exporting to a UNC path. In this case, the file
+separators will not work as expected and the export will not start. As a
+workaround, you can map your UNC path to a network drive and use that instead.

docs/docs/self-hosting/creating-accounts.md

@@ -0,0 +1,27 @@
+---
+title: Creating accounts
+description: Creating accounts on your deployment
+---
+
+# Creating accounts
+
+Once Ente is up and running, the Ente Photos web app will be accessible on
+`http://localhost:3000`. Open this URL in your browser and proceed with creating
+an account.
+
+The default API endpoint for museum will be `localhost:8080`.
+
+![endpoint](/endpoint.png)
+
+To complete your account registration you will need to enter a 6-digit
+verification code.
+
+This code can be found in the server logs, which should already be shown in your
+quickstart terminal. Alternatively, you can open the server logs with the
+following command from inside the `my-ente` folder:
+
+```sh
+sudo docker compose logs
+```
+
+![otp](/otp.png)

docs/docs/self-hosting/faq/environment.md

@@ -0,0 +1,52 @@
+---
+title: "Environment Variables and Ports"
+description:
+    "Information about all the Environment Variables needed to run Ente"
+---
+
+# Environment variables and ports
+
+A self-hosted Ente instance requires specific endpoints in both Museum (the
+server) and web apps. This document outlines the essential environment variables
+and port mappings of the web apps.
+
+Here's the list of important variables that a self hoster should know about:
+
+### Museum
+
+1. `NEXT_PUBLIC_ENTE_ENDPOINT`
+
+The above environment variable is used to configure Museums endpoint. Where
+Museum is running and which port it is listening on. This endpoint should be
+configured for all the apps to connect to your self hosted endpoint.
+
+All the apps (regardless of platform) by default connect to api.ente.io - which
+is our production instance of Museum.
+
+### Web Apps
+
+> [!IMPORTANT] Web apps don't need to be configured with the below endpoints.
+> Web app environment variables are being documented here just so that the users
+> know everything in detail. Checkout
+> [Configuring your Server](/self-hosting/museum) to configure endpoints for
+> particular app.
+
+In Ente, all the web apps are separate NextJS applications. Therefore, they are
+all configured via environment variables. The photos app (Ente Photos) has
+information about and connects to other web apps like albums, cast, etc.
+
+1. `NEXT_PUBLIC_ENTE_ALBUMS_ENDPOINT`
+
+This environment variable is used to configure and declare the endpoint for the
+Albums web app.
+
+## Ports
+
+The below format is according to how ports are mapped in Docker.
+Typically,`<host>:<container-port>`
+
+1. `8080:8080`: Museum (Ente's server)
+2. `3000:3000`: Ente Photos web app
+3. `3001:3001`: Ente Accounts web app
+4. `3003:3003`: [Ente Auth web app](https://ente.io/auth/)
+5. `3004:3004`: [Ente Cast web app](http://ente.io/cast)

docs/docs/self-hosting/faq/otp.md

@@ -12,6 +12,19 @@ verification code by:
 
 - Reading it from the DB (otts table)
 
+The easiest option when getting started is to look for it in the server (museum)
+logs. If you're already running the docker compose cluster using the quickstart
+script, you should be already seeing the logs in your terminal. Otherwise you
+can go to the folder (e.g. `my-ente`) where your `compose.yaml` is, then run
+`docker compose logs museum --follow`. Once you can see the logs, look for a
+line like:
+
+```
+... Skipping sending email to email@example.com: *Verification code: 112089*
+```
+
+That is the verification code.
+
 > [!TIP]
 >
 > You can also configure your instance to send out emails so that you can get

docs/docs/self-hosting/guides/admin.md

@@ -3,32 +3,6 @@ title: Server admin
 description: Administering your custom self-hosted Ente instance using the CLI
 ---
 
-# Administering your custom server
-
-You can use
-[Ente's CLI](https://github.com/ente-io/ente/releases?q=tag%3Acli-v0) to
-administer your self hosted server.
-
-First we need to get your CLI to connect to your custom server. Define a
-config.yaml and put it either in the same directory as CLI or path defined in
-env variable `ENTE_CLI_CONFIG_PATH`
-
-```yaml
-endpoint:
-    api: "http://localhost:8080"
-```
-
-Now you should be able to
-[add an account](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_account_add.md),
-and subsequently increase the
-[storage and account validity](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_admin_update-subscription.md)
-using the CLI.
-
-> [!NOTE]
->
-> The CLI command to add an account does not create Ente accounts. It only adds
-> existing accounts to the list of (existing) accounts that the CLI can use.
-
 ## Becoming an admin
 
 By default, the first user (and only the first user) created on the system is
@@ -78,6 +52,37 @@ You can use
 [account list](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_account_list.md)
 command to find the user id of any account.
 
+# Administering your custom server
+
+> [!NOTE] For the first user (admin) to perform administrative actions using the
+> CLI, their userID must be whitelisted in the `museum.yaml` configuration file
+> under `internal.admins`. While the first user is automatically granted admin
+> privileges on the server, this additional step is required for CLI operations.
+
+You can use
+[Ente's CLI](https://github.com/ente-io/ente/releases?q=tag%3Acli-v0) to
+administer your self hosted server.
+
+First we need to get your CLI to connect to your custom server. Define a
+config.yaml and put it either in the same directory as CLI or path defined in
+env variable `ENTE_CLI_CONFIG_PATH`
+
+```yaml
+endpoint:
+    api: "http://localhost:8080"
+```
+
+Now you should be able to
+[add an account](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_account_add.md),
+and subsequently increase the
+[storage and account validity](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_admin_update-subscription.md)
+using the CLI.
+
+> [!NOTE]
+>
+> The CLI command to add an account does not create Ente accounts. It only adds
+> existing accounts to the list of (existing) accounts that the CLI can use.
+
 ## Backups
 
 See this [FAQ](/self-hosting/faq/backup).

docs/docs/self-hosting/guides/configuring-s3.md

@@ -5,103 +5,108 @@ description:
     from outside localhost
 ---
 
-# Components of the Architecture
+# Architecture
 
 ![Client, Museum, S3](/client-museum-s3.png)
 
-There are three components involved in uploading:
+There are three components involved in uploading a file:
 
 1.  The client (e.g. the web app or the mobile app)
 2.  Ente's server (museum)
-3.  The S3-compatible object storage (e.g. minio in the default starter)
+3.  The S3-compatible object storage (e.g. MinIO in the default starter)
 
 For the uploads to work, all three of them need to be able to reach each other.
-This is because the client uploads directly to the object storage. The
-interaction goes something like this:
+This is because the client uploads directly to the object storage.
 
-1.  Client wants to upload, it asks museum where it should upload to.
-2.  Museum creates pre-signed URLs for the S3 bucket that was configured.
-3.  Client directly uploads to the S3 buckets these URLs.
+A file upload flows as follows:
+
+1.  Client that wants to upload a file asks museum where it should upload the
+    file to
+2.  museum creates pre-signed URLs for the S3 bucket that was configured
+3.  Client directly uploads to the S3 buckets these URLs
+4.  Client finally informs museum that a file has been uploaded to this URL
 
 The upshot of this is that _both_ the client and museum should be able to reach
 your S3 bucket.
 
-## Configuring S3 
+## Configuring S3
 
 The URL for the S3 bucket is configured in
 [scripts/compose/credentials.yaml](https://github.com/ente-io/ente/blob/main/server/scripts/compose/credentials.yaml#L10).
-You can edit this file directly when testing, though it is just simpler and more
-robust to create a `museum.yaml` (in the same folder as the Docker compose file)
-and put your custom configuration there (in your case, you can put an entire
-`s3` config object in your `museum.yaml`).
 
-> [!TIP]
-> For more details about these configuration objects, see the documentation for
-> the `s3` object in
+You can edit this file directly while testing, though it is more robust to
+create a `museum.yaml` (in the same folder as the Docker compose file) and to
+setup your custom configuration there.
+
+> [!TIP] For more details about these configuration objects, see the
+> documentation for the `s3` object in
 > [configurations/local.yaml](https://github.com/ente-io/ente/blob/main/server/configurations/local.yaml).
 
 By default, you only need to configure the endpoint for the first bucket.
 
-The docker compose file is shipped with MinIO as the Self Hosted S3 Compatible Storage. 
-By default, MinIO server is served on `localhost:3200` and the MinIO UI on 
-`localhost:3201`. 
-For example, in a localhost network situation, the way this 
-connection works is, Museum (`1`) and MinIO (`2`) run on the same docker network and 
-the web app (`3`) which will also be hosted on the localhost. This enables all the 
-three components of the setup being able to communicate with each other seamlessly.
+The Docker compose file is shipped with MinIO as the self hosted S3 compatible
+storage. By default, MinIO server is served on `localhost:3200` and the MinIO UI
+on `localhost:3201`.
+
+For example, in a localhost network situation, the way this connection works is,
+museum (`1`) and MinIO (`2`) run on the same Docker network and the web app
+(`3`) will also be hosted on your localhost. This enables all the three
+components of the setup to communicate with each other seamlessly.
 
 The same principle applies if you're deploying to your custom domain.
 
-## Replication 
+## Replication
 
 ![Replication](/replication.png)
-<p align="center">Community contributed diagram of Ente's Replication Process</p>
+
+<p align="center">Community contributed diagram of Ente's replication process</p>
 
 > [!IMPORTANT]
-> As of now, Replication works only if all the 3 storage type 
-> needs are fulfilled (1 Hot, 1 Cold and 1 Glacier Storage).
+>
+> As of now, replication works only if all the 3 storage type needs are
+> fulfilled (1 hot, 1 cold and 1 glacier storage).
 >
 > [Reference](https://github.com/ente-io/ente/discussions/3167#discussioncomment-10585970)
 
-If you're wondering why there are 3 buckets on MinIO UI - that's because our 
-production instance uses these to perform [replication](https://ente.io/reliability/).
+If you're wondering why there are 3 buckets on the MinIO UI - that's because our
+production instance uses these to perform
+[replication](https://ente.io/reliability/).
 
-If you're also wondering about why the bucket names are specifically what they are, 
-it's because that is exactly what we are using on our production instance. 
-We use `b2-eu-cen` as hot, `wasabi-eu-central-2-v3` as cold (also the secondary hot) 
-and `scw-eu-fr-v3` as glacier storage. As of now, all of this is hardcoded. 
+If you're also wondering about why the bucket names are specifically what they
+are, it's because that is exactly what we are using on our production instance.
+We use `b2-eu-cen` as hot, `wasabi-eu-central-2-v3` as cold (also the secondary
+hot) and `scw-eu-fr-v3` as glacier storage. As of now, all of this is hardcoded.
 Hence, the same hardcoded configuration is applied when you self host Ente.
 
-In a Self hosted Ente Instance replication is turned off by default.
-When replication is turned off, only the first bucket (`b2-eu-cen`) is used, 
-and the other two are ignored. Only the names here are specifically fixed, but 
-in the configuration body you can put any other keys. It does not have any relation 
+In a self hosted Ente instance replication is turned off by default. When
+replication is turned off, only the first bucket (`b2-eu-cen`) is used, and the
+other two are ignored. Only the names here are specifically fixed, but in the
+configuration body you can put any other keys. It does not have any relation
 with `b2`, `wasabi` or even `scaleway`.
 
-Use the `s3.hot_storage.primary` option if you'd like to set one of the other 
+Use the `s3.hot_storage.primary` option if you'd like to set one of the other
 predefined buckets as the primary bucket.
 
-## SSL Configuration 
+## SSL Configuration
 
 > [!NOTE]
 >
-> If you need to configure SSL, for example if you're running over the internet,
-> you'll need to turn off `s3.are_local_buckets` (which disables SSL in the
-> default starter compose template).
->
+> If you need to configure SSL, you'll need to turn off `s3.are_local_buckets`
+> (which disables SSL in the default starter compose template).
 
 Disabling `s3.are_local_buckets` also switches to the subdomain style URLs for
-the buckets. However, not all S3 providers support these, in particular, minio
-does not work with these in default configuration. So in such cases you'll
-also need to then enable `s3.use_path_style_urls`.
+the buckets. However, not all S3 providers support these. In particular, MinIO
+does not work with these in default configuration. So in such cases you'll also
+need to enable `s3.use_path_style_urls`.
 
 ## Summary
 
 Set the S3 bucket `endpoint` in `credentials.yaml` to a `yourserverip:3200` or
-some such IP/hostname that accessible from both where you are running the Ente
-clients (e.g. the mobile app) and also from within the Docker compose cluster.
+some such IP / hostname that is accessible from both where you are running the
+Ente clients (e.g. the mobile app) and also from within the Docker compose
+cluster.
 
-#### Example
+### Example
 
 An example `museum.yaml` when you're trying to connect to museum running on your
 computer from your phone on the same WiFi network:
@@ -116,50 +121,3 @@ s3:
         region: eu-central-2
         bucket: b2-eu-cen
 ```
-
-## FAE (Frequently Answered Errors)
-
-Here are some Frequently Answered Errors from the Community Chat with the reasoning
-for a particular error and its potential fix. 
-
-In most situations, the problem turns out to be some minute mistakes or misconfigurations 
-on the users end and that turns out to be the bottleneck of the whole problem. 
-Please make sure to `reverse_proxy` Museum to a domain as well as check your S3 
-Credentials and whole config for any minor mis-configurations.
-
-It is also suggested that the user setups Bucket CORS on MinIO or any external
-S3 Bucket they are connecting to. To setup Bucket CORS, help yourself by upload
-[this](https://help.ente.io/self-hosting/guides/external-s3#_5-fix-potential-cors-issue-with-your-bucket).
-
-### 403 Forbidden
-
-If museum (`2`) is able to make a network connection to your S3 bucket (`3`) but
-uploads are still failing, it could be a credentials or permissions issue. A
-telltale sign of this is that in the museum logs you can see `403 Forbidden`
-errors about it not able to find the size of a file even though the
-corresponding object exists in the S3 bucket.
-
-To fix these, you should ensure the following:
-
-1.  The bucket CORS rules do not allow museum to access these objects.
-    - For uploading files from the browser, you will need to currently set
-    allowedOrigins to "\*", and allow the "X-Auth-Token", "X-Client-Package"
-    headers configuration too.
-    [Here is an example of a working configuration](https://github.com/ente-io/ente/discussions/1764#discussioncomment-9478204).
-
-2.  The credentials are not being picked up (you might be setting the correct
-    creds, but not in the place where museum picks them from).
-
-### Mismatch in File Size 
-
-The "Mismatch in File Size" error mostly occurs in a situation where the client (`1`) 
-is re-uploading a file which is already in the bucket with a different File Size. The 
-reason for re-upload could be anything including Network issue, sudden killing of app
-before the upload is complete and etc. 
-
-This is also one of Museums (`2`) Validation Checks for the size of file being 
-re-uploaded from the client to the size of the file which is already 
-uploaded to the S3 Bucket.
-
-In most case, it is very unlikely that this error could be a cause of some mistake in 
-the configuration or Browser/Bucket CORS.

docs/docs/self-hosting/guides/custom-server/index.md

@@ -111,5 +111,5 @@ network, you need to use the public IP or hostname.
 > [!TIP]
 >
 > If you're having trouble uploading from your mobile app, it is likely that
-> museum is not able to connect to your S3 storage. See the [Configuring
-> S3](/self-hosting/guides/configuring-s3) guide for more details.
+> museum is not able to connect to your S3 storage. See the
+> [Configuring S3](/self-hosting/guides/configuring-s3) guide for more details.

docs/docs/self-hosting/guides/external-s3.md

@@ -250,64 +250,6 @@ docker compose exec -i postgres psql -U pguser -d ente_db -c "INSERT INTO storag
 
 After few reloads, you should see 1 To of quota.
 
-## 5. Fix potential CORS issue with your bucket
-
-### For AWS S3
-
-If you cannot upload a photo due to a CORS issue, you need to fix the CORS
-configuration of your bucket.
-
-Create a `cors.json` file with the following content:
-
-```json
-{
-    "CORSRules": [
-        {
-            "AllowedOrigins": ["*"],
-            "AllowedHeaders": ["*"],
-            "AllowedMethods": ["GET", "HEAD", "POST", "PUT", "DELETE"],
-            "MaxAgeSeconds": 3000,
-            "ExposeHeaders": ["Etag"]
-        }
-    ]
-}
-```
-
-You may want to change the `AllowedOrigins` to a more restrictive value.
-
-If you are using AWS for S3, you can execute the below command to get rid of
-CORS. Make sure to enter the right path for the `cors.json` file.
-
-```bash
-aws s3api put-bucket-cors --bucket YOUR_S3_BUCKET --cors-configuration /path/to/cors.json
-```
-
-### For Self-hosted Minio Instance
-
-> Important: MinIO does not take JSON CORS file as the input, instead you will
-> have to build a CORS.xml file or just convert the above `cors.json` to XML.
-
-A minor requirement here is the tool `mc` for managing buckets via command line
-interface. Checkout the `mc set alias` document to configure alias for your
-instance and bucket. After this you will be prompted for your AccessKey and
-Secret, which is your username and password, go ahead and enter that.
-
-```sh
-mc cors set <your-minio>/<your-bucket-name /path/to/cors.xml
-```
-
-or, if you just want to just set the `AllowedOrigins` Header, you can use the
-following command to do so.
-
-```sh
-mc admin config set <your-minio>/<your-bucket-name> api cors_allow_origin="*"
-```
-
-You can create also `.csv` file and dump the list of origins you would like to
-allow and replace the `*` with `path` to the CSV file.
-
-Now, uploads should be working fine.
-
 ## Related
 
 Some other users have also shared their setups.
@@ -315,3 +257,5 @@ Some other users have also shared their setups.
 - [Using Traefik](https://github.com/ente-io/ente/pull/3663)
 
 - [Building custom images from source (Linux)](https://github.com/ente-io/ente/discussions/3778)
+
+- [Troubleshooting Bucket CORS](/self-hosting/troubleshooting/bucket-cors)

docs/docs/self-hosting/guides/from-source.md

@@ -0,0 +1,229 @@
+---
+title: Ente from Source
+description: Getting started self hosting Ente Photos and/or Ente Auth
+---
+
+# Ente from Source
+
+> [!WARNING] NOTE The below documentation will cover instructions about
+> self-hosting the web app manually. If you want to deploy Ente hassle free, use
+> the [one line](https://ente.io/blog/self-hosting-quickstart/) command to setup
+> Ente. This guide might be deprecated in the near future.
+
+## Installing Docker
+
+Refer to
+[How to install Docker from the APT repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
+for detailed instructions.
+
+## Start the server
+
+```sh
+git clone https://github.com/ente-io/ente
+cd ente/server
+docker compose up --build
+```
+
+> [!TIP]
+>
+> You can also use a pre-built Docker image from `ghcr.io/ente-io/server`
+> ([More info](https://github.com/ente-io/ente/blob/main/server/docs/docker.md))
+
+Install the necessary dependencies for running the web client
+
+```sh
+# installing npm and yarn
+
+sudo apt update
+sudo apt install nodejs npm
+sudo npm install -g yarn // to install yarn globally
+```
+
+Then in a separate terminal, you can run (e.g) the web client
+
+```sh
+cd ente/web
+git submodule update --init --recursive
+yarn install
+NEXT_PUBLIC_ENTE_ENDPOINT=http://localhost:8080 yarn dev
+```
+
+That's about it. If you open http://localhost:3000, you will be able to create
+an account on a Ente Photos web app running on your machine, and this web app
+will be connecting to the server running on your local machine at
+`localhost:8080`.
+
+For the mobile apps, you don't even need to build, and can install normal Ente
+apps and configure them to use your
+[custom self-hosted server](/self-hosting/guides/custom-server/).
+
+> If you want to build the mobile apps from source, see the instructions
+> [here](/self-hosting/guides/mobile-build).
+
+## Web app with Docker and Compose
+
+The instructoins in previous section were just a temporary way to run the web
+app locally. To run the web apps as services, the user has to build a docker
+image manually.
+
+> [!IMPORTANT]
+>
+> Recurring changes might be made by the team or from community if more
+> improvements can be made so that we are able to build a full-fledged docker
+> image.
+
+```dockerfile
+FROM node:20-bookworm-slim as builder
+
+WORKDIR ./ente
+
+COPY . .
+COPY apps/ .
+
+# Will help default to yarn versoin 1.22.22
+RUN corepack enable
+
+# Endpoint for Ente Server
+ENV NEXT_PUBLIC_ENTE_ENDPOINT=https://your-ente-endpoint.com
+ENV NEXT_PUBLIC_ENTE_ALBUMS_ENDPOINT=https://your-albums-endpoint.com
+
+RUN yarn cache clean
+RUN yarn install --network-timeout 1000000000
+RUN yarn build:photos && yarn build:accounts && yarn build:auth && yarn build:cast
+
+FROM node:20-bookworm-slim
+
+WORKDIR /app
+
+COPY --from=builder /ente/apps/photos/out /app/photos
+COPY --from=builder /ente/apps/accounts/out /app/accounts
+COPY --from=builder /ente/apps/auth/out /app/auth
+COPY --from=builder /ente/apps/cast/out /app/cast
+
+RUN npm install -g serve
+
+ENV PHOTOS=3000
+EXPOSE ${PHOTOS}
+
+ENV ACCOUNTS=3001
+EXPOSE ${ACCOUNTS}
+
+ENV AUTH=3002
+EXPOSE ${AUTH}
+
+ENV CAST=3003
+EXPOSE ${CAST}
+
+# The albums app does not have navigable pages on it, but the
+# port will be exposed in-order to self up the albums endpoint
+# `apps.public-albums` in museum.yaml configuration file.
+ENV ALBUMS=3004
+EXPOSE ${ALBUMS}
+
+CMD ["sh", "-c", "serve /app/photos -l tcp://0.0.0.0:${PHOTOS} & serve /app/accounts -l tcp://0.0.0.0:${ACCOUNTS} & serve /app/auth -l tcp://0.0.0.0:${AUTH} & serve /app/cast -l tcp://0.0.0.0:${CAST}"]
+```
+
+The above is a multi-stage Dockerfile which creates a production ready static
+output of the 4 apps (Photos, Accounts, Auth and Cast) and serves the static
+content with Caddy.
+
+Looking at 2 different node base-images doing different tasks in the same
+Dockerfile would not make sense, but the Dockerfile is divided into two just to
+improve the build efficiency as building this Dockerfile will arguably take more
+time.
+
+Lets build a Docker image from the above Dockerfile. Copy and paste the above
+Dockerfile contents in the root of your web directory which is inside
+`ente/web`. Execute the below command to create an image from this Dockerfile.
+
+```sh
+# Build the image
+docker build -t <image-name>:<tag> --no-cache --progress plain .
+```
+
+You can always edit the Dockerfile and remove the steps for apps which you do
+not intend to install on your system (like auth or cast) and opt out of those.
+
+Regarding Albums App, take a note that they are not apps with navigable pages,
+if accessed on the web-browser they will simply redirect to ente.web.io.
+
+## compose.yaml
+
+Moving ahead, we need to paste the below contents into the compose.yaml inside
+`ente/server/compose.yaml` under the services section.
+
+```yaml
+ente-web:
+    image: <image-name> # name of the image you used while building
+    ports:
+        - 3000:3000
+        - 3001:3001
+        - 3002:3002
+        - 3003:3003
+        - 3004:3004
+    environment:
+        - NODE_ENV=development
+    restart: always
+```
+
+Now, we're good to go. All we are left to do now is start the containers.
+
+```sh
+docker compose up -d # --build
+
+# Accessing the logs
+docker compose logs <container-name>
+```
+
+## Configure App Endpoints
+
+> [!NOTE] Previously, this was dependent on the env variables
+> `NEXT_ENTE_PUBLIC_ACCOUNTS_ENDPOINT` and etc. Please check the below
+> documentation to update your setup configurations
+
+You can configure the web endpoints for the other apps including Accounts,
+Albums Family and Cast in your `museum.yaml` configuration file. Checkout
+[`local.yaml`](https://github.com/ente-io/ente/blob/543411254b2bb55bd00a0e515dcafa12d12d3b35/server/configurations/local.yaml#L76-L89)
+to configure the endpoints. Make sure to setup up your DNS Records accordingly
+to the similar URL's you set up in `museum.yaml`.
+
+Next part is to configure the web server.
+
+# Web server configuration
+
+The last step ahead is configuring reverse_proxy for the ports on which the apps
+are being served (you will have to make changes, if you have cusotmized the
+ports). The web server of choice in this guide is
+[Caddy](https://caddyserver.com) because with caddy you don't have to manually
+configure/setup SSL ceritifcates as caddy will take care of that.
+
+```groovy
+photos.yourdomain.com {
+	reverse_proxy http://localhost:3001
+    # for logging
+    log {
+        level error
+    }
+}
+
+auth.yourdomain.com {
+    reverse_proxy http://localhost:3002
+}
+# and so on ...
+```
+
+Next, start the caddy server :).
+
+```sh
+# If caddy service is not enabled
+sudo systemctl enable caddy
+
+sudo systemctl daemon-reload
+
+sudo systemctl start caddy
+```
+
+## Contributing
+
+Please start a discussion on the Github Repo if you have any suggestions for the
+Dockerfile, You can also share your setups on Github Discussions.

docs/docs/self-hosting/guides/tailscale.md

@@ -2,39 +2,56 @@
 title: Self Hosting with Tailscale (Community)
 description: Guides for self-hosting Ente Photos and/or Ente Auth with Tailscale
 ---
+
 # Guide
 
-This guide aims to achieve self-hosting Ente photos or Ente-Auth with tailscale (TSDPROXY) without exposing any port OR if someone is behind CGNAT and cannot open any port on the internet but want to run their own selfhosted service for themselves, friends and family only.
+This guide aims to achieve self-hosting Ente photos or Ente-Auth with tailscale
+(TSDPROXY) without exposing any port OR if someone is behind CGNAT and cannot
+open any port on the internet but want to run their own selfhosted service for
+themselves, friends and family only.
 
 Before getting start keep the following NOTE in mind.
 
-> [!NOTE]
-> If someone is behind double or triple CGNAT; must install tailscale system wide by running `curl -fsSL https://tailscale.com/install.sh | sh` in your linux terminal and `sudo tailscale up` otherwise dns resolver will fail and uploading will not work. This is not necessary for those who are not behing CGNAT.
-> This guide also work on docker rootless and normal.
+> [!NOTE] If someone is behind double or triple CGNAT; must install tailscale
+> system wide by running `curl -fsSL https://tailscale.com/install.sh | sh` in
+> your linux terminal and `sudo tailscale up` otherwise dns resolver will fail
+> and uploading will not work. This is not necessary for those who are not
+> behing CGNAT. This guide also work on docker rootless and normal.
 
-> [!CAUTION]
-Remember that current docker update 28.0.0 has some bug and cannot connect to external network. Make sure to install docker-ce 27.5.0, docker-ce-rootless-extras 27.5.0 and docker-ce-cli 27.5.0. Hopefully docker 28.1.0 will resolve this issue in next week. Refrence links are [Moby Github Repo Issues 49511](https://github.com/moby/moby/issues/49511) and [Moby Github Repo Issues 49519](https://github.com/moby/moby/issues/49519)
-
-> [!IMPORTANT]
-> For Docker rootless, the user must have local permissions for all directories required by the Ente-photos self-hosted server. This can be achieved by running `sudo chown -R 1000:1000 /home/ubuntu/docker/ente`. In the Linux terminal, you can check the UID with `id -u` or simply `id`. The first user typically has UID 1000.
-> To allow listening and pinging on any port without root privileges, create a file called `/etc/sysctl.d/99-rootless.conf` with the following content:
+> [!IMPORTANT] For Docker rootless, the user must have local permissions for all
+> directories required by the Ente-photos self-hosted server. This can be
+> achieved by running `sudo chown -R 1000:1000 /home/ubuntu/docker/ente`. In the
+> Linux terminal, you can check the UID with `id -u` or simply `id`. The first
+> user typically has UID 1000. To allow listening and pinging on any port
+> without root privileges, create a file called `/etc/sysctl.d/99-rootless.conf`
+> with the following content:
+>
 > ```
 > net.ipv4.ip_unprivileged_port_start=0
 > net.ipv4.ping_group_range = 0 2147483647
 > ```
-> than run `sudo sysctl --system`.
-> Create `~/.config/systemd/user/docker.service.d/override.conf` with the following content:
+>
+> then run `sudo sysctl --system`. Create
+> `~/.config/systemd/user/docker.service.d/override.conf` with the following
+> content:
+>
 > ```
 > [Service]
 > Environment="DOCKERD_ROOTLESS_ROOTLESSKIT_NET=slirp4netns"
 > Environment="DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns"
 > ```
-> and Restart the docker daemon
-> `systemctl --user restart docker`
-> Instead of `--volume /var/run/docker.sock:/var/run/docker.sock` in TSDPROXY compose.yaml, use `--volume $XDG_RUNTIME_DIR/docker.sock:/var/run/docker.sock`
+>
+> and Restart the docker daemon `systemctl --user restart docker` Instead of
+> `--volume /var/run/docker.sock:/var/run/docker.sock` in TSDPROXY compose.yaml,
+> use `--volume $XDG_RUNTIME_DIR/docker.sock:/var/run/docker.sock`
+
+## GETTING START WITH SETUP
+
+First of all create a directory
+`sudo mkdir -p /home/ubuntu/docker/tsdproxy/config` then `cd docker/tsdproxy`
+and create compose.yaml file by running `sudo nano compose.yaml`. Populate it
+with the following:
 
-## GETTING START WITH SETUP ##
-First of all create a directory `sudo mkdir -p /home/ubuntu/docker/tsdproxy/config` than `cd docker/tsdproxy` and create compose.yaml file by running `sudo nano compose.yaml`. Populate it with the following:
 ```
 services:
   tsdproxy:
@@ -62,9 +79,18 @@ networks:
   proxy:
     name: proxy
 ```
-Now login into your tailscale account admin counsle > settings > keys > Generate authkey. Give any description and must select resuable, because the key get purged if not selected after rebooting machine. It is advisable to create **Tags** in **ACLs settings** `tag: tsdproxy` `tag: ente` `tag: minio` as well. This will create a tag nodes with no key expirory. One is safe to reboot restart docker or machine.
-> Copy the generated authkey as it is shown only once.
-Make tsdproxy.yaml file in `cd docker/tsdproxy/config` by running `sudo nano tsdproxy.yaml` and pupolate it with the following contant:
+
+Now login into your tailscale account admin counsle > settings > keys > Generate
+authkey. Give any description and must select resuable, because the key get
+purged if not selected after rebooting machine. It is advisable to create
+**Tags** in **ACLs settings** `tag: tsdproxy` `tag: ente` `tag: minio` as well.
+This will create a tag nodes with no key expirory. One is safe to reboot restart
+docker or machine.
+
+> Copy the generated authkey as it is shown only once. Make tsdproxy.yaml file
+> in `cd docker/tsdproxy/config` by running `sudo nano tsdproxy.yaml` and
+> pupolate it with the following contant:
+
 ```
 defaultproxyprovider: default
 docker:
@@ -87,12 +113,20 @@ log:
   json: false
 proxyaccesslog: true
 ```
-In the same directory run `sudo nano authkey` and paste the authkey just copied earlier from tailscale admin counsel.
-> Here Tailscale (TSDPROXY) setup is complet in all respect. Just run `docker compose up -d`. Check your tailscale amdin counsel and you will see tsdproxy node up and running. Make sure that **HTTPS** is enabled in tailscale DNS settings.
-> You can visit the TSDPROXY web GUI by https://tsdproxy.xyz.ts.net. (xyz is change value for everyone)
 
-## ente Part ##
+In the same directory run `sudo nano authkey` and paste the authkey just copied
+earlier from tailscale admin counsel.
+
+> Here Tailscale (TSDPROXY) setup is complet in all respect. Just run
+> `docker compose up -d`. Check your tailscale amdin counsel and you will see
+> tsdproxy node up and running. Make sure that **HTTPS** is enabled in tailscale
+> DNS settings. You can visit the TSDPROXY web GUI by
+> https://tsdproxy.xyz.ts.net. (xyz is change value for everyone)
+
+## ente Part
+
 First make the following necessary files/directories:
+
 ```
 sudo mkdir -p /home/ubuntu/docker/ente/custom-logs
 sudo mkdir -p /home/ubuntu/docker/ente/data
@@ -100,9 +134,14 @@ sudo mkdir -p /home/ubuntu/docker/ente/minio-data
 sudo mkdir -p /home/ubuntu/docker/ente/postgres-data
 sudo mkdir -p /home/ubuntu/docker/ente/scripts/compose
 ```
-Than give user permission for each of the above directory. `sudo chown -R 1000:1000 /home/ubuntu/docker/ente/custom-logs` etc etc. Make sure not to skip `/home/ubuntu/docker/tsdproxy/config`
 
-`cd docker/ente/script/compose` and run `sudo nano credentials.yaml` than populate it with the following:
+Than give user permission for each of the above directory.
+`sudo chown -R 1000:1000 /home/ubuntu/docker/ente/custom-logs` etc etc. Make
+sure not to skip `/home/ubuntu/docker/tsdproxy/config`
+
+`cd docker/ente/script/compose` and run `sudo nano credentials.yaml` than
+populate it with the following:
+
 ```
 db:
     host: postgres
@@ -134,7 +173,9 @@ s3:
         bucket: scw-eu-fr-v3
 ```
 
-In the same directory run `sudo nano minio-provision.sh` and populate it with the following contant:
+In the same directory run `sudo nano minio-provision.sh` and populate it with
+the following contant:
+
 ```
 #!/bin/sh
 
@@ -154,7 +195,9 @@ mc mb -p wasabi-eu-central-2-v3
 mc mb -p scw-eu-fr-v3
 ```
 
-Now `cd docker/ente` and run `sudo nano docker-compose.yaml` and populate it with the following:
+Now `cd docker/ente` and run `sudo nano docker-compose.yaml` and populate it
+with the following:
+
 ```
 services:
   museum:
@@ -255,32 +298,52 @@ services:
 networks:
   ente:
     name: ente
-  
+
   proxy:
     external: true
 ```
 
-> Thats it. Run `docker compose up -d`. Wait till every container become healthy. Open web browser. Make sure tailscale is installed on the machine. Visit https://ente.xyz.ts.net/ping. It will pong. All good if you see it. First time it will take minute or two to get SSL cert. Downnload Desktop or mobile app. Tap 7 time on the screen, which will prompt developer mode. Add https://ente.xyz.ts.net. Add new user. When asked for OTP. Just go to linux terminal and run `docker logs ente-museum-1`. Search for userauth. Feed the six digit and Done.
+> Thats it. Run `docker compose up -d`. Wait till every container become
+> healthy. Open web browser. Make sure tailscale is installed on the machine.
+> Visit https://ente.xyz.ts.net/ping. It will pong. All good if you see it.
+> First time it will take minute or two to get SSL cert. Downnload Desktop or
+> mobile app. Tap 7 time on the screen, which will prompt developer mode. Add
+> https://ente.xyz.ts.net. Add new user. When asked for OTP. Just go to linux
+> terminal and run `docker logs ente-museum-1`. Search for userauth. Feed the
+> six digit and Done.
+
+> For getting 100TB (limitless) storage. Just Install ente-cli for windows.
+> Extract it and add folder. Name it **export**. Add config.yaml file along and
+> populate it with the following:
 
-> For getting 100TB (limitless) storage. Just Install ente-cli for windows. Extract it and add folder. Name it **export**. Add config.yaml file along and populate it with the following:
 ```
 endpoint:
   api: "https://ente.xyz.ts.net"
   accounts: "http://localhost:3001"
-  
+
 log: false
 ```
-Right-Click in the directory where you have extracted ente-cli. Select `open in terminal`. Run
+
+Right-Click in the directory where you have extracted ente-cli. Select
+`open in terminal`. Run
+
 ```
 .\ente.exe account bob # change bob to yours
 ```
-Hit Enter twice.
-For export directory, just write export. As already created **export** folder earlier.
-**Write email. The one which is already used befor when creating ente account in ente desktop app.**
-Type the same Password used before for the account.Run 
+
+Hit Enter twice. For export directory, just write export. As already created
+**export** folder earlier. **Write email. The one which is already used befor
+when creating ente account in ente desktop app.** Type the same Password used
+before for the account.Run
+
 ```
 .\ente.ext account list
 ```
+
 This will list all account details. Copy Acount ID.
-> Navigate to museum.yaml file. `cd docker/ente`. Run `sudo nano museum.yaml` and add the account ID under Admins. Delete any previous entries.
-Restart ente-museum-1 container from linux terminal. Run `docker restart ente-museum-1`. All well, now you will have 100TB storage. Repeat if for any other accounts you want to give unlimited storage access.
+
+> Navigate to museum.yaml file. `cd docker/ente`. Run `sudo nano museum.yaml`
+> and add the account ID under Admins. Delete any previous entries. Restart
+> ente-museum-1 container from linux terminal. Run
+> `docker restart ente-museum-1`. All well, now you will have 100TB storage.
+> Repeat if for any other accounts you want to give unlimited storage access.