TerribleDev/ente
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[docs] revamp administration and document configuration examples

Browse Source
This commit is contained in:
Keerthana
2025-07-25 17:22:52 +05:30
parent bd58becd38
commit aae1caf37d
22 changed files with 657 additions and 554 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
docs/docs/.vitepress/sidebar.ts
View File

@@ -263,21 +263,26 @@ export const sidebar = [
link: "/self-hosting/install/compose",
},
{
text: "Without Docker",
link: "/self-hosting/install/without-docker",
text: "Manual Setup (Without Docker)",
link: "/self-hosting/install/manual",
},
{
text: "Environment Variables and Defaults",
link: "/self-hosting/install/env-var",
},
{
text: "Configuration File",
link: "/self-hosting/install/config-file",
text: "Configuration",
link: "/self-hosting/install/config",
},
{
text: "Post-installation Steps",
link: "/self-hosting/install/post-install/",
},
{
text: "Upgrade",
link: "/self-hosting/install/upgrade/",
}
],
},
{
@@ -285,20 +290,24 @@ export const sidebar = [
collapsed: true,
items: [
{
text: "Configuring your server",
link: "/self-hosting/administration/museum",
text: "User Management",
link: "/self-hosting/administration/users",
},
{
text: "Configuring Object Storage",
link: "/self-hosting/administration/object-storage",
},
{
text: "Reverse proxy",
text: "Reverse Proxy",
link: "/self-hosting/administration/reverse-proxy",
},
{
text: "Configuring CLI for your instance",
link: "/self-hosting/administration/selfhost-cli",
text: "Object Storage",
link: "/self-hosting/administration/object-storage",
},
{
text: "Ente CLI",
link: "/self-hosting/administration/cli",
},
{
text: "Backup and Restore",
link: "/self-hosting/administration/backup",
},
],
@@ -321,6 +330,10 @@ export const sidebar = [
text: "Ente via Tailscale",
link: "/self-hosting/guides/tailscale",
},
{
text: "Running Ente with systemd",
link: "/self-hosting/guides/systemd",
},
],
},
{

0
docs/docs/self-hosting/administration/accounts.md
View File

42
docs/docs/self-hosting/administration/backup.md
View File

@@ -5,32 +5,17 @@ description: General introduction to backing up your self hosted Ente instance
# Backing up your Ente instance
> [!WARNING]
>
> This is not meant to be a comprehensive and bullet proof guide. There are many
> moving parts, and small mistakes might make your backups unusable.
>
> Please treat this only as a general introduction. And remember to test your
> restores.
At the minimum, a functional Ente backend needs three things:
A functional Ente backend needs three things:
1. Museum (the API server)
2. Postgres (the database)
3. Object storage (any S3-compatible object storage)
When thinking about backups, this translates into backing up the relevant state
from each of these:
Thus, when thinking about backups:
1. For Museum, you'd want to backup your `museum.yaml`, `credentials.yaml` or
any other custom configuration that you created. In particular, you should
backup the
[secrets that are specific to your instance](https://github.com/ente-io/ente/blob/74377a93d8e20e969d9a2531f32f577b5f0ef090/server/configurations/local.yaml#L188)
(`key.encryption`, `key.hash` and `jwt.secret`).
2. For PostgreSQL, the entire data volume needs to be backed up.
3. For Object Storage, the entire data volume needs to be backed up.
1. For Museum, you should backup your `museum.yaml`, `credentials.yaml` or
any other custom configuration that you created.
2. The entire data volume needs to be backed up for the database and object storage.
A common oversight is taking a lot of care for backing up the object storage,
even going as far as enabling replication and backing up the the multiple object
@@ -43,22 +28,19 @@ database contains information like a file specific encryption key.
Viewed differently, to decrypt your data you need three pieces of information:
1. The encrypted file data itself (which comes from the object storage backup).
2. The ([encrypted](https://ente.io/architecture/)) file and collection specific
encryption keys (which come from the database backup).
2. The encrypted file and collection specific encryption keys
(which come from the database backup).
3. The master key (which comes from your password).
---
If you're starting out with self hosting, we recommend keeping plaintext backup of your photos.
If you're starting out with self hosting, our recommendation is to start by
keeping a plaintext backup of your photos.
[You can use the CLI or the desktop app to automate this](/photos/faq/export).
Once you get more comfortable with the various parts, you can try backing up
your instance.
If you stop doing plaintext backups and instead rely on your instance backup,
ensure that you do the full restore process also to verify you can get back your
data. As the industry saying goes, a backup without a restore is no backup at
If you rely on your instance backup, ensure that you do full
restoration to verify that you are able to access your data.
As the industry saying goes, a backup without a restore is no backup at
all.

68
docs/docs/self-hosting/administration/cli.md Normal file
View File

@@ -0,0 +1,68 @@
---
title: Ente CLI for Self-hosted Instance
description: Guide to configuring Ente CLI for Self Hosted Instance
---
# Ente CLI for self-hosted instance
If you are self-hosting, you can configure Ente CLI to export data &
perform basic administrative actions.
## Step 1: Configure endpoint
To do this, first configure the CLI to use your server's endpoint.
Define `config.yaml` and place it in `~/.ente/` or directory
specified by `ENTE_CLI_CONFIG_DIR` or directory where Ente CLI
is present.
``` yaml
# Set the API endpoint to your domain where Museum is being served.
endpoint:
api: http://localhost:8080
```
## Step 2: Whitelist admin user
You can whitelist administrator user by following this
[guide](/self-hosting/administration/users#whitelist-admins)
## Step 3: Add an account
::: detail
You can not create new accounts using Ente CLI. You can only log in to your existing accounts.
To create a new account, use Ente Photos (or Ente Auth) web application, desktop or mobile.
:::
You can add your existing account using Ente CLI.
``` shell
ente account add
```
This should prompt you for authentication details.
Your account should be added and can be used for exporting data
(for plain-text backup), managing Ente Auth and performing
administrative actions.
## Step 4: Increase storage and account validity
You can use `ente admin update-subscription` to increase
storage quota and account validity (duration).
For infinite storage and validity, use the following command:
``` shell
ente admin update-subscription -a <admin-user-mail> -u <user-to-update> --no-limit
# Set a limit
ente admin update-subscription -a <admin-user-mail> -u <user-to-update> --no-limit False
```
For more information, check out the documentation for setting
[storage and account validity](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_admin_update-subscription.md)
using the CLI.
## References
1. [Ente CLI Documentation](https://github.com/ente-io/ente/blob/main/cli/docs/generated)

72
docs/docs/self-hosting/administration/museum.md
View File

@@ -1,72 +0,0 @@
---
title: Configuring your server
description: Guide to writing a museum.yaml
---
# Configuring your server
Ente's monolithic server is called **museum**.
`museum.yaml` is a YAML configuration file used to configure museum. By default,
[`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
is provided, but its settings are overridden with those from `museum.yaml`.
If you used our quickstart script, your `my-ente` directory will include a
`museum.yaml` file with preset configurations for encryption keys, secrets,
PostgreSQL and MinIO.
> [!TIP]
>
> Always do `docker compose down` inside your `my-ente` directory. If you've
> made changes to `museum.yaml`, restart the containers with
> `docker compose up -d ` to see your changes in action.
## S3 buckets
The `s3` section within `museum.yaml` is by default configured to use local
MinIO buckets.
If you wish to use an external S3 provider, you can edit the configuration with
your provider's credentials, and set `are_local_buckets` to `false`.
Check out [Configuring S3] to understand
more about configuring S3 buckets.
MinIO uses the port `3200` for API Endpoints and their web app runs over
`:3201`. You can login to MinIO Web Console by opening `localhost:3201` in your
browser.
If you face any issues related to uploads then checkout
[Troubleshooting bucket CORS] and
[Frequently encountered S3 errors].
## Web apps
The web apps for Ente Photos is divided into multiple sub-apps like albums,
cast, auth, etc. These endpoints are configurable in `museum.yaml` under the
`apps.*` section.
For example,
```yaml
apps:
public-albums: https://albums.myente.xyz
cast: https://cast.myente.xyz
accounts: https://accounts.myente.xyz
```
> [!IMPORTANT] By default, all the values redirect to our publicly hosted
> production services. For example, if `public-albums` is not configured your
> shared album will use the `albums.ente.io` URL.
After you are done with filling the values, restart museum and the app will
start utilizing those endpoints instead of Ente's production instances.
Once you have configured all the necessary endpoints, `cd` into `my-ente` and
stop all the Docker containers with `docker compose down` and restart them with
`docker compose up -d`.
Similarly, you can use the default
[`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
as a reference for building a functioning `museum.yaml` for many other
functionalities like SMTP, Hardcoded-OTTs, etc.

118
docs/docs/self-hosting/administration/object-storage.md
View File

@@ -6,44 +6,76 @@ description:
# Configuring Object Storage
## Replication
Ente relies on [S3-compatible](https://docs.aws.amazon.com/s3/) cloud storage for
storing files (photos, thumbnails and videos) as objects.
Ente ships MinIO as S3-compatible storage by default in quickstart and Docker Compose for quick testing.
This document outlines configuration of S3 buckets and enabling replication for further usage.
## Museum
The S3-compatible buckets have to be configured in `museum.yaml` file.
### General Configuration
Some of the common configuration that can be done at top-level are:
1. **SSL Configuration:** If you need to configure SSL (i. e., the buckets are accessible via HTTPS),
you'll need to set `s3.are_local_buckets` to `false`.
2. **Path-style URLs:** Disabling `s3.are_local_buckets` also switches to the subdomain-style URLs for
the buckets. However, some S3 providers such as MinIO do not support this.
Set `s3.use_path_style_urls` to `true` for such cases.
### Replication
> [!IMPORTANT]
>
> As of now, replication works only if all the 3 storage buckets are configured (2 hot and 1 cold storage).
> Replication works only if all 3 storage buckets are configured (2 hot and 1 cold storage).
>
> For more information, check this
> [discussion](https://github.com/ente-io/ente/discussions/3167#discussioncomment-10585970)
> For more information, check [this discussion](https://github.com/ente-io/ente/discussions/3167#discussioncomment-10585970)
> and our article on ensuring [reliability](https://ente.io/reliability/).
In a self hosted Ente instance replication is turned off by default. When
replication is turned off, only the first bucket (`b2-eu-cen`) is used, and the
other two are ignored. Only the names here are specifically fixed, but in the
configuration body you can put any other keys.
Replication is disabled by default in self-hosted instance. Only the first bucket (`b2-eu-cen`) is used.
Use the `s3.hot_storage.primary` option if you'd like to set one of the other
predefined buckets as the primary bucket.
Only the names are specifically fixed, you can put any other keys in configuration body.
## SSL Configuration
Use the `s3.hot_storage.primary` option if you'd like to set one of the other pre-defined buckets as the primary bucket.
> [!NOTE]
>
> If you need to configure SSL, you'll need to turn off `s3.are_local_buckets`
> (which disables SSL in the default starter compose template).
### Bucket configuration
Disabling `s3.are_local_buckets` also switches to the subdomain style URLs for
the buckets. However, not all S3 providers support these. In particular, MinIO
does not work with these in default configuration. So in such cases you'll also
need to enable `s3.use_path_style_urls`.
The keys `b2-eu-cen` (primary storage), `wasabi-eu-central-2-v3` (secondary storage)
and `scw-eu-fr-v3` (cold storage) are hardcoded, however, the keys and secret can be anything.
# Fix potential CORS issues with your Buckets
It has no relation to Backblaze, Wasabi or Scaleway.
## For AWS S3
Each bucket's endpoint, region, key and secret should be configured accordingly
if using an external bucket.
If you cannot upload a photo due to a CORS issue, you need to fix the CORS
Additionally, you can enable SSL and path-style URL for specific buckets, which provides
flexibility for storage. If this is not configured, top level configuration (`s3.are_local_buckets`
and `s3.use_path_style_urls`) is used.
A sample configuration for `b2-eu-cen` is provided, which can be used for other 2 buckets as well:
``` yaml
b2-eu-cen:
are_local_buckets: true
use_path_style_urls: true
key: <key>
secret: <secret>
endpoint: localhost:3200
region: eu-central-2
bucket: b2-eu-cen
```
## CORS (Cross-Origin Resource Sharing)
If you cannot upload a photo due to CORS error, you need to fix the CORS
configuration of your bucket.
Create a `cors.json` file with the following content:
Use the content provided below for creating a `cors.json` file:
``` json
{
@@ -59,27 +91,37 @@ Create a `cors.json` file with the following content:
}
```
You may want to change the `AllowedOrigins` to a more restrictive value.
You may have to change the `AllowedOrigins` to allow only certain origins
(your Ente web apps and Museum.) for security.
If you are using AWS for S3, you can execute the below command to get rid of
CORS. Make sure to enter the right path for the `cors.json` file.
Assuming you have AWS CLI on your system and that you have configured
it with your access key and secret, you can execute the below command to
set bucket CORS. Make sure to enter the right path for the `cors.json` file.
```bash
``` shell
aws s3api put-bucket-cors --bucket YOUR_S3_BUCKET --cors-configuration /path/to/cors.json
```
## For MinIO
### MinIO
Checkout the `mc set alias` document to configure alias for your
instance and bucket. After this you will be prompted for your AccessKey and
Secret, which is your username and password.
Assuming you have configured an alias for MinIO account using the command:
To set the `AllowedOrigins` Header, you can use the
following command to do so.
```sh
mc admin config set <your-minio>/<your-bucket-name> api cors_allow_origin="*"
``` shell
mc alias set storage-account-alias minio-endpoint minio-key minio-secret
```
You can create also `.csv` file and dump the list of origins you would like to
allow and replace the `*` with `path` to the CSV file.
where,
1. `storage-account-alias` is a valid storage account alias name
2. `minio-endpoint` is the endpoint where MinIO is being served without the protocol (http or https). Example: `localhost:3200`
3. `minio-key` is the MinIO username defined in `MINIO_ROOT_USER`
4. `minio-secret` is the MinIO password defined in `MINIO_PASSWORD`
To set the `AllowedOrigins` Header, you can use the
following command:.
``` shell
mc admin config set storage-account-alias api cors_allow_origin="*"
```
You can create also `.csv` file and dump the list of origins you would like to allow and replace the `*` with path to the CSV file.

14
docs/docs/self-hosting/administration/reverse-proxy.md
View File

@@ -5,8 +5,9 @@ Description: Configuring reverse proxy for Museum and other services
# Reverse proxy
Configuring reverse proxy is a way to make the service accessible via the public
Internet without exposing multiple ports for various services.
Reverse proxy helps in making application services
accessible via the Internet without exposing multiple
ports for various services.
It also allows configuration of HTTPS through SSL certificate management.
@@ -48,10 +49,10 @@ DNS propagation can take a few minutes to take effect.
## Step 2: Configure reverse proxy
After installing Caddy, a `Caddyfile` is created on the path
After installing Caddy, `Caddyfile` is created at
`/etc/caddy/`. Edit `/etc/caddy/Caddyfile` to configure reverse proxies.
Here is a ready-to-use configuration that can be used with your own domain.
You can edit the minimal configuration provided below for your own needs.
> yourdomain.tld is an example. Replace it with your own domain
@@ -97,9 +98,8 @@ sudo systemctl caddy reload
## Step 4: Verify the setup
Ente Photos web app should be up on https://web.ente.yourdomain.tld.
Museum should be accessible at https://api.ente.yourdomain.tld.
Ente Photos web app should be up on https://web.ente.yourdomain.tld and
Museum at https://api.ente.yourdomain.tld.
> [!TIP]
> If you are using other reverse proxy servers such as NGINX,

42
docs/docs/self-hosting/administration/selfhost-cli.md
View File

@@ -1,42 +0,0 @@
---
title: CLI for Self Hosted Instance
description: Guide to configuring Ente CLI for Self Hosted Instance
---
# Ente CLI for self-hosted instance
If you are self-hosting, you can configure CLI to export data &
perform basic administrative actions.
## Step 1: Configure endpoint
To do this, first configure the CLI to use your server's endpoint.
Define `config.yaml` and place it in `~/.ente/` directory or directory
specified by `ENTE_CLI_CONFIG_DIR` or CLI's directory.
``` yaml
endpoint:
api: http://localhost:8080
```
You should be able to
[add an account](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_account_add.md),
and subsequently increase the
[storage and account validity](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_admin_update-subscription.md)
using the CLI.
For administrative actions, you first need to whitelist admin users.
You can create `server/museum.yaml`, and whitelist add the admin user ID `internal.admins`. See
[local.yaml](https://github.com/ente-io/ente/blob/main/server/configurations/local.yaml#L211C1-L232C1)
in the server source code for details about how to define this.
You can use
[account list](https://github.com/ente-io/ente/blob/main/cli/docs/generated/ente_account_list.md)
command to find the user id of any account.
```yaml
internal:
admins:
- 1580559962386440
```

116
docs/docs/self-hosting/administration/users.md Normal file
View File

@@ -0,0 +1,116 @@
---
title: User Management - Self-hosting
description: Guide to configuring Ente CLI for Self Hosted Instance
---
# User Management
You may wish to self-host Ente for your family or close circle. In such cases,
you may wish to enable administrative access for few users, disable new registrations,
manage one-time tokens (OTTs), etc.
This document covers the details on how you can administer users on your server.
## Whitelist admins
The administrator users have to be explicitly whitelisted in `museum.yaml`. You can achieve this
the following steps:
1. Connect to `ente_db` (the database used for storing data
related to Ente).
``` shell
# Change the DB name and DB user name if you use different values.
# If using Docker
docker exec -it <postgres-ente-container-name> psql -U pguser -d ente_db
# Or when using psql directly
psql -U pguser -d ente_db
```
2. Get the user ID of the first user by running the following PSQL command:
``` sql
SELECT * from users;
```
3. Edit `internal.admins` or `internal.admin` (if you wish to whitelist only single user) in `museum.yaml`
to add the user ID you wish to whitelist.
- For multiple admins:
``` yaml
internal:
admins:
- <user_id>
```
- For single admin:
``` yaml
internal:
admin: <user_id>
```
4. Restart Museum by restarting the cluster
## Increase storage and account validity
You can use Ente CLI for increasing storage quota and account validity for
users on your instance. Check this guide for more
[information](/self-hosting/administration/cli#step-4-increase-storage-and-account-validity)
## Handle user verification codes
Ente currently relies on verification codes for completion of registration.
These are accessible in server logs. If using Docker Compose, they can be
accessed by running `sudo docker compose logs` in the cluster folder where Compose
file resides.
However, you may wish to streamline this workflow. You can follow one of the 2 methods
if you wish to have many users in the system.
### Use hardcoded OTTs
You can configure to use hardcoded OTTs only for specific emails, or based on suffix.
A sample configuration for the same is provided below, which is to be used in `museum.yaml`:
``` yaml
internal:
hardcoded-ott:
emails:
- "example@example.org,123456"
local-domain-suffix: "@example.org"
local-domain-value: 012345
```
This sets OTT to 123456 for the email address example@example.com and 012345 for emails having @example.com as suffix.
### Send email with verification code
You can configure SMTP for sending verification code e-mails to users, which
is efficient if you do not know mail addresses of people for who you want to hardcode
OTTs or if you are serving larger audience.
Set the host and port accordingly with your credentials in `museum.yaml`
``` yaml
smtp:
host:
port:
# Optional username and password if using local relay server
username:
password:
# Email address used for sending emails (this mail's credentials have to be provided)
email:
# Optional name for sender
sender-name:
```
## Disable registrations
For security purposes, you may choose to disable registrations on your instance. You can disable new
registrations by using the following configuration in `museum.yaml`.
``` yaml
internal:
disable-registration: true
```

17
docs/docs/self-hosting/guides/systemd.md
View File

@@ -5,24 +5,15 @@ description: Running Ente services (Museum and web application) via systemd
# Running Ente using `systemd`
On Linux distributions using `systemd` as initialization system, Ente can be configured to run as a background service, upon system startup by using service files.
On Linux distributions using `systemd` as initialization system, Ente can be configured to run as a background service, upon system startup by service files.
## Museum as a background service
Please check the below links if you want to run Museum as a service, both of
them are battle tested.
Please check the below links if you want to run Museum as a service, both of them are battle tested.
1. [How to run museum as a systemd service](https://gist.github.com/mngshm/a0edb097c91d1dc45aeed755af310323)
2. [Museum.service](https://github.com/ente-io/ente/blob/23e678889189157ecc389c258267685934b83631/server/scripts/deploy/museum.service#L4)
Once you are done with setting and running Museum, all you are left to do is run
the web app and reverse_proxy it with a webserver. You can check the following
resources for Deploying your web app.
Once you are done with setting and running Museum, all you are left to do is run the web app and set up reverse proxy. Check out the documentation for [more information](/self-hosting/install/manual#step-3-configure-web-application).
1. [Running Ente Web app as a systemd Service](https://gist.github.com/mngshm/72e32bd483c2129621ed0d74412492fd)
## References
1. [How to run museum as a systemd service](https://gist.github.com/mngshm/a0edb097c91d1dc45aeed755af310323)
2. [Museum.service](https://github.com/ente-io/ente/blob/23e678889189157ecc389c258267685934b83631/server/scripts/deploy/museum.service#L4)
3. [Running Ente Web app as a systemd Service](https://gist.github.com/mngshm/72e32bd483c2129621ed0d74412492fd)
> **Credits:** [mngshm](https://github.com/mngshm)

2
docs/docs/self-hosting/guides/tailscale.md
View File

@@ -347,3 +347,5 @@ This will list all account details. Copy Acount ID.
> ente-museum-1 container from linux terminal. Run
> `docker restart ente-museum-1`. All well, now you will have 100TB storage.
> Repeat if for any other accounts you want to give unlimited storage access.
> **Credits:** [A4alli](https://github.com/A4alli)

195
docs/docs/self-hosting/guides/web-app.md
View File

@@ -1,195 +0,0 @@
---
title: Hosting the web apps
description:
Building and hosting Ente's web apps, connecting it to your self-hosted
server
---
> [!WARNING] NOTE This page covers documentation around self-hosting the web app
> manually. If you want to deploy Ente hassle free, please use the
> [one line](https://ente.io/blog/self-hosting-quickstart/) command to setup
> Ente. This guide might be deprecated in the near future.
# Web app
The getting started instructions mention using `yarn dev` (which is an alias of
`yarn dev:photos`) to serve your web app.
> [!IMPORTANT] Please note that Ente's Web App supports the Yarn version 1.22.xx
> or 1.22.22 specifically. Make sure to install the right version or modify your
> yarn installation to meet the requirements. The user might end up into unknown
> version and dependency related errors if yarn is on different version.
```sh
cd ente/web
yarn install
NEXT_PUBLIC_ENTE_ENDPOINT=http://localhost:8080 yarn dev:photos
```
This is fine for trying the web app and verifying that your self-hosted server
is working as expected etc. But if you would like to use the web app for a
longer term, then it is recommended to follow the Docker approach.
## With Docker/Docker Compose (Recommended)
> [!IMPORTANT]
>
> Recurring changes might be made by the team or from community if more
> improvements can be made so that we are able to build a full-fledged docker
> image.
```dockerfile
FROM node:20-bookworm-slim as builder
WORKDIR ./ente
COPY . .
COPY apps/ .
# Will help default to yarn versoin 1.22.22
RUN corepack enable
# Endpoint for Ente Server
ENV NEXT_PUBLIC_ENTE_ENDPOINT=https://changeme.com
ENV NEXT_PUBLIC_ENTE_ALBUMS_ENDPOINT=https://changeme.com
RUN yarn cache clean
RUN yarn install --network-timeout 1000000000
RUN yarn build:photos && yarn build:accounts && yarn build:auth && yarn build:cast
FROM node:20-bookworm-slim
WORKDIR /app
COPY --from=builder /ente/apps/photos/out /app/photos
COPY --from=builder /ente/apps/accounts/out /app/accounts
COPY --from=builder /ente/apps/auth/out /app/auth
COPY --from=builder /ente/apps/cast/out /app/cast
RUN npm install -g serve
ENV PHOTOS=3000
EXPOSE ${PHOTOS}
ENV ACCOUNTS=3001
EXPOSE ${ACCOUNTS}
ENV AUTH=3002
EXPOSE ${AUTH}
ENV CAST=3003
EXPOSE ${CAST}
# The albums app does not have navigable pages on it, but the
# port will be exposed in-order to self up the albums endpoint
# `apps.public-albums` in museum.yaml configuration file.
ENV ALBUMS=3004
EXPOSE ${ALBUMS}
CMD ["sh", "-c", "serve /app/photos -l tcp://0.0.0.0:${PHOTOS} & serve /app/accounts -l tcp://0.0.0.0:${ACCOUNTS} & serve /app/auth -l tcp://0.0.0.0:${AUTH} & serve /app/cast -l tcp://0.0.0.0:${CAST}"]
```
The above is a multi-stage Dockerfile which creates a production ready static
output of the 4 apps (Photos, Accounts, Auth and Cast) and serves the static
content with Caddy.
Looking at 2 different node base-images doing different tasks in the same
Dockerfile would not make sense, but the Dockerfile is divided into two just to
improve the build efficiency as building this Dockerfile will arguably take more
time.
Lets build a Docker image from the above Dockerfile. Copy and paste the above
Dockerfile contents in the root of your web directory which is inside
`ente/web`. Execute the below command to create an image from this Dockerfile.
```sh
# Build the image
docker build -t <image-name>:<tag> --no-cache --progress plain .
```
You can always edit the Dockerfile and remove the steps for apps which you do
not intend to install on your system (like auth or cast) and opt out of those.
Regarding Albums App, please take a note that they are not web pages with
navigable pages, if accessed on the web-browser they will simply redirect to
ente.web.io.
## compose.yaml
Moving ahead, we need to paste the below contents into the compose.yaml inside
`ente/server/compose.yaml` under the services section.
```yaml
ente-web:
image: <image-name> # name of the image you used while building
ports:
- 3000:3000
- 3001:3001
- 3002:3002
- 3003:3003
- 3004:3004
environment:
- NODE_ENV=development
restart: always
```
Now, we're good to go. All we are left to do now is start the containers.
```sh
docker compose up -d # --build
# Accessing the logs
docker compose logs <container-name>
```
## Configure App Endpoints
> [!NOTE] Previously, this was dependent on the env variables
> `NEXT_ENTE_PUBLIC_ACCOUNTS_ENDPOINT` and etc. Please check the below
> documentation to update your setup configurations
You can configure the web endpoints for the other apps including Accounts,
Albums Family and Cast in your `museum.yaml` configuration file. Checkout
[`local.yaml`](https://github.com/ente-io/ente/blob/543411254b2bb55bd00a0e515dcafa12d12d3b35/server/configurations/local.yaml#L76-L89)
to configure the endpoints. Make sure to setup up your DNS Records accordingly
to the similar URL's you set up in `museum.yaml`.
Next part is to configure the web server.
# Web server configuration
The last step ahead is configuring reverse_proxy for the ports on which the apps
are being served (you will have to make changes, if you have cusotmized the
ports). The web server of choice in this guide is
[Caddy](https://caddyserver.com) because with caddy you don't have to manually
configure/setup SSL ceritifcates as caddy will take care of that.
```sh
photos.yourdomain.com {
reverse_proxy http://localhost:3001
# for logging
log {
level error
}
}
auth.yourdomain.com {
reverse_proxy http://localhost:3002
}
# and so on ...
```
Next, start the caddy server :).
```sh
# If caddy service is not enabled
sudo systemctl enable caddy
sudo systemctl daemon-reload
sudo systemctl start caddy
```
## Contributing
Please start a discussion on the Github Repo if you have any suggestions for the
Dockerfile, You can also share your setups on Github Discussions.

130
docs/docs/self-hosting/install/config-file.md → docs/docs/self-hosting/install/config.md
View File

@@ -1,35 +1,53 @@
---
title: "Configuration File - Self-hosting"
title: "Configuration - Self-hosting"
description:
"Information about all the configuration variables needed to run Ente with
museum.yaml"
---
# Configuration File
# Configuration
Museum uses YAML-based configuration file that is used for handling database
connectivity, bucket configuration, internal configuration, etc.
Museum is designed to be configured either via environment variables or via
YAML. We recommend using YAML for maintaining your configuration as it can be
backed up easily, helping in restoration.
If the `ENVIRONMENT` environment variable is set, a corresponding file from
`configurations/` is loaded, by default, `local.yaml` configuration is loaded.
## Configuration File
Museum's configuration file (`museum.yaml`) is responsible for making database
configuration, bucket configuration, internal configuration, etc. accessible for
other internal services.
By default, Museum runs in local environment, thus `local.yaml` configuration is
loaded.
If `ENVIRONMENT` environment variable is set (say, to `production`), Museum will
attempt to load `configurations/production.yaml`.
If `credentials-file` is defined and found, it overrides the defaults.
Self-hosted clusters generally use `museum.yaml` file for reading configuration
specifications.
Self-hosted clusters generally use `museum.yaml` file for declaring
configuration over directly editing `local.yaml`.
All configuration values can be overridden via environment variables using the
`ENTE_` prefix and replacing dots (`.`) or hyphens (`-`) with underscores (`_`).
The configuration variables declared in `museum.yaml` are read by Museum.
Additionally, environment variables prefixed by `ENTE_` are read by Museum and
used internally in same manner as configuration variables.
Museum reads configuration from `museum.yaml`. Any environment variables
prefixed with `ENTE_` takes precedence.
For example, `s3.b2-eu-cen` in `museum.yaml` and `ENTE_S3_B2_EU_CEN` declared as
environment variable are the same and `ENTE_S3_B2_EU_CEN` overrides
`s3.b2-eu-cen`.
For example,
## General Settings
```yaml
s3:
b2-eu-cen:
endpoint:
```
in `museum.yaml` is read as `s3.b2-eu-cen.endpoint` by Museum.
`ENTE_S3_B2_EU_CEN_ENDPOINT` declared as environment variable is same as the
above and `ENTE_S3_B2_EU_CEN_ENDPOINT` overrides `s3.b2-eu-cen.endpoint`.
### General Settings
| Variable | Description | Default |
| ------------------ | --------------------------------------------------------- | ------------------ |
@@ -37,13 +55,21 @@ environment variable are the same and `ENTE_S3_B2_EU_CEN` overrides
| `credentials-dir` | Directory to look for credentials (TLS, service accounts) | `credentials/` |
| `log-file` | Log output path. Required in production. | `""` |
## HTTP
### HTTP
| Variable | Description | Default |
| -------------- | --------------------------------- | ------- |
| `http.use-tls` | Enables TLS and binds to port 443 | `false` |
## App Endpoints
### App Endpoints
The web apps for Ente (Auth, Cast, Albums) use different endpoints.
These endpoints are configurable in `museum.yaml` under the apps.\* section.
Upon configuration, the application will start utilizing the specified endpoints
instead of Ente's production instances or local endpoints (overridden values
used for Compose and quickstart for ease of use.)
| Variable | Description | Default |
| -------------------- | ------------------------------------------------------- | -------------------------- |
@@ -51,7 +77,7 @@ environment variable are the same and `ENTE_S3_B2_EU_CEN` overrides
| `apps.cast` | Cast app base endpoint | `https://cast.ente.io` |
| `apps.accounts` | Accounts app base endpoint (used for passkey-based 2FA) | `https://accounts.ente.io` |
## Database
### Database
| Variable | Description | Default |
| ------------- | -------------------------- | ----------- |
@@ -63,7 +89,19 @@ environment variable are the same and `ENTE_S3_B2_EU_CEN` overrides
| `db.password` | Database password | |
| `db.extra` | Additional DSN parameters | |
## Object Storage
### Object Storage
The `s3` section within `museum.yaml` is by default configured to use local
MinIO buckets when using `quickstart.sh` or Docker Compose.
If you wish to use an external S3 provider, you can edit the configuration with
your provider's credentials, and set `are_local_buckets` to `false`.
MinIO uses the port `3200` for API Endpoints. Web Console can be accessed at
http://localhost:3201 by enabling port `3201` in the Compose file.
If you face any issues related to uploads then checkout [Troubleshooting bucket
CORS] and [Frequently encountered S3 errors].
| Variable | Description | Default |
| -------------------------------------- | -------------------------------------------- | ------- |
@@ -74,20 +112,39 @@ environment variable are the same and `ENTE_S3_B2_EU_CEN` overrides
| `s3.are_local_buckets` | Use local MinIO-compatible storage | `false` |
| `s3.use_path_style_urls` | Enable path-style URLs for MinIO | `false` |
## Encryption Keys
### Encryption Keys
These values are used for encryption of user e-mails. Default values are
provided by Museum.
They are generated by random in quickstart script, so no intervention is
necessary if using quickstart.
However, if you are using Ente for long-term needs and you have not installed
Ente via quickstart, consider generating values for these along with [JWT](#jwt)
by following the steps described below:
```shell
# If you have not cloned already
git clone https://github.com/ente-io/ente
# Generate the values
cd ente/server
go run tools/gen-random-keys/main.go
```
| Variable | Description | Default |
| ---------------- | -------------------------------------- | ----------- |
| ---------------- | ------------------------------ | ----------- |
| `key.encryption` | Key for encrypting user emails | Pre-defined |
| `key.hash` | Hash key for verifying email integrity | Pre-defined |
| `key.hash` | Hash key | Pre-defined |
## JWT
### JWT
| Variable | Description | Default |
| ------------ | ----------------------- | ---------- |
| `jwt.secret` | Secret for signing JWTs | Predefined |
## Email
### Email
| Variable | Description | Default |
| ------------------ | ---------------------------- | ------- |
@@ -99,34 +156,47 @@ environment variable are the same and `ENTE_S3_B2_EU_CEN` overrides
| `smtp.sender-name` | Custom name for email sender | |
| `transmail.key` | Zeptomail API key | |
## WebAuthn Passkey Support
### WebAuthn Passkey Support
| Variable | Description | Default |
| -------------------- | ---------------------------- | --------------------------- |
| `webauthn.rpid` | Relying Party ID | `localhost` |
| `webauthn.rporigins` | Allowed origins for WebAuthn | `["http://localhost:3001"]` |
## Internal
### Internal
| Variable | Description | Default |
| ------------------------------- | --------------------------------------------- | ------- |
| -------------------------------------------- | --------------------------------------------- | ------- |
| `internal.silent` | Suppress external effects (e.g. email alerts) | `false` |
| `internal.health-check-url` | External healthcheck URL | |
| `internal.hardcoded-ott` | Predefined OTPs for testing | |
| `internal.hardcoded-ott.emails` | E-mail addresses with hardcoded OTTs | `[]` |
| `internal.hardcoded-ott.local-domain-suffix` | Suffix for which hardcoded OTT is to be used | |
| `internal.hardcoded-ott.local-domain-value` | Hardcoded OTT value for the above suffix | |
| `internal.admins` | List of admin user IDs | `[]` |
| `internal.admin` | Single admin user ID | |
| `internal.disable-registration` | Disable user registration | `false` |
## Replication
### Replication
By default, replication of objects (photos, thumbnails, videos) is disabled and
only one bucket is used.
To enable replication, set `replication.enabled` to `true`. For this to work, 3
buckets have to be configured in total.
| Variable | Description | Default |
| -------------------------- | ------------------------------------ | ----------------- |
| `replication.enabled` | Enable cross-datacenter replication | `false` |
| `replication.enabled` | Enable replication across buckets | `false` |
| `replication.worker-url` | Cloudflare Worker for replication | |
| `replication.worker-count` | Number of goroutines for replication | `6` |
| `replication.tmp-storage` | Temp directory for replication | `tmp/replication` |
## Background Jobs
### Background Jobs
This configuration is for enabling background cron jobs for tasks such as
sending mails, removing unused objects (clean up) and worker configuration for
the same.
| Variable | Description | Default |
| --------------------------------------------- | --------------------------------------- | ------- |

4
docs/docs/self-hosting/install/without-docker.md → docs/docs/self-hosting/install/manual.md
View File

@@ -1,9 +1,9 @@
---
title: Running Ente Without Docker - Self-hosting
title: Manual Setup (Without Docker) - Self-hosting
description: Installing and setting up Ente without Docker
---
# Running Ente without Docker
# Manual Setup (Without Docker)
If you wish to run Ente from source without using Docker, follow the steps described below:

127
docs/docs/self-hosting/install/post-install/index.md
View File

@@ -9,45 +9,119 @@ A list of steps that should be done after installing Ente are described below:
## Step 1: Creating first user
The first user (and the only user) to be created will be treated as an admin user by default.
The first user to be created will be treated as an admin user by default.
Once Ente is up and running, the Ente Photos web app will be accessible on
`http://localhost:3000`. Open this URL in your browser and proceed with creating
an account.
To complete your account registration you will need to enter a 6-digit
verification code.
Select **Don't have an account?** to create a new user.
This code can be found in the server logs, which should already be shown in your
quickstart terminal. Alternatively, you can open the server logs with the
following command from inside the `my-ente` folder:
![Onboarding Screen](/onboarding.png)
```sh
Follow the prompts to sign up.
![Sign Up Page](/sign-up.png)
Enter a 6-digit verification code to complete registration.
This code can be found in the server logs, which should be shown in your terminal where you started the Docker Compose cluster.
If not, access the server logs inside the folder where Compose file resides.
``` shell
sudo docker compose logs
```
If running Museum without Docker, the code should be visible in the terminal (stdout).
![otp](/otp.png)
## Step 2: Download mobile and desktop app
## Step 2: Whitelist admins
## Step 2: Configure apps to use your server
1. Connect to `ente_db` (the database used for storing data
related to Ente).
``` shell
# Change the DB name and DB user name if you use different values.
# If using Docker
docker exec -it <postgres-ente-container-name> psql -U pguser -d ente_db
# Or when using psql directly
psql -U pguser -d ente_db
```
2. Get the user ID of the first user by running the following PSQL command:
``` sql
SELECT * from users;
```
3. Edit `internal.admins` or `internal.admin` (if you wish to whitelist only single user) in `museum.yaml`
to add the user ID you wish to whitelist.
- For multiple admins:
``` yaml
internal:
admins:
- <user_id>
```
- For single admin:
``` yaml
internal:
admin: <user_id>
```
4. Restart Museum by restarting the cluster
::: tip
Restart your Compose clusters whenever you make changes to Compose file or edit configuration file.
:::
## Step 3: Configure application endpoints
You may wish to access some of the applications such as Auth, Albums, Cast via your instance's endpoints through the application instead of our production instances.
You can do so by editing the `apps` section in `museum.yaml` to use the base endpoints of the corresponding web applications.
``` yaml
# Replace yourdomain.tld with actual domain
apps:
public-albums: https://albums.ente.yourdomain.tld
cast: https://cast.ente.yourdomain.tld
auth: https://auth.ente.yourdomain.tld
```
## Step 4: Make it publicly accessible
You may wish to access Ente on public Internet. You can do so by configuring a reverse proxy
with software such as Caddy, NGINX, Traefik.
Check out our [documentation](/self-hosting/administration/reverse-proxy) for more information.
If you do not wish to make it accessible via Internet, we recommend you to use
[Tailscale](/self-hosting/guides/tailscale) for convenience. Alternately, you
can use your IP address for accessing the application in your local network, though
this poses challenges with respect to object storage.
## Step 5: Download mobile and desktop app
You can install Ente Photos by following the [installation section](/photos/faq/installing).
You can also install Ente Auth (if you are planning to use Auth) by following the [installation section](/auth/faq/installing).
## Step 6: Configure apps to use your server
You can modify Ente mobile apps and CLI to connect to your server.
### Mobile
The pre-built Ente apps from GitHub / App Store / Play Store / F-Droid can be
easily configured to use a custom server.
You can tap 7 times on the onboarding screen to bring up a page where you can
configure the endpoint the app should be connecting to.
Tap 7 times on the onboarding screen to configure the
server endpoint to be used.
![Setting a custom server on the onboarding screen](custom-server.png)
### Desktop
Same as the mobile app, you can tap 7 times on the onboarding screen to
configure the endpoint the app should connect to.
Tap 7 times on the onboarding screen to configure the server endpoint to be used.
<div align="center">
@@ -56,21 +130,12 @@ apps](web-dev-settings.png){width=400px}
</div>
## Step 3: Configure Ente CLI
## Step 7: Configure Ente CLI
> [!NOTE]
>
> You can download the CLI from
> [here](https://github.com/ente-io/ente/releases?q=tag%3Acli-v0)
You can download Ente CLI from [here](https://github.com/ente-io/ente/releases?q=tag%3Acli)
Define a config.yaml and put it either in the same directory as where you run
the CLI from ("current working directory"), or in the path defined in env
variable `ENTE_CLI_CONFIG_PATH`:
Check our [documentation](/self-hosting/administration/cli) on how to use Ente CLI for managing self-hosted instances.
```yaml
endpoint:
api: "http://localhost:8080"
```
(Another
[example](https://github.com/ente-io/ente/blob/main/cli/config.yaml.example))
::: info For upgradation
Check out our [documentation](/self-hosting/install/upgrade) for various installation methods.
:::

12
docs/docs/self-hosting/install/requirements.md
View File

@@ -5,19 +5,25 @@ description: Requirements for self-hosting Ente
# Requirements
Ensure your system meets these requirements and has the needed software installed for a smooth experience.
## Hardware
The server is capable of running on minimal resource requirements as a
lightweight Go binary, since most of the intensive computational tasks are done on the client. It performs well on small cloud instances, old laptops, and even
lightweight Go binary, since most of the intensive computational tasks are done on the client.
It performs well on small cloud instances, old laptops, and even
[low-end embedded devices](https://github.com/ente-io/ente/discussions/594).
- **Storage:** An Unix-compatible filesystem such as ZFS, EXT4, BTRFS, etc. if using PostgreSQL container as it requires a filesystem that supports user/group permissions.
- **Storage:** An Unix-compatible filesystem such as ZFS, EXT4, BTRFS, etc. if using
PostgreSQL container as it requires a filesystem that supports user/group permissions.
- **RAM:** A minimum of 1 GB of RAM is required for running the cluster (if using quickstart script).
- **CPU:** A minimum of 1 CPU core is required.
## Software
- **Operating System:** Any Linux or \*nix operating system, Ubuntu or Debian is recommended to have a good Docker experience. Non-Linux operating systems tend to provide poor experience with Docker and difficulty with troubleshooting and assistance.
- **Operating System:** Any Linux or \*nix operating system, Ubuntu or Debian is recommended
to have a good Docker experience. Non-Linux operating systems tend to provide poor
experience with Docker and difficulty with troubleshooting and assistance.
- **Docker:** Required for running Ente's server, web application and dependent services
(database and object storage). Ente also requires **Docker Compose plugin** to be installed.

69
docs/docs/self-hosting/install/upgrade.md Normal file
View File

@@ -0,0 +1,69 @@
---
title: Upgrade - Self-hosting
description: Upgradation of self-hosted Ente
---
# Upgrade your server
Upgrading Ente depends on the method of installation you have chosen.
## Quickstart
Upgrade and restart Ente by pulling the latest images in the directory where the Compose file resides.
The directory name is generally `my-ente`.
Run this command inside `my-ente/`
``` shell
docker compose pull && docker compose up -d
```
## Docker Compose
You can pull in the latest source code from Git and build a new cluster
based on the updated source code.
1. Pull the latest changes from `main`.
``` shell
# Assuming you have cloned repository to ente
cd ente
# Pull changes
git pull
```
2. Recreate the cluster.
``` shell
cd server/config
# Stop and remove containers if they are running
docker compose down
# Build with latest code
docker compose up --build
```
## Manual Setup
You can pull in the latest source code from Git and build a new cluster
based on the updated source code.
1. Pull the latest changes from `main`.
``` shell
# Assuming you have cloned repository to ente
cd ente
# Pull changes
git pull
```
2. Follow the steps described in [manual setup](/self-hosting/install/manual) for Museum and web applications.
::: tip
If using Docker, you can free up some disk space by deleting older images
that were used by obsolette containers
``` shell
docker image prune
```
:::

10
docs/docs/self-hosting/install/upgrading.md
View File

@@ -1,10 +0,0 @@
---
title: Upgrading - Self-hosting
description: Upgradation of self-hosted Ente
---
# Upgrading
A list of steps that should be done after installing Ente are described below:
## Step 1:

56
docs/docs/self-hosting/troubleshooting/docker.md
View File

@@ -1,19 +1,19 @@
---
title: Docker Errors
description: Fixing docker related errors when trying to self host Ente
title: Troubleshooting Docker-related errors - Self-hosting
description: Fixing Docker-related errors when trying to self-host Ente
---
# Docker
# Troubleshooting Docker-related errors
## configs
Remember to restart your cluster to ensure changes that you make in the
`configs` section in `compose.yaml` get picked up.
```sh
docker compose down
docker compose up
```
> [!TIP] Restart after changes
>
> Remember to restart your cluster to ensure changes that you make in the
> `compose.yaml` and `museum.yaml` get picked up.
>
> ``` shell
> docker compose down
> docker compose up
> ```
## post_start
@@ -90,29 +90,23 @@ museum-1 | /etc/ente/cmd/museum/main.go:124 +0x44c
museum-1 exited with code 2
```
Then the issue is that the password you're using is not the password postgres is
expecting (duh), and a potential scenario where that can happen is something
like this:
Then the issue is that the password you're using is not the password PostgreSQL is
expecting.
1. On a machine, you create a new cluster with `quickstart.sh`.
There are 2 possibilities:
2. Later you delete that folder, but then create another cluster with
`quickstart.sh`. Each time `quickstart.sh` runs, it creates new credentials,
and then when it tries to spin up the docker compose cluster, use them to
connect to the postgres running within.
1. When you have created a cluster in `my-ente` directory on
running `quickstart.sh` and later deleted it, only to create
another cluster with same `my-ente` directory.
3. However, you would already have a docker volume from the first run of
`quickstart.sh`. Since the folder name is the same in both cases `my-ente`,
Docker will reuse the existing volumes (`my-ente_postgres-data`,
`my-ente_minio-data`). So your postgres is running off the old credentials,
and you're trying to connect to it using the new ones, and the error arises.
However, by deleting the directory, the Docker volumes are not deleted.
The solution is to delete the stale docker volume. **Be careful**, this will
delete all data in those volumes (any thing you uploaded etc), so first
understand if this is the exact problem you are facing before deleting those
volumes.
Thus the older volumes with previous cluster's credentials are used
for new cluster and the error arises.
If you're sure of what you're doing, the volumes can be deleted by
Deletion of the stale Docker volume can solve this. **Be careful**, this will
delete all data in those volumes (any thing you uploaded etc). Do this
if you are sure this is the exact problem.
```sh
docker volume ls
@@ -122,7 +116,7 @@ to list them, and then delete the ones that begin with `my-ente` using
`docker volume rm`. You can delete all stale volumes by using
`docker system prune` with the `--volumes` flag, but be _really_ careful,
that'll delete all volumes (Ente or otherwise) on your machine that are not
currently in use by a running docker container.
currently in use by a running Docker container.
An alternative way is to delete the volumes along with removal of cluster's
containers using `docker compose` inside `my-ente` directory.

2
docs/docs/self-hosting/troubleshooting/misc.md
View File

@@ -3,6 +3,8 @@ title: General troubleshooting cases
description: Fixing various errors when trying to self host Ente
---
# Troubleshooting
## Functionality not working on self hosted instance
If some specific functionality (e.g. album listing, video playback) does not

1
server/.gitignore vendored
View File

@@ -13,3 +13,4 @@ __debug_bin*
config/.env
config/museum.yaml
main
credentials

19
server/config/example.yaml
View File

@@ -51,6 +51,16 @@ s3:
region: eu-central-2
bucket: scw-eu-fr-v3
# Specify the base endpoints for various web apps
apps:
# If you're running a self hosted instance and wish to serve public links,
# set this to the URL where your albums web app is running.
public-albums: http://localhost:3002
cast: http://localhost:3004
# Set this to the URL where your accounts web app is running, primarily used for
# passkey based 2FA.
accounts: http://localhost:3001
# Key used for encrypting customer emails before storing them in DB
#
# To make it easy to get started, some randomly generated (but fixed) values are
@@ -65,12 +75,3 @@ key:
jwt:
secret: i2DecQmfGreG6q1vBj5tCokhlN41gcfS2cjOs9Po-u8=
# Specify the base endpoints for various web apps
apps:
# If you're running a self hosted instance and wish to serve public links,
# set this to the URL where your albums web app is running.
public-albums: http://localhost:3002
cast: http://localhost:3004
# Set this to the URL where your accounts web app is running, primarily used for
# passkey based 2FA.
accounts: http://localhost:3001