TerribleDev/ente
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[docs] self-hosting docs clean up & revamp (part x/x) (#5647)

Browse Source
This commit is contained in:
Neeraj
2025-04-21 10:43:57 +05:30
committed by GitHub
parent 8dd9dc16ad 93dd0c4943
commit 5dea3fd8b0
8 changed files with 455 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

90
docs/docs/.vitepress/sidebar.ts
View File

@@ -238,74 +238,43 @@ export const sidebar = [
items: [
{ text: "Getting started", link: "/self-hosting/" },
{
text: "System requirements",
link: "/self-hosting/guides/system-requirements",
text: "Connect to custom server",
link: "/self-hosting/guides/custom-server/",
},
{
text: "Configuring S3",
link: "/self-hosting/guides/configuring-s3",
},
{
text: "Guides",
items: [
{ text: "Introduction", link: "/self-hosting/guides/" },
{
text: "Connect to custom server",
link: "/self-hosting/guides/custom-server/",
},
{
text: "Hosting the web app",
link: "/self-hosting/guides/web-app",
},
{
text: "Configuring S3",
link: "/self-hosting/guides/configuring-s3",
},
{
text: "Hosting Ente with external S3 (Community)",
link: "/self-hosting/guides/external-s3",
},
{
text: "DB migration",
link: "/self-hosting/guides/db-migration",
text: "Hosting Ente from source",
link: "/self-hosting/guides/from-source",
},
{
text: "Hosting Ente without Docker",
link: "/self-hosting/guides/standalone-ente",
},
{
text: "Ente via Tailscale (Community)",
link: "/self-hosting/guides/Tailscale.md",
text: "Administering your server",
link: "/self-hosting/guides/admin",
},
{
text: "Configure CLI for Self Hosted Instance",
link: "/self-hosting/guides/selfhost-cli",
},
{
text: "Administering your server",
link: "/self-hosting/guides/admin",
text: "DB migration",
link: "/self-hosting/guides/db-migration",
},
{
text: "Mobile build",
link: "/self-hosting/guides/mobile-build",
},
],
},
{
text: "FAQ",
items: [
{ text: "General", link: "/self-hosting/faq/" },
{
text: "Verification code",
link: "/self-hosting/faq/otp",
},
{
text: "Shared albums",
link: "/self-hosting/faq/sharing",
},
{
text: "Backups",
link: "/self-hosting/faq/backup",
},
],
},
{
text: "Troubleshooting",
items: [
@@ -313,6 +282,10 @@ export const sidebar = [
text: "General",
link: "/self-hosting/troubleshooting/misc",
},
{
text: "Bucket CORS",
link: '/self-hosting/troubleshooting/bucket-cors'
},
{
text: "Uploads",
link: "/self-hosting/troubleshooting/uploads",
@@ -331,6 +304,37 @@ export const sidebar = [
},
],
},
{
text: "Community Guides",
items :[
{
text: "Ente via Tailscale",
link: "/self-hosting/guides/Tailscale",
},
{
text: "Ente with External S3",
link: "/self-hosting/guides/external-s3",
}
]
},
{
text: "FAQ",
items: [
{ text: "General", link: "/self-hosting/faq/" },
{
text: "Verification code",
link: "/self-hosting/faq/otp",
},
{
text: "Shared albums",
link: "/self-hosting/faq/sharing",
},
{
text: "Backups",
link: "/self-hosting/faq/backup",
},
],
},
],
},
{

BIN
docs/docs/public/cloudflare.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 63 KiB

BIN
docs/docs/public/endpoint.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 33 KiB

BIN
docs/docs/public/quickstart.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 134 KiB

228
docs/docs/self-hosting/guides/from-source.md Normal file
View File

@@ -0,0 +1,228 @@
---
title: Ente from Source
description: Getting started self hosting Ente Photos and/or Ente Auth
---
# Ente from Source
> [!WARNING] NOTE
> The below documentation will cover instructions about self-hosting the web app manually. If you
> want to deploy Ente hassle free, use the [one line](https://ente.io/blog/self-hosting-quickstart/)
> command to setup Ente. This guide might be deprecated in the near future.
## Installing Docker
Refer to
[How to install Docker from the APT repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
for detailed instructions.
## Start the server
```sh
git clone https://github.com/ente-io/ente
cd ente/server
docker compose up --build
```
> [!TIP]
>
> You can also use a pre-built Docker image from `ghcr.io/ente-io/server`
> ([More info](https://github.com/ente-io/ente/blob/main/server/docs/docker.md))
Install the necessary dependencies for running the web client
```sh
# installing npm and yarn
sudo apt update
sudo apt install nodejs npm
sudo npm install -g yarn // to install yarn globally
```
Then in a separate terminal, you can run (e.g) the web client
```sh
cd ente/web
git submodule update --init --recursive
yarn install
NEXT_PUBLIC_ENTE_ENDPOINT=http://localhost:8080 yarn dev
```
That's about it. If you open http://localhost:3000, you will be able to create
an account on a Ente Photos web app running on your machine, and this web app
will be connecting to the server running on your local machine at
`localhost:8080`.
For the mobile apps, you don't even need to build, and can install normal Ente
apps and configure them to use your
[custom self-hosted server](/self-hosting/guides/custom-server/).
> If you want to build the mobile apps from source, see the instructions
> [here](/self-hosting/guides/mobile-build).
## Web app with Docker and Compose
The instructoins in previous section were just a temporary way to run the web app locally.
To run the web apps as services, the user has to build a docker image manually.
> [!IMPORTANT]
>
> Recurring changes might be made by the team or from community if more
> improvements can be made so that we are able to build a full-fledged docker
> image.
```dockerfile
FROM node:20-bookworm-slim as builder
WORKDIR ./ente
COPY . .
COPY apps/ .
# Will help default to yarn versoin 1.22.22
RUN corepack enable
# Endpoint for Ente Server
ENV NEXT_PUBLIC_ENTE_ENDPOINT=https://your-ente-endpoint.com
ENV NEXT_PUBLIC_ENTE_ALBUMS_ENDPOINT=https://your-albums-endpoint.com
RUN yarn cache clean
RUN yarn install --network-timeout 1000000000
RUN yarn build:photos && yarn build:accounts && yarn build:auth && yarn build:cast
FROM node:20-bookworm-slim
WORKDIR /app
COPY --from=builder /ente/apps/photos/out /app/photos
COPY --from=builder /ente/apps/accounts/out /app/accounts
COPY --from=builder /ente/apps/auth/out /app/auth
COPY --from=builder /ente/apps/cast/out /app/cast
RUN npm install -g serve
ENV PHOTOS=3000
EXPOSE ${PHOTOS}
ENV ACCOUNTS=3001
EXPOSE ${ACCOUNTS}
ENV AUTH=3002
EXPOSE ${AUTH}
ENV CAST=3003
EXPOSE ${CAST}
# The albums app does not have navigable pages on it, but the
# port will be exposed in-order to self up the albums endpoint
# `apps.public-albums` in museum.yaml configuration file.
ENV ALBUMS=3004
EXPOSE ${ALBUMS}
CMD ["sh", "-c", "serve /app/photos -l tcp://0.0.0.0:${PHOTOS} & serve /app/accounts -l tcp://0.0.0.0:${ACCOUNTS} & serve /app/auth -l tcp://0.0.0.0:${AUTH} & serve /app/cast -l tcp://0.0.0.0:${CAST}"]
```
The above is a multi-stage Dockerfile which creates a production ready static
output of the 4 apps (Photos, Accounts, Auth and Cast) and serves the static
content with Caddy.
Looking at 2 different node base-images doing different tasks in the same
Dockerfile would not make sense, but the Dockerfile is divided into two just to
improve the build efficiency as building this Dockerfile will arguably take more
time.
Lets build a Docker image from the above Dockerfile. Copy and paste the above
Dockerfile contents in the root of your web directory which is inside
`ente/web`. Execute the below command to create an image from this Dockerfile.
```sh
# Build the image
docker build -t <image-name>:<tag> --no-cache --progress plain .
```
You can always edit the Dockerfile and remove the steps for apps which you do
not intend to install on your system (like auth or cast) and opt out of those.
Regarding Albums App, take a note that they are not apps with navigable pages,
if accessed on the web-browser they will simply redirect to ente.web.io.
## compose.yaml
Moving ahead, we need to paste the below contents into the compose.yaml inside
`ente/server/compose.yaml` under the services section.
```yaml
ente-web:
image: <image-name> # name of the image you used while building
ports:
- 3000:3000
- 3001:3001
- 3002:3002
- 3003:3003
- 3004:3004
environment:
- NODE_ENV=development
restart: always
```
Now, we're good to go. All we are left to do now is start the containers.
```sh
docker compose up -d # --build
# Accessing the logs
docker compose logs <container-name>
```
## Configure App Endpoints
> [!NOTE]
> Previously, this was dependent on the env variables `NEXT_ENTE_PUBLIC_ACCOUNTS_ENDPOINT`
> and etc. Please check the below documentation to update your setup configurations
You can configure the web endpoints for the other apps including Accounts, Albums
Family and Cast in your `museum.yaml` configuration file. Checkout
[`local.yaml`](https://github.com/ente-io/ente/blob/543411254b2bb55bd00a0e515dcafa12d12d3b35/server/configurations/local.yaml#L76-L89)
to configure the endpoints. Make sure to setup up your DNS Records accordingly to the
similar URL's you set up in `museum.yaml`.
Next part is to configure the web server.
# Web server configuration
The last step ahead is configuring reverse_proxy for the ports on which the apps
are being served (you will have to make changes, if you have cusotmized the
ports). The web server of choice in this guide is
[Caddy](https://caddyserver.com) because with caddy you don't have to manually
configure/setup SSL ceritifcates as caddy will take care of that.
```sh
photos.yourdomain.com {
reverse_proxy http://localhost:3001
# for logging
log {
level error
}
}
auth.yourdomain.com {
reverse_proxy http://localhost:3002
}
# and so on ...
```
Next, start the caddy server :).
```sh
# If caddy service is not enabled
sudo systemctl enable caddy
sudo systemctl daemon-reload
sudo systemctl start caddy
```
## Contributing
Please start a discussion on the Github Repo if you have any suggestions for the
Dockerfile, You can also share your setups on Github Discussions.

6
docs/docs/self-hosting/guides/web-app.md
View File

@@ -5,6 +5,12 @@ description:
server
---
> [!WARNING] NOTE
> This page covers documentation around self-hosting the web app manually. If you
> want to deploy Ente hassle free, please use the [one line](https://ente.io/blog/self-hosting-quickstart/)
> command to setup Ente. This guide might be deprecated in the near future.
# Web app
The getting started instructions mention using `yarn dev` (which is an alias of

179
docs/docs/self-hosting/index.md
View File

@@ -14,100 +14,145 @@ the same code we use for our own cloud service.
> [blog post](https://ente.io/blog/open-sourcing-our-server/) announcing the
> open sourcing of our server useful.
## Getting started - Quickstart
Install [Docker](https://www.docker.com). Then, paste the following command in a
your terminal:
## System Requirements
The server has minimal resource requirements, running as a lightweight Go binary
with no server-side ML. It performs well on small cloud instances, old laptops,
and even [low-end embedded devices](https://github.com/ente-io/ente/discussions/594)
reported by community members. Virtually any reasonable hardware should be sufficient.
## Getting started
Execute the below one-liner command in your terminal to setup Ente on your system.
```sh
sh -c "$(curl -fsSL https://raw.githubusercontent.com/ente-io/ente/main/server/quickstart.sh)"
```
> [!TIP]
>
> For more details about what this does, see [the quickstart
> README](https://github.com/ente-io/ente/blob/main/server/docs/quickstart.md).
The above `curl` command is a simple shell-script, which pulls the docker images,
creates a directory `my-ente` in the current working directory and starts all the
containers required to run Ente on your system.
That's about it. If you open http://localhost:3000 from the machine where the
server is running, you will be able to create an account on a Ente Photos web
app. This web app will be connecting to the server running on your local machine
at `localhost:8080`.
![quickstart](/quickstart.png)
To complete your account registration you need to enter a 6-digit verification
code. These can be found in the server logs which should already be shown in
your quickstart terminal. Otherwise you can open the server logs with the
following command from inside the `my-ente` folder:
The docker containers will be up and listening on their desired ports. The Ente Photos
web app will be accessible on `http://localhost:3000`. Open the URL in your browser
and proceed with creating an account. By default, the API Endpoint will be `localhost:8080`
as Museum (our server endpoint) will listen on `:8080`.
```sh
![endpoint](/endpoint.png)
To complete your account registration you need to enter a 6-digit verification code.
This can be found in the server logs, which should already be shown in your quickstart
terminal. Alternatively, you can open the server logs with the following command from
inside the `my-ente` folder:
```sh
sudo docker compose logs
```
In the logs, find the code at the end of a message that resembles the following:
```sh
museum | INFO[0102]email.go:130 sendViaTransmail Skipping sending email to email@example.com: *Verification code: 112089*
```
## Reverse Proxy
There are [prebuilt apps](https://ente.io/download) for iPad, iPhone, Android,
Linux, Mac, and Windows. These can easily be configured to use your [custom
self-hosted server](guides/custom-server/).
This step isn't really the direct next step after creating an account. It is
one of the most essential steps to avoid certain CORS errors and will help you through
the configuration coming ahead.
## Getting started - From source
Museum runs on port `:8080`, Ente Photos web app runs on `:3000` and so on the other apps
are lined up after each other from ports `3001-3004`.
The quickstart method above uses pre-built images. Alternatively, if you want to
build the self hosted server images from source, you can use the steps in this
section.
We highly recommend using HTTPS for Museum (`8080`). Primarily, because for security reasons Museum
won't accept any incoming HTTP traffic. Hence, all the requests will fail.
#### Installing Docker
Head over to your DNS Management Dashboard and setup the appropriate records for the endpoints.
Mostly, `A` or `AAAA` records targeting towards your server's IP address should be sufficient. The rest of the work
will be done by the web server sitting on your server machine.
Refer to
[How to install Docker from the APT repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
for detailed instructions.
![cloudflare](/cloudflare.png)
#### Start the server
### With Caddy
Setting up a reverse proxy with Caddy is pretty easy and straightforward. Firstly, install Caddy
on your server machine.
```sh
git clone https://github.com/ente-io/ente
cd ente/server
docker compose up --build
sudo apt install caddy
```
After the installation is complete, a `Caddyfile` is created on the path `/etc/caddy/`. This file is
used to configure reverse proxies and a whole lot of different things.
```yaml
# Caddyfile - myente.xyz is just an example.
api.myente.xyz {
reverse_proxy http://localhost:8080
}
ente.myente.xyz {
reverse_proxy http://localhost:3000
}
```
Install the necessary dependencies for running the web client
After a few hard-reloads, Ente Photos web app should be working on https://ente.myente.xyz. You can check out
the documentation for any other reverse proxy tool (like nginx) you want to use.
```sh
# installing npm and yarn
## Configuring `museum.yaml`
sudo apt update
sudo apt install nodejs npm
sudo npm install -g yarn // to install yarn globally
`Museum.yaml` is a YAML configuration file used to configure various things for museum.
By default, [`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
is also available, but it is overridden if `museum.yaml` file is found. We highly
recommend creating and building your own `museum.yaml` instead of editing `configurations/local.yaml`.
The `my-ente` directory will include a `museum.yaml` file with some configurations around encryption
keys and secrets, postgres DB, and MinIO.
> [!TIP]
> Always do `docker compose down` inside `my-ente` directory, if you've made any changes to `museum.yaml`
> and then restart the containers with `docker compose up -d ` to see the changes in action.
### S3 Buckets
By default, the `s3` section is configured to use local minIO buckets and for the same reason
`are_local_buckets` is set to `true`. If you wish to bring any external S3 provider,
you just have to edit the configuration with appropriate credentails and details given by the provider.
And set `are_local_buckets` to false. Check out [Configuring S3](/self-hosting/guides/configuring-s3.md)
to understand more on how to configure S3 buckets and how the communication happens.
MinIO makes use of the port `3200` for API Endpoints and the Client Web App is run over `:3201`
(both on localhost). You can login to MinIO Console Web UI by accessing `localhost:3201` in your web-browser
and setting up all the things related to regions there itself.
If you face any issues related to uploads then checkout
[Troubleshooting Bucket CORS](/self-hosting/troubleshooting/bucket-cors) and
[Frequently Answered Error related to S3](/self-hosting/guides/configuring-s3#fae-frequently-answered-errors)
### App Endpoints
Ente Photos Web app is divided into multiple sub-apps like albums, cast, auth, etc.
These endpoints are configurable in the museum.yaml under the `apps.*` section.
For example,
```yaml
apps:
public-albums: albums.myente.xyz
cast: cast.myente.xyz
accounts: accounts.myente.xyz
family: family.myente.xyz
```
Then in a separate terminal, you can run (e.g) the web client
By default, all the values redirect to our publicly hosted production services.
After you are done with filling the values, restart museum and the App will start utilizing
those endpoints for everything instead of the Ente's prod instances.
```sh
cd ente/web
yarn install
NEXT_PUBLIC_ENTE_ENDPOINT=http://localhost:8080 yarn dev
```
Once you configure all the necessary endpoints, `cd` into `my-ente` and stop all the docker
containers with `docker compose down` to completely stop all the containers and restart them
with `docker compose up -d`.
> If you want to build the mobile apps from source, see the instructions
> [here](guides/mobile-build).
Similarly, you can read the default [`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
and build a functioning `museum.yaml` for many other functionalities like SMTP, Discord
Notifications, Hardcoded-OTT's, etc.
## Next steps
## Queries?
- More details about the server are in its
[README](https://github.com/ente-io/ente/tree/main/server#readme)
- More details about running the server (with or without Docker) are in
[RUNNING](https://github.com/ente-io/ente/blob/main/server/RUNNING.md)
- If you have questions around self-hosting that are not answered in any of the
existing documentation, you can ask in our
[GitHub Discussions](https://github.com/ente-io/ente/discussions). **Please
remember to search first if the query has been already asked and answered.**
## Contributing!
One particular way in which you can help is by adding new [guides](guides/) on
this help site. The documentation is written in Markdown and adding new pages is
[easy](https://github.com/ente-io/ente/tree/main/docs#readme). Editing existing
pages is even easier: at the bottom of each page is an _Edit this page_ link.
If you need any help or support, do not hesitate to drop your queries on our community
[discord channel](https://discord.gg/z2YVKkycX3) or create a
[Github Discussion](https://github.com/ente-io/ente/discussions) where 100s of self-hosters help each other :p.

62
docs/docs/self-hosting/troubleshooting/bucket-cors.md Normal file
View File

@@ -0,0 +1,62 @@
---
title: Bucket CORS
description: Troubleshooting CORS issues with S3 Buckets
---
# Fix potential CORS issues with your Buckets
## For AWS S3
If you cannot upload a photo due to a CORS issue, you need to fix the CORS
configuration of your bucket.
Create a `cors.json` file with the following content:
```json
{
"CORSRules": [
{
"AllowedOrigins": ["*"],
"AllowedHeaders": ["*"],
"AllowedMethods": ["GET", "HEAD", "POST", "PUT", "DELETE"],
"MaxAgeSeconds": 3000,
"ExposeHeaders": ["Etag"]
}
]
}
```
You may want to change the `AllowedOrigins` to a more restrictive value.
If you are using AWS for S3, you can execute the below command to get rid of
CORS. Make sure to enter the right path for the `cors.json` file.
```bash
aws s3api put-bucket-cors --bucket YOUR_S3_BUCKET --cors-configuration /path/to/cors.json
```
## For Self-hosted Minio Instance
> Important: MinIO does not take JSON CORS file as the input, instead you will
> have to build a CORS.xml file or just convert the above `cors.json` to XML.
A minor requirement here is the tool `mc` for managing buckets via command line
interface. Checkout the `mc set alias` document to configure alias for your
instance and bucket. After this you will be prompted for your AccessKey and
Secret, which is your username and password, go ahead and enter that.
```sh
mc cors set <your-minio>/<your-bucket-name /path/to/cors.xml
```
or, if you just want to just set the `AllowedOrigins` Header, you can use the
following command to do so.
```sh
mc admin config set <your-minio>/<your-bucket-name> api cors_allow_origin="*"
```
You can create also `.csv` file and dump the list of origins you would like to
allow and replace the `*` with `path` to the CSV file.
Now, uploads should be working fine.