diff --git a/server/RUNNING.md b/server/RUNNING.md index 444367ed7e..0d2b17c15c 100644 --- a/server/RUNNING.md +++ b/server/RUNNING.md @@ -10,18 +10,23 @@ static go binary. This document describes these approaches, and also outlines configuration. -- [Run using Docker using a pre-built Docker image](docs/docker.md) -- [Run using Docker but build an image from source](#build-and-run-using-docker) -- [Running without Docker](#running-without-docker) +- **[Run using pre-built Docker images](quickstart/README.md)** +- [Run using Docker, building image from source](#build-and-run-using-docker) +- [Run without Docker](#running-without-docker) +- [External DB or S3](#pre-built-images) - [Configuration](#configuration) If your mobile app is able to connect to your self hosted instance but is not able to view or upload images, see [help.ente.io/self-hosting/guides/configuring-s3](https://help.ente.io/self-hosting/guides/configuring-s3). +## Run using pre-built Docker images + +See [quickstart/README.md](quickstart/README.md). + ## Build and run using Docker -Start the cluster +Start the cluster (in the `ente/server` directory) docker compose up --build @@ -179,6 +184,17 @@ For running the tests, you can use the following command: ENV="test" go test -v ./pkg/... go clean -testcache && ENV="test" go test -v ./pkg/... ``` +## Pre-built images + +## server + +If you have setup the database and object storage externally and only want to +run Ente's server, you can just pull and run the image from +**`ghcr.io/ente-io/server`**. + +```sh +docker pull ghcr.io/ente-io/server +``` ## Configuration diff --git a/server/compose.yaml b/server/compose.yaml index 8c33de3845..27c1ebc9cf 100644 --- a/server/compose.yaml +++ b/server/compose.yaml @@ -18,11 +18,8 @@ services: - source: credentials_yaml target: /credentials.yaml volumes: - - custom-logs:/var/logs - ./museum.yaml:/museum.yaml:ro - ./data:/data:ro - networks: - - internal # Resolve "localhost:3200" in the museum container to the minio container. socat: @@ -40,22 +37,11 @@ services: POSTGRES_DB: ente_db # Wait for postgres to accept connections before starting museum. healthcheck: - test: - [ - "CMD", - "pg_isready", - "-q", - "-d", - "ente_db", - "-U", - "pguser" - ] + test: pg_isready -q -d ente_db -U pguser start_period: 40s start_interval: 1s volumes: - postgres-data:/var/lib/postgresql/data - networks: - - internal minio: image: minio/minio @@ -68,8 +54,6 @@ services: command: server /data --address ":3200" --console-address ":3201" volumes: - minio-data:/data - networks: - - internal post_start: - command: | sh -c ' @@ -121,9 +105,5 @@ configs: bucket: scw-eu-fr-v3 volumes: - custom-logs: postgres-data: minio-data: - -networks: - internal: diff --git a/server/docs/docker.md b/server/docs/docker.md index 7c85fa7f23..9be7aa304c 100644 --- a/server/docs/docker.md +++ b/server/docs/docker.md @@ -1,77 +1 @@ -# Running using published Docker images - -Here we describe a way to run an Ente instance using a starter Docker compose -file and using the pre-built Docker images that we publish. This method does not -require you to clone the repository or build any images. - -1. Create a directory where you'll run Ente - - ```sh - mkdir ente && cd ente - ``` - -2. Copy the starter compose.yaml and two of its support files from the - repository onto your directory. You can do it by hand, or use (e.g.) curl - - ```sh - # compose.yaml - curl -LO https://raw.githubusercontent.com/ente-io/ente/main/server/compose.yaml - ``` - -3. Modify `compose.yaml`. Instead of building from source, we want directly use - the published Docker image from `ghcr.io/ente-io/server` - - ```diff - --- a/server/compose.yaml - +++ b/server/compose.yaml - @@ -1,9 +1,6 @@ - services: - museum: - - build: - - context: . - - args: - - GIT_COMMIT: development-cluster - + image: ghcr.io/ente-io/server - ``` - -4. Create an (empty) configuration file. You can later put your custom - configuration in this if needed. - - ```sh - touch museum.yaml - ``` - -5. That is all. You can now start everything. - - ```sh - docker compose up - ``` - -This will start a cluster containing: - -- Ente's own server -- PostgresQL (DB) -- MinIO (the S3 layer) - -For each of these, it'll use the latest published Docker image. - -You can do a quick smoke test by pinging the API: - -```sh -curl localhost:8080/ping -``` - -## Only the server - -Alternatively, if you have setup the database and object storage externally and -only want to run Ente's server, you can skip the steps above and directly pull -and run the image from **`ghcr.io/ente-io/server`**. - -```sh -docker pull ghcr.io/ente-io/server -``` - -> [!TIP] -> -> For more documentation around self-hosting, see -> **[help.ente.io/self-hosting](https://help.ente.io/self-hosting)**. +Moved to [quickstart/README.md](../quickstart/README.md). diff --git a/server/quickstart/README.md b/server/quickstart/README.md new file mode 100644 index 0000000000..8ebe68a144 --- /dev/null +++ b/server/quickstart/README.md @@ -0,0 +1,107 @@ +A variant docker compose file that does not require cloning the repository, and +uses pre-built images instead. + +# WARNING: WIP - THIS DOESN'T WORK AS WRITTEN YET + +### Quickstart + +Copy paste the following command into your terminal + +```sh +sh -c "$(curl -fsSL https://raw.githubusercontent.com/ente-io/ente/HEAD/server/hello.sh)" +``` + +Alternatively, you can run the following four steps manually (that's all the +command above does for you): + +1. Create a directory on your system and switch to it. You can name it + anything, and it can be at any place on your machine. In particular, you + don't need to clone this repository. + + ```sh + mkdir my-ente && cd my-ente + ``` + + > [!TIP] + > + > "Ente" (pronounced _en-tay_) means "mine" in Malayalam, our Founder's mother + > tongue (the product literally thus means "My Photos"), so a directory named + > `my-ente` would translate to "my-my". + +2. Download the sample quickstart Docker compose file. + + ```sh + curl -fsSOL https://raw.githubusercontent.com/ente-io/ente/HEAD/server/quickstart/compose.yaml + ``` + +3. Create an empty `museum.yaml` (you can use it later to put your config). + + ```sh + touch museum.yaml + ``` + +4. Start your self hosted instance. + + ```sh + docker compose up + ``` + +That's it. You can now open **http://localhost:3000** in your browser to use +Ente's web app. + +## Details + +The quickstart steps above created a Docker compose cluster containing: + +- Ente's own server, museum +- Ente's web app +- Postgres (DB) +- Minio (S3 storage) + +For each of these, it'll use the latest published Docker image. + +You can do a quick smoke test by pinging the API: + +```sh +curl localhost:8080/ping +``` + +And start using the web app by opening http://localhost:3000 in your browser. + +The cluster will keep running as long as the `docker compose up` command (or the +`hello.sh` script you curl-ed) is running. If you want to keep it running in the +background, you can instead. + +```sh +cd /path/to/my-ente # Or whichever directory you created +docker compose up -d +``` + +And then later, to stop the cluster, you can + +```sh +cd /path/to/my-ente +docker compose down +``` + +### Caveat + +This sample setup is only intended to make it easy for people to get started. If +you're intending to use your self hosted instance for serious purposes, we +strongly recommend understanding all the moving parts. Some particular things to +call out: + +1. Change all the hardcoded credentials. +2. Consider if you want an external DB or an external S3 instead of the samples. +3. Keep a plaintext backup of your photos until you are proficient. + +### Next steps + +* Get a login [verification + code](https://help.ente.io/self-hosting/faq/otp#verification-code). + +* Connect to your self hosted instance [from your mobile + app](https://help.ente.io/self-hosting/guides/custom-server/). + +* Modify your setup to allow [uploading from your mobile + app](https://help.ente.io/self-hosting/guides/configuring-s3). diff --git a/server/quickstart/compose.yaml b/server/quickstart/compose.yaml new file mode 100644 index 0000000000..adcb23167c --- /dev/null +++ b/server/quickstart/compose.yaml @@ -0,0 +1,130 @@ +# Sample docker compose file, not meant for production use. + +services: + museum: + image: ghcr.io/ente-io/server + ports: + - 8080:8080 # API + depends_on: + postgres: + condition: service_healthy + environment: + ENTE_CREDENTIALS_FILE: /credentials.yaml + configs: + - source: credentials_yaml + target: /credentials.yaml + volumes: + - ./museum.yaml:/museum.yaml:ro + - ./data:/data:ro + + # Resolve "localhost:3200" in the museum container to the minio container. + socat-museum-3200: + image: alpine/socat + network_mode: service:museum + depends_on: [museum] + command: "TCP-LISTEN:3200,fork,reuseaddr TCP:minio:3200" + + web: + # image: ghcr.io/ente-io/web + image: web-test:latest # WIP + # Uncomment what you need to tweak. + ports: + - 3000:3000 # Photos web app + # - 3001:3001 # Accounts + - 3002:3002 # Public albums + # - 3003:3003 # Auth + # - 3004:3004 # Cast + # environment: + # ENTE_API_ORIGIN: http://localhost:8080 + # ENTE_ALBUMS_ORIGIN: https://localhost:3002 + + # Resolve "localhost:3200" in the web container to the minio container. + socat-web-3200: + image: alpine/socat + network_mode: service:web + depends_on: [web] + command: "TCP-LISTEN:3200,fork,reuseaddr TCP:minio:3200" + + # Resolve "localhost:8080" in the web container to the museum container. + socat-web-8080: + image: alpine/socat + network_mode: service:web + depends_on: [web] + command: "TCP-LISTEN:8080,fork,reuseaddr TCP:minio:8080" + + postgres: + image: postgres:15 + environment: + POSTGRES_USER: pguser + POSTGRES_PASSWORD: pgpass + POSTGRES_DB: ente_db + healthcheck: + test: pg_isready -q -d ente_db -U pguser + start_period: 40s + start_interval: 1s + volumes: + - postgres-data:/var/lib/postgresql/data + + minio: + image: minio/minio + ports: + - 3200:3200 # MinIO API + environment: + MINIO_ROOT_USER: changeme + MINIO_ROOT_PASSWORD: changeme1234 + command: server /data --address ":3200" --console-address ":3201" + volumes: + - minio-data:/data + post_start: + - command: | + sh -c ' + #!/bin/sh + + while ! mc config host add h0 http://minio:3200 changeme changeme1234 2>/dev/null + do + echo "Waiting for minio..." + sleep 0.5 + done + + cd /data + + mc mb -p b2-eu-cen + mc mb -p wasabi-eu-central-2-v3 + mc mb -p scw-eu-fr-v3 + ' + +configs: + credentials_yaml: + content: | + db: + host: postgres + port: 5432 + name: ente_db + user: pguser + password: pgpass + + s3: + are_local_buckets: true + b2-eu-cen: + key: changeme + secret: changeme1234 + endpoint: localhost:3200 + region: eu-central-2 + bucket: b2-eu-cen + wasabi-eu-central-2-v3: + key: changeme + secret: changeme1234 + endpoint: localhost:3200 + region: eu-central-2 + bucket: wasabi-eu-central-2-v3 + compliance: false + scw-eu-fr-v3: + key: changeme + secret: changeme1234 + endpoint: localhost:3200 + region: eu-central-2 + bucket: scw-eu-fr-v3 + +volumes: + postgres-data: + minio-data: diff --git a/server/scripts/compose/README.md b/server/scripts/compose/README.md deleted file mode 100644 index 688e306a7c..0000000000 --- a/server/scripts/compose/README.md +++ /dev/null @@ -1,22 +0,0 @@ -A variant docker compose file that does not require cloning the repository, and -instead uses pre-built images. - -**TODO: Not done yet, a future standalone compose file will live here.** - -### Details - -This folder contains a `compose.yaml` file that is a variant of the top level -`server/compose.yaml`. The difference between the two are: - -- `server/compose.yaml` builds Ente images from source, assuming that the - `ente-io/ente` repository has been checked out. - -- `server/scripts/compose/compose.yaml` (the `compose.yaml` in this directory) - uses the pre-build Ente images, and does not require the `ente-io/ente` - repository to be cloned. That is, it can be run standalone by just curl-ing - it, and a few required files, to any folder on your machine. - -For more details about how to use it, see [docker.md](../../docs/docker.md). - -This folder also contains a credentials file that is required by (both) these -docker compose files.