updates...
This commit is contained in:
Amar Saljic
@ -1,9 +1,22 @@
|
||||
# NAxS Homelab
|
||||
Prerequisites:
|
||||
- Create a default network called homelab
|
||||
## Prerequisites
|
||||
0. Make sure that your user is part of the docker group
|
||||
- `cat /etc/group | grep docker` - if the entry looks like `docker:x:<Group ID>:<username>`, you're good to go
|
||||
- Otherwise please run `sudo usermod -aG docker <username>`, followed by logging out & in again for these changes to take into effect
|
||||
|
||||
1. Create a default network called homelab
|
||||
```
|
||||
docker network create homelab
|
||||
```
|
||||
2. Set up 1Password for access to secrets
|
||||
- Install `pass` & `gpg`
|
||||
- Generate key with `gpg --full-generate-key`
|
||||
- stick to defaults
|
||||
- as password, use `GPG cert password` stored inside the `NAxS Homelab` vault in 1Password
|
||||
- Initialize password storage with `pass init "GPG key ID"`
|
||||
- You can check out the ID by using `gpg --list-secret-keys --keyid-format LONG` - you should see a line with `sec`, containing the following information `<encryption technology>/ID`
|
||||
- Store the 1Password service account token in `pass` as `op-sa_token` by executing `pass insert op-sa_token`
|
||||
- Make sure your .zshrc file loads the token into the `OP_SERVICE_ACCOUNT_TOKEN` (this is needed by the 1Password CLI for authentication purposes when loading the secrets) environment variable by executing `export OP_SERVICE_ACCOUNT_TOKEN="$(pass op-sa_token)"`
|
||||
|
||||
// TODO: Create template script
|
||||
Template script which helps with setting up new applications (asks for potential secrets needs, adds default network to compose file, creates new users/groups to run containers rootless)
|
||||
@ -0,0 +1,180 @@
|
||||
#!/bin/zsh
|
||||
while [[ "$#" -gt 0 ]]
|
||||
do
|
||||
case $1 in
|
||||
--app_name) app_name="$2"
|
||||
shift;;
|
||||
*) echo "Unknown parameter passed: $1"
|
||||
exit 1;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
# Make sure app_name exists and is not empty
|
||||
if [[ -z "$app_name" ]]; then
|
||||
echo "app_name is unset or empty."
|
||||
exit 1;
|
||||
fi
|
||||
|
||||
# Define the base directory for the new application structure
|
||||
# This will be one level up from where this script is executed.
|
||||
app_base_dir="../${app_name}"
|
||||
app_data_dir="${app_base_dir}/data"
|
||||
compose_file="${app_base_dir}/compose.yaml"
|
||||
readme_file="${app_base_dir}/README.md"
|
||||
start_script="${app_base_dir}/start.sh"
|
||||
|
||||
echo "--- Setting up files and directories for application: ${app_name} ---"
|
||||
echo "Base directory: ${app_base_dir}"
|
||||
|
||||
# 1. Create the application base directory and data directory
|
||||
echo "Creating directory: ${app_data_dir}"
|
||||
mkdir -p "${app_data_dir}" || { echo "Error: Failed to create directory ${app_data_dir}"; exit 1; }
|
||||
|
||||
# 2. Create compose.yaml and README.md
|
||||
echo "Creating file: ${compose_file}"
|
||||
touch "${compose_file}" || { echo "Error: Failed to create file ${compose_file}"; exit 1; }
|
||||
|
||||
echo "Creating file: ${readme_file}"
|
||||
touch "${readme_file}" || { echo "Error: Failed to create file ${readme_file}"; exit 1; }
|
||||
|
||||
# 3. Prepopulate README.md with title case app name
|
||||
# Convert app_name to title case (first letter of each word capitalized)
|
||||
# This is a simple conversion; for more robust title casing, a function would be better.
|
||||
app_name_title_case=$(echo "$app_name" | awk '{for(i=1;i<=NF;i++){ $i=toupper(substr($i,1,1)) tolower(substr($i,2)) }}1')
|
||||
|
||||
echo "Prepopulating ${readme_file}..."
|
||||
cat > "${readme_file}" << EOF
|
||||
# ${app_name_title_case}
|
||||
|
||||
This is the README file for the **${app_name}** application.
|
||||
|
||||
## Overview
|
||||
Provide a brief description of your application here.
|
||||
|
||||
## Setup
|
||||
Instructions for setting up the application.
|
||||
|
||||
EOF
|
||||
|
||||
# 4. Ask the user if secrets will be required
|
||||
read "requires_secrets?Will this application require secrets (yes/no)? "
|
||||
requires_secrets=$(echo "$requires_secrets" | tr '[:upper:]' '[:lower:]') # Convert to lowercase
|
||||
|
||||
# 5. Prepopulate compose.yaml based on user selection re. secrets
|
||||
echo "Prepopulating ${compose_file} based on secrets requirement..."
|
||||
if [[ "$requires_secrets" == "yes" || "$requires_secrets" == "y" ]]; then
|
||||
cat > "${compose_file}" << EOF
|
||||
version: '3.8'
|
||||
services:
|
||||
${app_name}:
|
||||
image: your-app-image:latest
|
||||
container_name: ${app_name}-container
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
environment:
|
||||
# Example of how to pass secrets as environment variables
|
||||
# These would typically be loaded from a .env file or a secrets management system
|
||||
- DATABASE_URL=\${DATABASE_URL}
|
||||
- API_KEY=\${API_KEY}
|
||||
# Example of secrets usage with Docker Compose secrets (requires Docker Swarm or specific setup)
|
||||
# secrets:
|
||||
# - my_database_password
|
||||
# - my_api_key
|
||||
# deploy:
|
||||
# resources:
|
||||
# limits:
|
||||
# cpus: '0.5'
|
||||
# memory: 512M
|
||||
secrets:
|
||||
my_database_password:
|
||||
external: true # Assumes secret is created externally (e.g., docker secret create)
|
||||
my_api_key:
|
||||
external: true
|
||||
EOF
|
||||
else
|
||||
cat > "${compose_file}" << EOF
|
||||
services:
|
||||
${app_name}:
|
||||
image: your-app-image:latest
|
||||
container_name: ${app_name}-container
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
# No secrets configuration needed for this version
|
||||
# deploy:
|
||||
# resources:
|
||||
# limits:
|
||||
# cpus: '0.5'
|
||||
# memory: 512M
|
||||
EOF
|
||||
fi
|
||||
|
||||
# 6. If secrets will be used - touch ../"app_name"/start.sh and prepopulate it
|
||||
if [[ "$requires_secrets" == "yes" || "$requires_secrets" == "y" ]]; then
|
||||
echo "Creating and prepopulating ${start_script}..."
|
||||
cat > "${start_script}" << EOF
|
||||
#!/bin/zsh
|
||||
|
||||
# Example start script for an application using secrets
|
||||
# This script assumes you have a .env file or similar mechanism
|
||||
# for loading secrets into the environment before starting Docker Compose.
|
||||
|
||||
# --- IMPORTANT: Replace with your actual secret loading mechanism ---
|
||||
# Example 1: Load from a .env file
|
||||
# if [[ -f ".env" ]]; then
|
||||
# echo "Loading environment variables from .env"
|
||||
# source .env
|
||||
# else
|
||||
# echo "Warning: .env file not found. Secrets might be missing."
|
||||
# fi
|
||||
|
||||
# Example 2: Using a secrets management tool (e.g., HashiCorp Vault, AWS Secrets Manager)
|
||||
# echo "Fetching secrets from Vault..."
|
||||
# export DATABASE_URL=\$(vault read -field=value secret/myapp/database_url)
|
||||
# export API_KEY=\$(vault read -field=value secret/myapp/api_key)
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
echo "Starting Docker Compose services for ${app_name}..."
|
||||
docker compose -f "${compose_file}" up -d
|
||||
|
||||
echo "Application ${app_name} started. Check logs with: docker compose logs -f ${app_name}"
|
||||
EOF
|
||||
chmod +x "${start_script}" # Make the start script executable
|
||||
fi
|
||||
|
||||
# 7. If secrets will be used, append another chapter at the end of the README.md file
|
||||
if [[ "$requires_secrets" == "yes" || "$requires_secrets" == "y" ]]; then
|
||||
echo "Appending 'Secrets Management' chapter to ${readme_file}..."
|
||||
cat >> "${readme_file}" << EOF
|
||||
|
||||
## Secrets Management
|
||||
|
||||
This application utilizes secrets for sensitive configurations (e.g., database credentials, API keys).
|
||||
|
||||
### Configuration
|
||||
Secrets are expected to be provided via environment variables or Docker Compose secrets.
|
||||
Refer to the \`start.sh\` script for an example of how to load these secrets before application startup.
|
||||
|
||||
**Example Environment Variables (to be set in your environment or a \`.env\` file):**
|
||||
\`\`\`
|
||||
DATABASE_URL="postgres://user:password@host:port/database"
|
||||
API_KEY="your_super_secret_api_key"
|
||||
\`\`\`
|
||||
|
||||
**For Docker Compose secrets:**
|
||||
Ensure the necessary Docker secrets are created on your Docker host or Swarm cluster.
|
||||
\`\`\`bash
|
||||
docker secret create my_database_password <(echo "your_db_password")
|
||||
docker secret create my_api_key <(echo "your_api_key")
|
||||
\`\`\`
|
||||
Then, update the \`compose.yaml\` to reference these secrets.
|
||||
|
||||
EOF
|
||||
fi
|
||||
|
||||
echo "--- Setup complete for ${app_name} ---"
|
||||
echo "Check the '${app_base_dir}' directory for your new files."
|
||||
80
homelab/applications/app-template-generator/create-user.sh
Normal file
80
homelab/applications/app-template-generator/create-user.sh
Normal file
@ -0,0 +1,80 @@
|
||||
#!/bin/zsh
|
||||
while [[ "$#" -gt 0 ]]
|
||||
do
|
||||
case $1 in
|
||||
--app_name) app_name="$2"
|
||||
shift;;
|
||||
--id) desired_id="$2"
|
||||
shift;;
|
||||
*) echo "Unknown parameter passed: $1"
|
||||
exit 1;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
# Validate desired_id is a number
|
||||
if ! [[ "$desired_id" =~ ^[0-9]+$ ]]; then
|
||||
echo "Error: Invalid UID/GID. Please enter a numeric value."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
user_name="${app_name}-user"
|
||||
group_name="${app_name}-group"
|
||||
|
||||
echo "--- Checking/Creating User and Group for ${app_name} ---"
|
||||
|
||||
# --- Handle Group ---
|
||||
echo "Checking group: ${group_name}"
|
||||
existing_gid=$(getent group "${group_name}" | cut -d: -f3)
|
||||
|
||||
if [[ -n "$existing_gid" ]]; then
|
||||
if [[ "$existing_gid" -eq "$desired_id" ]]; then
|
||||
echo "Group '${group_name}' already exists with the correct GID (${desired_id})."
|
||||
else
|
||||
echo "Group '${group_name}' exists with GID ${existing_gid}, but desired GID is ${desired_id}."
|
||||
echo "Attempting to modify group GID..."
|
||||
if sudo groupmod -g "$desired_id" "${group_name}"; then
|
||||
echo "Successfully adjusted group '${group_name}' to GID ${desired_id}."
|
||||
else
|
||||
echo "Failed to adjust group '${group_name}' GID. Please check permissions or try manually."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "Group '${group_name}' does not exist. Creating..."
|
||||
if sudo groupadd -g "$desired_id" "${group_name}"; then
|
||||
echo "Successfully created group '${group_name}' with GID ${desired_id}."
|
||||
else
|
||||
echo "Failed to create group '${group_name}'. Please check permissions or try manually."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Handle User ---
|
||||
echo "Checking user: ${user_name}"
|
||||
existing_uid=$(getent passwd "${user_name}" | cut -d: -f3)
|
||||
|
||||
if [[ -n "$existing_uid" ]]; then
|
||||
if [[ "$existing_uid" -eq "$desired_id" ]]; then
|
||||
echo "User '${user_name}' already exists with the correct UID (${desired_id})."
|
||||
else
|
||||
echo "User '${user_name}' exists with UID ${existing_uid}, but desired UID is ${desired_id}."
|
||||
echo "Attempting to modify user UID..."
|
||||
if sudo usermod -u "$desired_id" -g "$desired_id" "${user_name}"; then
|
||||
echo "Successfully adjusted user '${user_name}' to UID ${desired_id} and primary GID ${desired_id}."
|
||||
else
|
||||
echo "Failed to adjust user '${user_name}' UID/GID. Please check permissions or try manually."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "User '${user_name}' does not exist. Creating..."
|
||||
if sudo useradd -u "$desired_id" -g "$desired_id" -s /sbin/nologin -c "Application User for ${app_name}" "${user_name}"; then
|
||||
echo "Successfully created user '${user_name}' with UID ${desired_id} and primary GID ${desired_id}."
|
||||
else
|
||||
echo "Failed to create user '${user_name}'. Please check permissions or try manually."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "--- Operation complete for ${app_name} ---"
|
||||
8
homelab/applications/app-template-generator/main.sh
Normal file
8
homelab/applications/app-template-generator/main.sh
Normal file
@ -0,0 +1,8 @@
|
||||
#!/bin/zsh
|
||||
# Ask for the application name
|
||||
read "app_name?Enter the name of the application: "
|
||||
# Ask for the desired UID/GID
|
||||
read "desired_id?Enter the desired UID/GID for the application (e.g., 1001): "
|
||||
|
||||
./create-user.sh --app_name ${app_name} --id ${desired_id}
|
||||
./create-files+directories.sh --app_name ${app_name}
|
||||
@ -25,13 +25,4 @@ In case user/group doesn't exist at all, please create the user & group incl. th
|
||||
```
|
||||
sudo groupadd -g 1003 gitea-group
|
||||
sudo useradd -g gitea-group -u 1003 gitea-user
|
||||
```
|
||||
|
||||
## About secrets
|
||||
In order to manage secrets centrally in 1Password and due to the need for secrets in Gitea, using `docker compose` directly in the terminal does not work.
|
||||
|
||||
## Bring up/tear down container
|
||||
Please use the `start.sh` to spin up the container
|
||||
### Prerequisites start.sh
|
||||
- User executing the script is part of the `docker` group
|
||||
- Env variable `OP_SERVICE_ACCOUNT_TOKEN` is set up \[check out top-level README.md for more information on how to set this up\]
|
||||
```
|
||||
@ -1,6 +1,3 @@
|
||||
secrets:
|
||||
gitea_postgres_password:
|
||||
environment: GITEA_POSTGRES_PASSWORD
|
||||
services:
|
||||
gitea:
|
||||
image: docker.gitea.com/gitea:1-rootless
|
||||
@ -14,11 +11,9 @@ services:
|
||||
GITEA__database__HOST: postgres:5432
|
||||
GITEA__database__NAME: gitea
|
||||
GITEA__database__USER: gitea
|
||||
GITEA__database__PASSWD_FILE: /run/secrets/gitea_postgres_password
|
||||
restart: always
|
||||
volumes: ['./data:/var/lib/gitea', './config:/etc/gitea']
|
||||
ports: ['8030:3000', '2222:2222']
|
||||
secrets: ['gitea_postgres_password']
|
||||
networks: ['homelab']
|
||||
|
||||
networks:
|
||||
|
||||
@ -1,12 +0,0 @@
|
||||
#!/bin/zsh
|
||||
# Exit immediately if a command exits with a non-zero status.
|
||||
set -e
|
||||
|
||||
echo "--- Starting Docker Secret Management ---"
|
||||
# Mount secrets
|
||||
export GITEA_POSTGRES_PASSWORD="$(op read 'op://NAxS Homelab/Gitea Postgres credentials/password')"
|
||||
|
||||
# Bring up container
|
||||
docker compose up -d
|
||||
|
||||
echo "--- Docker Secret Management Complete ---"
|
||||
@ -2,20 +2,26 @@
|
||||
## Set up non-root user for container
|
||||
We are providing a non-root user to the container to limit the attack surface for privilege escalations. In order for this to work in our setup, please make sure to check if you have a user called `postgres` set up.
|
||||
|
||||
1. Check if user `postgres` exists and if the UID is 1002
|
||||
1. Check if user `postgres` exists
|
||||
|
||||
```
|
||||
cat /etc/passwd | grep postgres
|
||||
```
|
||||
|
||||
In case the `postgres` user exists but the UID is not 1002, please adjust it via
|
||||
In case the `postgres` user doesn't exist, please create the user by running
|
||||
```
|
||||
sudo usermod -u 1002 postgres
|
||||
sudo useradd postgres
|
||||
```
|
||||
2. `data` folder ownership
|
||||
Also you need to make sure that the `postgres` owner owns the volumes mounted for docker
|
||||
```
|
||||
sudo chown -R postgres:postgres data
|
||||
sudo chmod 770 data
|
||||
```
|
||||
|
||||
In case the `postgres` user doesn't exist at all, please create the user incl. the right UID by running
|
||||
3. Adjust compose.yml
|
||||
Within `services > postgres > user`, make sure to replare `postgres` with the UID of the user on your machine
|
||||
```
|
||||
sudo useradd -u 1002 postgres
|
||||
cat /etc/passwd | grep postgres
|
||||
```
|
||||
|
||||
## About secrets
|
||||
@ -25,4 +31,4 @@ In order to manage secrets centrally in 1Password and due to the need for secret
|
||||
Please use the `start.sh` to spin up the container
|
||||
### Prerequisites start.sh
|
||||
- User executing the script is part of the `docker` group
|
||||
- Env variable `OP_SERVICE_ACCOUNT_TOKEN` is set up \[check out top-level README.md for more information on how to set this up\]
|
||||
- Environment variable `OP_SERVICE_ACCOUNT_TOKEN` is set up \[check out top-level README.md for more information on how to set this up\]
|
||||
@ -101,6 +101,4 @@ op --version
|
||||
apt install pass gnupg2
|
||||
|
||||
|
||||
|
||||
|
||||
echo "Done. Recommended: log out and back in (or reboot) to start using zsh and ensure all services are active."
|
||||
Reference in New Issue
Block a user