feat(scripts): add ip-check, project-inventory and vault-migration scripts #11
4 changed files with 787 additions and 9 deletions
|
|
@ -6,12 +6,37 @@ Helper scripts for working with STACKIT services.
|
|||
|
||||
## Overview
|
||||
|
||||
| Script | Purpose | Required tools |
|
||||
| ---------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | ---------------------------------- |
|
||||
| [`create-kubeconfig-multiple-projects.sh`](#create-kubeconfig-multiple-projectssh) | Generate kubeconfig entries for every SKE cluster across one or more STACKIT projects. | `stackit`, `yq` |
|
||||
| [`delete-unused-volumes.sh`](#delete-unused-volumessh) | Delete all STACKIT volumes whose status is `AVAILABLE` (i.e. not attached). | `stackit`, `yq` |
|
||||
| [`ske-show-versions.sh`](#ske-show-versionssh) | Print overview of SKE cluster Kubernetes versions and nodepool image versions, marking deprecated versions. | `stackit` (>= 0.59.0), `jq`, `awk` |
|
||||
| [`smctl.sh`](#smctlsh) | Unified CLI wrapper around HashiCorp Vault for the STACKIT Secret Manager (KV v2). | `vault`, `jq` |
|
||||
| Script | Purpose | Required tools |
|
||||
| ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ---------------------------------- |
|
||||
| [`check-stackit-ip.sh`](#check-stackit-ipsh) | Check whether a given IP address belongs to STACKIT's public IP ranges. | `stackit`, `jq`, `grepcidr` |
|
||||
| [`create-kubeconfig-multiple-projects.sh`](#create-kubeconfig-multiple-projectssh) | Generate kubeconfig entries for every SKE cluster across one or more STACKIT projects. | `stackit`, `yq` |
|
||||
| [`delete-unused-volumes.sh`](#delete-unused-volumessh) | Delete all STACKIT volumes whose status is `AVAILABLE` (i.e. not attached). | `stackit`, `yq` |
|
||||
| [`list-project-resources.sh`](#list-project-resourcessh) | Render a Markdown inventory of resources (DNS, SKE, databases, storage, …) for one or more STACKIT projects. | `stackit`, `jq` |
|
||||
| [`ske-show-versions.sh`](#ske-show-versionssh) | Print overview of SKE cluster Kubernetes versions and nodepool image versions, marking deprecated versions. | `stackit` (>= 0.59.0), `jq`, `awk` |
|
||||
| [`smctl.sh`](#smctlsh) | Unified CLI wrapper around HashiCorp Vault for the STACKIT Secret Manager (KV v2), think `kubectl` for secrets | `vault`, `jq` |
|
||||
| [`vault-migrate.sh`](#vault-migratesh) | Migrate secrets between two Vault instances using the KV v2 API (supports userpass and LDAP for source). | `vault`, `jq` |
|
||||
|
||||
---
|
||||
|
||||
## `check-stackit-ip.sh`
|
||||
|
||||
Looks up the STACKIT public IP ranges (via `stackit curl https://iaas.api.eu01.stackit.cloud/v1/networks/public-ip-ranges`) and tells you whether a given IP falls inside any of them. Useful to verify whether a public-facing address actually originates from STACKIT.
|
||||
|
||||
Exits `0` if the IP is found, `1` otherwise.
|
||||
|
||||
### Example
|
||||
|
||||
```bash
|
||||
# Check a single IP
|
||||
./check-stackit-ip.sh 45.129.40.1
|
||||
```
|
||||
|
||||
Sample output:
|
||||
|
||||
```
|
||||
Fetching STACKIT IP ranges...
|
||||
Found: 45.129.40.1 is in the range 45.129.40.0/22
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -78,8 +103,8 @@ Sample output:
|
|||
```
|
||||
CLUSTER NAME SKE VERSION NODEPOOL FLATCAR VERSION MACHINE TYPE PROJECT ID
|
||||
------------ ----------- -------- --------------- ------------ ----------
|
||||
prod-cluster 1.30.4 (supported) default 4081.2.0 (supported) c1.4 90bc91eb-...
|
||||
legacy-cluster 1.27.9 (exp. 2026-05-01) default 3815.2.5 (exp. 2026-03-15) c1.2 be20ca97-...
|
||||
prod-cluster 1.30.4 (supported) default 4081.2.0 (supported) c1.4 xxxxxxxx-...
|
||||
legacy-cluster 1.27.9 (exp. 2026-05-01) default 3815.2.5 (exp. 2026-03-15) c1.2 yyyyyyyy-...
|
||||
|
||||
Summary:
|
||||
Total clusters: 2
|
||||
|
|
@ -89,7 +114,7 @@ Total clusters: 2
|
|||
|
||||
## `smctl.sh`
|
||||
|
||||
Wrapper around the `vault` CLI that targets the STACKIT Secret Manager endpoint (`https://prod.sm.eu01.stackit.cloud`) and authenticates with userpass.
|
||||
Wrapper around the `vault` CLI to works with secrets in STACKIT Secrets Manager, think about it as `kubectl` but for secrets.
|
||||
|
||||
### Required environment variables
|
||||
|
||||
|
|
@ -139,3 +164,97 @@ eval "$(./smctl.sh get postgresql all-export)"
|
|||
# Write a value from a file (e.g. a full .env)
|
||||
./smctl.sh put terraform secret-env < .env
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `list-project-resources.sh`
|
||||
|
||||
Iterates over the listable STACKIT services (DNS, Git, Load Balancers, SKE etc.) and prints a Markdown report containing one section per project with a table per resource type. Intended to be redirected into a `.md` file.
|
||||
|
||||
Project IDs are passed as arguments — at least one is required.
|
||||
|
||||
### Example
|
||||
|
||||
```bash
|
||||
# Single project
|
||||
./list-project-resources.sh xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > project-inventory.md
|
||||
|
||||
# Multiple projects in one report
|
||||
./list-project-resources.sh \
|
||||
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
|
||||
yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy \
|
||||
> inventory.md
|
||||
```
|
||||
|
||||
Sample output (excerpt):
|
||||
|
||||
```markdown
|
||||
## Project: my-prod-project (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
|
||||
|
||||
**SKE Clusters:**
|
||||
|name|status|...|
|
||||
|---|---|---|
|
||||
|prod-cluster|HEALTHY|...|
|
||||
|
||||
**Public IPs:**
|
||||
N/A
|
||||
|
||||
last update: Thu, 16-Apr-2026 14:42:11 CEST
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `vault-migrate.sh`
|
||||
|
||||
Migrates secrets between two KV v2-compatible secret backends, typically from a HashiCorp Vault instance into STACKIT Secrets Manager, or from one STACKIT Secrets Manager instance to another (both expose the Vault KV v2 API). Source authentication can be `userpass` or `ldap` (LDAP triggers an interactive password prompt); the target always uses `userpass`. Recursively walks all paths under the source mount unless `MIGRATE_PATHS` is set.
|
||||
|
||||
### Flags
|
||||
|
||||
- `-d, --dry-run` — show what would be migrated without writing.
|
||||
- `-v, --verbose` — verbose progress output.
|
||||
- `-s, --skip-existing` — skip secrets that already exist in the target.
|
||||
- `--debug` — debug output (implies `--verbose`).
|
||||
- `-h, --help` — show usage.
|
||||
|
||||
### Required environment variables
|
||||
|
||||
| Variable | Description |
|
||||
| ---------------------- | ----------------------------------------------------------------- |
|
||||
| `SOURCE_VAULT_ADDR` | Source Vault address (e.g. `https://vault.example.com`). |
|
||||
| `SOURCE_SM_ID` | Source KV mount path. |
|
||||
| `SOURCE_AUTH_METHOD` | `userpass` (default) or `ldap`. |
|
||||
| `SOURCE_SM_USERNAME` | Source username — required when `SOURCE_AUTH_METHOD=userpass`. |
|
||||
| `SOURCE_SM_PASSWORD` | Source password — required when `SOURCE_AUTH_METHOD=userpass`. |
|
||||
| `SOURCE_LDAP_USERNAME` | LDAP username — required when `SOURCE_AUTH_METHOD=ldap`. |
|
||||
| `TARGET_VAULT_ADDR` | Target Vault address. |
|
||||
| `TARGET_SM_USERNAME` | Target Vault username (always userpass). |
|
||||
| `TARGET_SM_PASSWORD` | Target Vault password. |
|
||||
| `TARGET_SM_ID` | Target KV mount path. |
|
||||
| `MIGRATE_PATHS` | Optional space-separated list of paths to migrate (default: all). |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Common target setup
|
||||
export TARGET_VAULT_ADDR="https://prod.sm.eu01.stackit.cloud"
|
||||
export TARGET_SM_USERNAME="target-user"
|
||||
export TARGET_SM_PASSWORD='<your-target-password>'
|
||||
export TARGET_SM_ID="sm-target-xxxx"
|
||||
|
||||
# 1) Dry run — migrate everything from a userpass source
|
||||
export SOURCE_VAULT_ADDR="https://old.vault.example.com"
|
||||
export SOURCE_SM_ID="sm-source-xxxx"
|
||||
export SOURCE_AUTH_METHOD="userpass"
|
||||
export SOURCE_SM_USERNAME="source-user"
|
||||
export SOURCE_SM_PASSWORD='<your-source-password>'
|
||||
./vault-migrate.sh --dry-run --verbose
|
||||
|
||||
# 2) Migrate from an LDAP source (will prompt for password)
|
||||
export SOURCE_AUTH_METHOD="ldap"
|
||||
export SOURCE_LDAP_USERNAME="kopps"
|
||||
./vault-migrate.sh
|
||||
|
||||
# 3) Migrate only selected paths, skipping ones that already exist
|
||||
export MIGRATE_PATHS="postgresql redis terraform"
|
||||
./vault-migrate.sh --skip-existing --verbose
|
||||
```
|
||||
|
|
|
|||
59
scripts/check-stackit-ip.sh
Executable file
59
scripts/check-stackit-ip.sh
Executable file
|
|
@ -0,0 +1,59 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
# check-ip: Checks if a given IP address is in the STACKIT public IP ranges.
|
||||
# Usage: ./check-ip <ip-address>
|
||||
# Example: ./check-ip 45.129.40.1
|
||||
|
||||
# Check if an IP address was provided as an argument
|
||||
if [ -z "$1" ]; then
|
||||
echo "Usage: $0 <ip-address-to-check>"
|
||||
echo "Example: $0 45.129.40.1"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
IP_TO_CHECK="$1"
|
||||
|
||||
# Check for required commands.
|
||||
# We need 'stackit', 'jq', and 'grepcidr'.
|
||||
for cmd in stackit jq grepcidr; do
|
||||
if ! command -v $cmd &> /dev/null; then
|
||||
echo "Error: Required command '$cmd' is not installed." >&2
|
||||
echo "Please install it and ensure it's in your PATH." >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Fetching STACKIT IP ranges..." >&2
|
||||
|
||||
FOUND=0
|
||||
RANGES_CHECKED=0
|
||||
|
||||
while read -r RANGE; do
|
||||
# Skip empty lines, just in case
|
||||
if [ -z "$RANGE" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
RANGES_CHECKED=$((RANGES_CHECKED + 1))
|
||||
|
||||
if echo "$IP_TO_CHECK" | grepcidr -s "$RANGE"; then
|
||||
echo "Found: $IP_TO_CHECK is in the range $RANGE"
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
done < <(stackit curl "https://iaas.api.eu01.stackit.cloud/v1/networks/public-ip-ranges" | jq -r '.items[].cidr')
|
||||
|
||||
# Check if we processed any ranges at all
|
||||
if [ $RANGES_CHECKED -eq 0 ]; then
|
||||
echo "Error: Failed to fetch or parse IP ranges from STACKIT API." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check the flag after the loop
|
||||
if [ $FOUND -eq 0 ]; then
|
||||
# If the loop finished without finding anything, print a "not found" message
|
||||
echo "Not found: $IP_TO_CHECK is not in any of the STACKIT ranges."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exit 0
|
||||
133
scripts/list-project-resources.sh
Executable file
133
scripts/list-project-resources.sh
Executable file
|
|
@ -0,0 +1,133 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Script to list (all) resources of a STACKIT project
|
||||
# Ideally, you redirect the output into a markdown file.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Check if stackit cli is installed
|
||||
if ! command -v stackit &> /dev/null; then
|
||||
echo "Error: stackit command not found"
|
||||
echo "Please install STACKIT CLI from:"
|
||||
echo "https://github.com/stackitcloud/stackit-cli/blob/main/INSTALLATION.md"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if stackit is properly authenticated; only login if the session is gone
|
||||
if ! stackit auth get-access-token &> /dev/null; then
|
||||
echo "Session expired. Logging in..."
|
||||
stackit auth login || { echo "Error: stackit authentication failed."; exit 1; }
|
||||
fi
|
||||
|
||||
# Check if jq is installed
|
||||
if ! command -v jq &> /dev/null; then
|
||||
echo "Error: 'jq' is not installed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Function to display usage and handle errors
|
||||
display_usage() {
|
||||
echo "Usage: $0 <project-id-1> <project-id-2> ..."
|
||||
echo "Example: $0 xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy >output.md"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Function to get project information
|
||||
get_project_name() {
|
||||
local project_id=$1
|
||||
|
||||
local project_name=""
|
||||
project_name=$(stackit project describe --project-id "$project_id" --output-format json 2>/dev/null \
|
||||
| jq -r '.name // empty')
|
||||
# If the CLI call fails or .name is missing, fall back to the raw ID
|
||||
[[ -z "$project_name" ]] && project_name="$project_id"
|
||||
echo "## Project: $project_name ($project_id)"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# Function to get resource information
|
||||
fetch_resources() {
|
||||
local service=$1
|
||||
local type=$2
|
||||
local label=$3
|
||||
local fields=${4:-} # comma-separated list of fields to keep; empty = all
|
||||
|
||||
# Build the CLI command - omit the type if it's empty
|
||||
if [[ -z "$type" ]]; then
|
||||
json_output=$(stackit "$service" list --project-id "$project_id" --output-format json)
|
||||
else
|
||||
json_output=$(stackit "$service" "$type" list --project-id "$project_id" --output-format json)
|
||||
fi
|
||||
|
||||
local header
|
||||
local separation_line
|
||||
local data
|
||||
|
||||
# Check whether json output is empty/null - indicating no resource of that service type found - and print it
|
||||
if [[ -z "$json_output" ]] || [[ "$json_output" == "[]" ]] || [[ "$json_output" == "null" ]]; then
|
||||
echo "**$label:**"
|
||||
echo "N/A"
|
||||
echo -e "\n"
|
||||
return
|
||||
fi
|
||||
|
||||
# Narrow output to only the requested fields (preserving their order)
|
||||
if [[ -n "$fields" ]]; then
|
||||
json_output=$(jq --arg fields "$fields" '
|
||||
($fields | split(",")) as $keep |
|
||||
map(. as $item | reduce $keep[] as $k ({}; . + {($k): $item[$k]}))
|
||||
' <<< "$json_output")
|
||||
fi
|
||||
|
||||
# Format the json output into a markdown table
|
||||
header=$(jq -r '.[0] | to_entries | map(.key) | join("\t")' <<< "$json_output" | sed 's/\t/|/g; s/^/|/; s/$/|/')
|
||||
separation_line=$(jq -r '.[0] | to_entries | map("|---") | join("")' <<< "$json_output")
|
||||
data=$(jq -r '.[] | to_entries | map(.value|tostring) | join("\t")' <<< "$json_output" | sed 's/\t/|/g; s/^/|/; s/$/|/')
|
||||
|
||||
echo "**$label:**"
|
||||
echo "$header"
|
||||
echo "$separation_line"
|
||||
echo "$data"
|
||||
echo -e "\n"
|
||||
}
|
||||
|
||||
get_project_resources() {
|
||||
|
||||
# call function to get project name
|
||||
get_project_name "$project_id"
|
||||
|
||||
# Format: service, type, label, fields (comma-separated; empty = all fields)
|
||||
services=("dns" "zone" "DNS Zones" "name,dnsName,type,visibility,state,recordCount,id"
|
||||
"git" "instance" "Git Instances" ""
|
||||
"load-balancer" "" "Load Balancers" ""
|
||||
"logme" "instance" "LogMe Instances" ""
|
||||
"mariadb" "instance" "MariaDB Instances" ""
|
||||
"mongodbflex" "instance" "MongoDB Flex Instances" ""
|
||||
"object-storage" "bucket" "Object Storage Buckets" ""
|
||||
"observability" "instance" "Observability Instances" ""
|
||||
"opensearch" "instance" "OpenSearch Instances" ""
|
||||
"postgresflex" "instance" "Postgres Flex Instances" ""
|
||||
"public-ip" "" "Public IPs" ""
|
||||
"rabbitmq" "instance" "RabbitMQ Instances" ""
|
||||
"redis" "instance" "Redis Instances" ""
|
||||
"secrets-manager" "instance" "Secrets Manager Instances" ""
|
||||
"service-account" "" "Service Accounts" ""
|
||||
"ske" "cluster" "SKE Clusters" "")
|
||||
|
||||
for ((i=0; i<${#services[@]}; i+=4)); do
|
||||
fetch_resources "${services[i]}" "${services[i+1]}" "${services[i+2]}" "${services[i+3]}"
|
||||
done
|
||||
|
||||
echo "last update: $(date +"%a, %d-%b-%Y %H:%M:%S %Z")"
|
||||
}
|
||||
|
||||
# Main script
|
||||
if [ $# -eq 0 ]; then
|
||||
echo "Error: No project IDs provided"
|
||||
display_usage
|
||||
fi
|
||||
|
||||
# Process each project ID
|
||||
for project_id in "$@"; do
|
||||
get_project_resources "$project_id"
|
||||
done
|
||||
467
scripts/vault-migrate.sh
Executable file
467
scripts/vault-migrate.sh
Executable file
|
|
@ -0,0 +1,467 @@
|
|||
#!/usr/bin/env bash
|
||||
# Vault Secret Migration Script - Migrate secrets between Vault instances using v2 API
|
||||
# Usage: vault-migrate.sh [options]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Default values
|
||||
DRY_RUN=false
|
||||
VERBOSE=false
|
||||
SKIP_EXISTING=false
|
||||
DEBUG=false
|
||||
|
||||
# Source Vault configuration
|
||||
: "${SOURCE_VAULT_ADDR:?Environment variable SOURCE_VAULT_ADDR is not set}"
|
||||
: "${SOURCE_SM_ID:?Environment variable SOURCE_SM_ID is not set (KV mount path)}"
|
||||
|
||||
# Source authentication method (userpass or ldap)
|
||||
SOURCE_AUTH_METHOD="${SOURCE_AUTH_METHOD:-userpass}"
|
||||
|
||||
# Validate source authentication variables based on method
|
||||
if [ "${SOURCE_AUTH_METHOD}" = "userpass" ]; then
|
||||
: "${SOURCE_SM_USERNAME:?Environment variable SOURCE_SM_USERNAME is not set (required for userpass auth)}"
|
||||
: "${SOURCE_SM_PASSWORD:?Environment variable SOURCE_SM_PASSWORD is not set (required for userpass auth)}"
|
||||
elif [ "${SOURCE_AUTH_METHOD}" = "ldap" ]; then
|
||||
: "${SOURCE_LDAP_USERNAME:?Environment variable SOURCE_LDAP_USERNAME is not set (required for ldap auth)}"
|
||||
# Note: LDAP password is NOT required as env var - will use interactive login
|
||||
else
|
||||
echo "Error: SOURCE_AUTH_METHOD must be 'userpass' or 'ldap' (got: ${SOURCE_AUTH_METHOD})"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Target Vault configuration
|
||||
: "${TARGET_VAULT_ADDR:?Environment variable TARGET_VAULT_ADDR is not set}"
|
||||
: "${TARGET_SM_USERNAME:?Environment variable TARGET_SM_USERNAME is not set}"
|
||||
: "${TARGET_SM_PASSWORD:?Environment variable TARGET_SM_PASSWORD is not set}"
|
||||
: "${TARGET_SM_ID:?Environment variable TARGET_SM_ID is not set (KV mount path)}"
|
||||
|
||||
# Optional: specify paths to migrate (if not set, migrate all)
|
||||
MIGRATE_PATHS="${MIGRATE_PATHS:-}"
|
||||
|
||||
# Check for required dependencies
|
||||
for cmd in vault jq; do
|
||||
if ! command -v "$cmd" &>/dev/null; then
|
||||
echo "Error: Required command '$cmd' not found in PATH"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# Function to display usage
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: $0 [options]
|
||||
|
||||
Migrate secrets from one Vault instance to another using Vault v2 KV API.
|
||||
|
||||
Options:
|
||||
-d, --dry-run Show what would be migrated without making changes
|
||||
-v, --verbose Enable verbose output
|
||||
-s, --skip-existing Skip secrets that already exist in target
|
||||
--debug Enable debug output (shows all vault commands)
|
||||
-h, --help Show this help message
|
||||
|
||||
Environment variables required:
|
||||
|
||||
Source Vault:
|
||||
SOURCE_VAULT_ADDR Source Vault address (e.g., https://vault.example.com)
|
||||
SOURCE_SM_ID Source KV secrets engine mount path
|
||||
SOURCE_AUTH_METHOD Authentication method: "userpass" or "ldap" (default: userpass)
|
||||
|
||||
For userpass authentication:
|
||||
SOURCE_SM_USERNAME Source Vault username
|
||||
SOURCE_SM_PASSWORD Source Vault password
|
||||
|
||||
For LDAP authentication (interactive password prompt):
|
||||
SOURCE_LDAP_USERNAME LDAP username
|
||||
|
||||
Target Vault (always uses userpass):
|
||||
TARGET_VAULT_ADDR Target Vault address
|
||||
TARGET_SM_USERNAME Target Vault username for authentication
|
||||
TARGET_SM_PASSWORD Target Vault password for authentication
|
||||
TARGET_SM_ID Target KV secrets engine mount path
|
||||
|
||||
Optional:
|
||||
MIGRATE_PATHS Space-separated list of paths to migrate (default: all)
|
||||
Example: export MIGRATE_PATHS="postgresql redis terraform"
|
||||
|
||||
Examples:
|
||||
# Migrate all secrets (source uses userpass)
|
||||
$ export SOURCE_AUTH_METHOD="userpass"
|
||||
$ export SOURCE_SM_USERNAME="user" SOURCE_SM_PASSWORD="pass"
|
||||
$ vault-migrate.sh
|
||||
|
||||
# Migrate using LDAP for source authentication (will prompt for password)
|
||||
$ export SOURCE_AUTH_METHOD="ldap"
|
||||
$ export SOURCE_LDAP_USERNAME="kopps"
|
||||
$ vault-migrate.sh --dry-run --verbose
|
||||
|
||||
# Migrate specific paths only with LDAP
|
||||
$ export SOURCE_AUTH_METHOD="ldap"
|
||||
$ export SOURCE_LDAP_USERNAME="kopps"
|
||||
$ export MIGRATE_PATHS="postgresql redis"
|
||||
$ vault-migrate.sh
|
||||
|
||||
# Skip secrets that already exist in target
|
||||
$ vault-migrate.sh --skip-existing --verbose
|
||||
|
||||
EOF
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Function to authenticate to source Vault
|
||||
authenticate_source() {
|
||||
if [ -z "${SOURCE_VAULT_TOKEN:-}" ]; then
|
||||
if [ "${SOURCE_AUTH_METHOD}" = "userpass" ]; then
|
||||
log_verbose "Authenticating to source Vault at ${SOURCE_VAULT_ADDR} (method: userpass, user: ${SOURCE_SM_USERNAME})..."
|
||||
SOURCE_VAULT_TOKEN=$(vault login --address "${SOURCE_VAULT_ADDR}" -no-store -format=json \
|
||||
--method=userpass username="${SOURCE_SM_USERNAME}" password="${SOURCE_SM_PASSWORD}" 2>/dev/null | jq -r .auth.client_token)
|
||||
elif [ "${SOURCE_AUTH_METHOD}" = "ldap" ]; then
|
||||
log_info "Authenticating to source Vault at ${SOURCE_VAULT_ADDR} (method: ldap, user: ${SOURCE_LDAP_USERNAME})..."
|
||||
|
||||
# Interactive LDAP login - vault will prompt for password
|
||||
if ! vault login -address="${SOURCE_VAULT_ADDR}" -method=ldap username="${SOURCE_LDAP_USERNAME}" >/dev/null 2>&1; then
|
||||
echo "Error: Failed to authenticate to source Vault using LDAP"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Read token from vault token file
|
||||
if [ -f "${HOME}/.vault-token" ]; then
|
||||
SOURCE_VAULT_TOKEN=$(cat "${HOME}/.vault-token")
|
||||
else
|
||||
echo "Error: Vault token file not found at ${HOME}/.vault-token"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "${SOURCE_VAULT_TOKEN}" ] || [ "${SOURCE_VAULT_TOKEN}" = "null" ]; then
|
||||
echo "Error: Failed to authenticate to source Vault using ${SOURCE_AUTH_METHOD}"
|
||||
exit 1
|
||||
fi
|
||||
log_verbose "✓ Source authentication successful"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to authenticate to target Vault
|
||||
authenticate_target() {
|
||||
if [ -z "${TARGET_VAULT_TOKEN:-}" ]; then
|
||||
log_verbose "Authenticating to target Vault at ${TARGET_VAULT_ADDR}..."
|
||||
TARGET_VAULT_TOKEN=$(vault login --address "${TARGET_VAULT_ADDR}" -no-store -format=json \
|
||||
--method=userpass username="${TARGET_SM_USERNAME}" password="${TARGET_SM_PASSWORD}" 2>/dev/null | jq -r .auth.client_token)
|
||||
|
||||
if [ -z "${TARGET_VAULT_TOKEN}" ] || [ "${TARGET_VAULT_TOKEN}" = "null" ]; then
|
||||
echo "Error: Failed to authenticate to target Vault"
|
||||
exit 1
|
||||
fi
|
||||
log_verbose "✓ Target authentication successful"
|
||||
fi
|
||||
}
|
||||
|
||||
# Logging functions
|
||||
log_verbose() {
|
||||
if [ "${VERBOSE}" = true ]; then
|
||||
echo "$@" >&2
|
||||
fi
|
||||
}
|
||||
|
||||
log_debug() {
|
||||
if [ "${DEBUG}" = true ]; then
|
||||
echo "[DEBUG] $@" >&2
|
||||
fi
|
||||
}
|
||||
|
||||
log_info() {
|
||||
echo "$@" >&2
|
||||
}
|
||||
|
||||
log_error() {
|
||||
echo "ERROR: $@" >&2
|
||||
}
|
||||
|
||||
# Function to list paths at a given location
|
||||
list_paths_at() {
|
||||
local base_path="$1"
|
||||
local error_output
|
||||
error_output=$(mktemp)
|
||||
|
||||
local paths
|
||||
# Try JSON format first
|
||||
paths=$(VAULT_ADDR="${SOURCE_VAULT_ADDR}" VAULT_TOKEN="${SOURCE_VAULT_TOKEN}" \
|
||||
vault kv list -mount="${SOURCE_SM_ID}" "${base_path}" -format=json 2>"${error_output}" | jq -r '.[]' 2>/dev/null || echo "")
|
||||
|
||||
# If JSON failed, try table format
|
||||
if [ -z "${paths}" ]; then
|
||||
paths=$(VAULT_ADDR="${SOURCE_VAULT_ADDR}" VAULT_TOKEN="${SOURCE_VAULT_TOKEN}" \
|
||||
vault kv list -mount="${SOURCE_SM_ID}" "${base_path}" 2>"${error_output}" | grep -v "^Keys$" | grep -v "^----$" | grep -v "^$" || echo "")
|
||||
fi
|
||||
|
||||
rm -f "${error_output}"
|
||||
echo "${paths}"
|
||||
}
|
||||
|
||||
# Function to recursively get all secret paths from source Vault
|
||||
get_all_paths_recursive() {
|
||||
local prefix="$1"
|
||||
local paths
|
||||
|
||||
paths=$(list_paths_at "${prefix}")
|
||||
|
||||
for item in ${paths}; do
|
||||
if [[ "${item}" == */ ]]; then
|
||||
# It's a folder, recurse into it
|
||||
log_debug "Found folder: ${prefix}${item}"
|
||||
get_all_paths_recursive "${prefix}${item}"
|
||||
else
|
||||
# It's a secret, add it to the list
|
||||
echo "${prefix}${item}"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# Function to get all paths from source Vault
|
||||
get_all_paths() {
|
||||
authenticate_source
|
||||
|
||||
log_verbose "Fetching all secret paths from source Vault..."
|
||||
|
||||
local cmd="VAULT_ADDR=\"${SOURCE_VAULT_ADDR}\" VAULT_TOKEN=\"${SOURCE_VAULT_TOKEN}\" vault kv list -mount=\"${SOURCE_SM_ID}\""
|
||||
log_debug "Command: ${cmd}"
|
||||
|
||||
# Get all paths recursively starting from root
|
||||
local all_secrets
|
||||
all_secrets=$(get_all_paths_recursive "")
|
||||
|
||||
if [ -z "${all_secrets}" ]; then
|
||||
log_error "No secrets found in source Vault"
|
||||
log_error "Mount path: ${SOURCE_SM_ID}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log_debug "Found secrets: ${all_secrets}"
|
||||
echo "${all_secrets}"
|
||||
}
|
||||
|
||||
# Function to get secret from source
|
||||
get_source_secret() {
|
||||
local path="$1"
|
||||
|
||||
authenticate_source
|
||||
|
||||
local cmd="VAULT_ADDR=\"${SOURCE_VAULT_ADDR}\" VAULT_TOKEN=\"<hidden>\" vault kv get -mount=\"${SOURCE_SM_ID}\" -format=json \"${path}\""
|
||||
log_debug "Command: ${cmd}"
|
||||
|
||||
local error_output
|
||||
error_output=$(mktemp)
|
||||
|
||||
local result
|
||||
result=$(VAULT_ADDR="${SOURCE_VAULT_ADDR}" VAULT_TOKEN="${SOURCE_VAULT_TOKEN}" \
|
||||
vault kv get -mount="${SOURCE_SM_ID}" -format=json "${path}" 2>"${error_output}" | jq -r '.data.data')
|
||||
|
||||
local exit_code=$?
|
||||
|
||||
if [ ${exit_code} -ne 0 ] || [ -z "${result}" ] || [ "${result}" = "null" ]; then
|
||||
log_debug "Failed to read secret at path: ${path}"
|
||||
if [ -s "${error_output}" ]; then
|
||||
log_debug "Vault error output:"
|
||||
cat "${error_output}" >&2
|
||||
fi
|
||||
rm -f "${error_output}"
|
||||
echo ""
|
||||
return 1
|
||||
fi
|
||||
|
||||
rm -f "${error_output}"
|
||||
echo "${result}"
|
||||
}
|
||||
|
||||
# Function to check if secret exists in target
|
||||
target_secret_exists() {
|
||||
local path="$1"
|
||||
|
||||
authenticate_target
|
||||
|
||||
VAULT_ADDR="${TARGET_VAULT_ADDR}" VAULT_TOKEN="${TARGET_VAULT_TOKEN}" \
|
||||
vault kv get -mount="${TARGET_SM_ID}" -format=json "${path}" 2>/dev/null >/dev/null
|
||||
|
||||
return $?
|
||||
}
|
||||
|
||||
# Function to put secret to target
|
||||
put_target_secret() {
|
||||
local path="$1"
|
||||
local secret_data="$2"
|
||||
|
||||
authenticate_target
|
||||
|
||||
# Convert JSON object to key=value arguments
|
||||
local put_args=()
|
||||
while IFS= read -r key; do
|
||||
local value=$(echo "${secret_data}" | jq -r --arg k "$key" '.[$k]')
|
||||
put_args+=("${key}=${value}")
|
||||
done < <(echo "${secret_data}" | jq -r 'keys[]')
|
||||
|
||||
if [ ${#put_args[@]} -eq 0 ]; then
|
||||
log_error "No key-value pairs found in secret data for path '${path}'"
|
||||
return 1
|
||||
fi
|
||||
|
||||
VAULT_ADDR="${TARGET_VAULT_ADDR}" VAULT_TOKEN="${TARGET_VAULT_TOKEN}" \
|
||||
vault kv put -mount="${TARGET_SM_ID}" "${path}" "${put_args[@]}" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
# Function to migrate a single secret path
|
||||
migrate_secret() {
|
||||
local path="$1"
|
||||
local migrated_count=0
|
||||
local skipped_count=0
|
||||
local failed_count=0
|
||||
|
||||
log_info "Processing: ${path}"
|
||||
|
||||
# Check if secret exists in target and skip if requested
|
||||
if [ "${SKIP_EXISTING}" = true ]; then
|
||||
if target_secret_exists "${path}"; then
|
||||
log_info " ⊘ Skipped (already exists in target)"
|
||||
echo "0:1:0"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# Get secret from source
|
||||
local secret_data
|
||||
secret_data=$(get_source_secret "${path}")
|
||||
|
||||
if [ -z "${secret_data}" ] || [ "${secret_data}" = "null" ]; then
|
||||
log_info " ⊘ Skipped (empty or deleted secret)"
|
||||
log_verbose " Path: ${path}"
|
||||
echo "0:1:0"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Count keys
|
||||
local key_count
|
||||
key_count=$(echo "${secret_data}" | jq 'keys | length')
|
||||
log_verbose " Found ${key_count} keys"
|
||||
|
||||
# Dry run mode
|
||||
if [ "${DRY_RUN}" = true ]; then
|
||||
log_info " → Would migrate ${key_count} keys (dry-run)"
|
||||
if [ "${VERBOSE}" = true ]; then
|
||||
echo "${secret_data}" | jq -r 'to_entries[] | " - \(.key)"' >&2
|
||||
fi
|
||||
echo "0:0:0"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Put secret to target
|
||||
if put_target_secret "${path}" "${secret_data}"; then
|
||||
log_info " ✓ Migrated ${key_count} keys"
|
||||
echo "1:0:0"
|
||||
else
|
||||
log_error " ✗ Failed to write to target"
|
||||
echo "0:0:1"
|
||||
fi
|
||||
}
|
||||
|
||||
# Parse command line arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
-d|--dry-run)
|
||||
DRY_RUN=true
|
||||
shift
|
||||
;;
|
||||
-v|--verbose)
|
||||
VERBOSE=true
|
||||
shift
|
||||
;;
|
||||
-s|--skip-existing)
|
||||
SKIP_EXISTING=true
|
||||
shift
|
||||
;;
|
||||
--debug)
|
||||
DEBUG=true
|
||||
VERBOSE=true # Debug implies verbose
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
echo "Error: Unknown option: $1"
|
||||
echo
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Main migration logic
|
||||
main() {
|
||||
log_info "=== Vault Secret Migration ==="
|
||||
log_info "Source: ${SOURCE_VAULT_ADDR} (mount: ${SOURCE_SM_ID}, auth: ${SOURCE_AUTH_METHOD})"
|
||||
log_info "Target: ${TARGET_VAULT_ADDR} (mount: ${TARGET_SM_ID}, auth: userpass)"
|
||||
|
||||
if [ "${DRY_RUN}" = true ]; then
|
||||
log_info "Mode: DRY RUN (no changes will be made)"
|
||||
fi
|
||||
|
||||
log_info ""
|
||||
|
||||
# Authenticate to both vaults
|
||||
authenticate_source
|
||||
authenticate_target
|
||||
|
||||
# Determine which paths to migrate
|
||||
local paths_to_migrate
|
||||
if [ -n "${MIGRATE_PATHS}" ]; then
|
||||
log_info "Migrating specified paths: ${MIGRATE_PATHS}"
|
||||
paths_to_migrate="${MIGRATE_PATHS}"
|
||||
else
|
||||
log_info "Migrating all paths from source Vault..."
|
||||
paths_to_migrate=$(get_all_paths)
|
||||
fi
|
||||
|
||||
log_info ""
|
||||
|
||||
# Statistics
|
||||
local total_migrated=0
|
||||
local total_skipped=0
|
||||
local total_failed=0
|
||||
local total_paths=0
|
||||
|
||||
# Migrate each path
|
||||
for path in ${paths_to_migrate}; do
|
||||
((total_paths++)) || true
|
||||
|
||||
result=$(migrate_secret "${path}")
|
||||
IFS=':' read -r migrated skipped failed <<< "${result}"
|
||||
|
||||
((total_migrated+=migrated)) || true
|
||||
((total_skipped+=skipped)) || true
|
||||
((total_failed+=failed)) || true
|
||||
done
|
||||
|
||||
# Print summary
|
||||
log_info ""
|
||||
log_info "=== Migration Summary ==="
|
||||
log_info "Total paths processed: ${total_paths}"
|
||||
|
||||
if [ "${DRY_RUN}" = false ]; then
|
||||
log_info "Successfully migrated: ${total_migrated}"
|
||||
if [ "${SKIP_EXISTING}" = true ]; then
|
||||
log_info "Skipped (existing): ${total_skipped}"
|
||||
fi
|
||||
log_info "Failed: ${total_failed}"
|
||||
|
||||
if [ ${total_failed} -gt 0 ]; then
|
||||
log_error "Migration completed with errors"
|
||||
exit 1
|
||||
else
|
||||
log_info ""
|
||||
log_info "✓ Migration completed successfully!"
|
||||
fi
|
||||
else
|
||||
log_info "✓ Dry run completed"
|
||||
fi
|
||||
}
|
||||
|
||||
# Run main function
|
||||
main
|
||||
Loading…
Add table
Add a link
Reference in a new issue