s3-sse-c script #27
2 changed files with 152 additions and 9 deletions
|
|
@ -6,15 +6,84 @@ Helper scripts for working with STACKIT services.
|
|||
|
||||
## Overview
|
||||
|
||||
| Script | Purpose | Required tools |
|
||||
| ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ---------------------------------- |
|
||||
| [`check-stackit-ip.sh`](#check-stackit-ipsh) | Check whether a given IP address belongs to STACKIT's public IP ranges. | `stackit`, `jq`, `grepcidr` |
|
||||
| [`create-kubeconfig-multiple-projects.sh`](#create-kubeconfig-multiple-projectssh) | Generate kubeconfig entries for every SKE cluster across one or more STACKIT projects. | `stackit`, `yq` |
|
||||
| [`delete-unused-volumes.sh`](#delete-unused-volumessh) | Delete all STACKIT volumes whose status is `AVAILABLE` (i.e. not attached). | `stackit`, `yq` |
|
||||
| [`list-project-resources.sh`](#list-project-resourcessh) | Render a Markdown inventory of resources (DNS, SKE, databases, storage, …) for one or more STACKIT projects. | `stackit`, `jq` |
|
||||
| [`ske-show-versions.sh`](#ske-show-versionssh) | Print overview of SKE cluster Kubernetes versions and nodepool image versions, marking deprecated versions. | `stackit` (>= 0.59.0), `jq`, `awk` |
|
||||
| [`smctl.sh`](#smctlsh) | Unified CLI wrapper around HashiCorp Vault for the STACKIT Secret Manager (KV v2), think `kubectl` for secrets | `vault`, `jq` |
|
||||
| [`vault-migrate.sh`](#vault-migratesh) | Migrate secrets between two Vault instances using the KV v2 API (supports userpass and LDAP for source). | `vault`, `jq` |
|
||||
| Script | Purpose | Required tools |
|
||||
| ---------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ---------------------------------- |
|
||||
| [`s3-ssec.sh`](#s3-ssec) | Upload and Download Files to a STACKIT S3 Bucket (Object Storage) and enable Server Side Encryption with Custom Keys. | `awscli`, `python3-awscrt` |
|
||||
| [`check-stackit-ip.sh`](#check-stackit-ipsh) | Check whether a given IP address belongs to STACKIT's public IP ranges. | `stackit`, `jq`, `grepcidr` |
|
||||
| [`create-kubeconfig-multiple-projects.sh`](#create-kubeconfig-multiple-projectssh) | Generate kubeconfig entries for every SKE cluster across one or more STACKIT projects. | `stackit`, `yq` |
|
||||
| [`delete-unused-volumes.sh`](#delete-unused-volumessh) | Delete all STACKIT volumes whose status is `AVAILABLE` (i.e. not attached). | `stackit`, `yq` |
|
||||
| [`list-project-resources.sh`](#list-project-resourcessh) | Render a Markdown inventory of resources (DNS, SKE, databases, storage, …) for one or more STACKIT projects. | `stackit`, `jq` |
|
||||
| [`ske-show-versions.sh`](#ske-show-versionssh) | Print overview of SKE cluster Kubernetes versions and nodepool image versions, marking deprecated versions. | `stackit` (>= 0.59.0), `jq`, `awk` |
|
||||
| [`smctl.sh`](#smctlsh) | Unified CLI wrapper around HashiCorp Vault for the STACKIT Secret Manager (KV v2), think `kubectl` for secrets | `vault`, `jq` |
|
||||
| [`vault-migrate.sh`](#vault-migratesh) | Migrate secrets between two Vault instances using the KV v2 API (supports userpass and LDAP for source). | `vault`, `jq` |
|
||||
|
||||
---
|
||||
|
||||
## `s3-ssec.sh`
|
||||
|
||||
### STACKIT S3 Object Storage SSE-C Automation Script
|
||||
|
||||
This script automates secure Server-Side Encryption with Customer-Provided Keys (SSE-C) using the native aws-cli tool against STACKIT Object Storage.
|
||||
|
||||
### Why This Script Exists
|
||||
|
||||
Standard S3 tools (s3cmd, rclone) often fail during SSE-C operations on Linux due to a Binary-to-Text Encoding Mismatch when passing keys via the command line. Furthermore, third-party tools frequently throw false alarms regarding data corruption because the server-side encrypted file's MD5 hash (ETag) no longer matches the local unencrypted file's hash.
|
||||
|
||||
The Solution: This script utilizes the official aws-cli along with a binary key file reference (fileb://). This approach bypasses shell encoding issues entirely. The AWS CLI natively understands SSE-C protocols, managing hash verifications and setting all necessary headers automatically.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
AWS CLI installed:
|
||||
|
||||
```bash
|
||||
sudo apt install awscli -y # Debian/Ubuntu
|
||||
sudo dnf install aws-cli -y # RHEL/CentOS
|
||||
```
|
||||
|
||||
Permissions: Root access (or appropriate sudo permissions) to store the encryption key safely under /root/ssec.key.
|
||||
|
||||
### Setup & Configuration
|
||||
|
||||
Open the script and fill in your STACKIT credentials and bucket name under the configuration block:
|
||||
|
||||
```bash
|
||||
export AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY"
|
||||
export AWS_SECRET_ACCESS_KEY="YOUR_SECRET_ACCESS_KEY"
|
||||
BUCKET="s3://your-bucket-name"
|
||||
```
|
||||
|
||||
Note: On its very first run, the script will automatically generate a cryptographically secure 32-byte binary AES key using openssl and lock down its file permissions (chmod 400).
|
||||
|
||||
### Usage
|
||||
|
||||
Make the script executable:
|
||||
|
||||
```bash
|
||||
chmod +x s3-ssec.sh
|
||||
```
|
||||
|
||||
1. Uploading a File (Encrypted)
|
||||
To upload and encrypt a local file, pass upload followed by the file path:
|
||||
|
||||
```bash
|
||||
./s3-ssec.sh upload /path/to/local/file.txt
|
||||
```
|
||||
|
||||
The file will be encrypted on the fly by the STACKIT storage backend using your generated key.
|
||||
|
||||
2. Downloading a File (Decrypted)
|
||||
To download and decrypt an object from the bucket, pass download followed by the remote object name:
|
||||
|
||||
```bash
|
||||
./s3-ssec.sh download file.txt
|
||||
```
|
||||
|
||||
The script will fetch the file, provide the required key headers, and save the decrypted file locally as ./file.txt_downloaded.
|
||||
|
||||
Security Warning ⚠️
|
||||
Backup your Key: If you lose the /root/ssec.key file, any data encrypted with it in the bucket cannot be recovered.
|
||||
|
||||
Credential Rotation: Never commit the script to public repositories while it contains live AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY strings. Use environment variables or secret managers in production environments.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
74
scripts/s3-ssec.sh
Executable file
74
scripts/s3-ssec.sh
Executable file
|
|
@ -0,0 +1,74 @@
|
|||
#!/bin/bash
|
||||
set -e # Exit immediately if a command exits with a non-zero status
|
||||
|
||||
# Help / Usage output
|
||||
usage() {
|
||||
echo "Usage: $0 <upload|download> <filename>"
|
||||
echo "Examples:"
|
||||
echo " $0 upload /path/to/local/file.txt"
|
||||
echo " $0 download file.txt"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Check for required arguments
|
||||
if [ -z "$1" ] || [ -z "$2" ]; then
|
||||
usage
|
||||
fi
|
||||
|
||||
ACTION="$1"
|
||||
TARGET_FILE="$2"
|
||||
|
||||
# ==============================================================================
|
||||
# CONFIGURATION
|
||||
# ==============================================================================
|
||||
export AWS_ACCESS_KEY_ID=""
|
||||
export AWS_SECRET_ACCESS_KEY=""
|
||||
export AWS_DEFAULT_REGION="eu01"
|
||||
|
||||
ENDPOINT="https://object.storage.eu01.onstackit.cloud"
|
||||
BUCKET="s3://mybucket"
|
||||
KEY_FILE="/root/ssec.key"
|
||||
|
||||
# ==============================================================================
|
||||
# 1. GENERATE BINARY KEY (Only required once if the key file doesn't exist)
|
||||
# ==============================================================================
|
||||
if [ ! -f "$KEY_FILE" ]; then
|
||||
echo "[*] Generating new binary 32-byte AES key..."
|
||||
openssl rand 32 > "$KEY_FILE"
|
||||
chmod 600 "$KEY_FILE"
|
||||
fi
|
||||
|
||||
# ==============================================================================
|
||||
# 2. ACTION EXECUTION (UPLOAD OR DOWNLOAD)
|
||||
# ==============================================================================
|
||||
if [ "$ACTION" == "upload" ]; then
|
||||
# Check if the local file exists before uploading
|
||||
if [ ! -f "$TARGET_FILE" ]; then
|
||||
echo "Error: Local file '$TARGET_FILE' does not exist!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
REMOTE_NAME=$(basename "$TARGET_FILE")
|
||||
|
||||
echo "[*] Starting SSE-C upload of '$TARGET_FILE' to STACKIT..."
|
||||
aws --endpoint-url "$ENDPOINT" s3 cp "$TARGET_FILE" "$BUCKET/$REMOTE_NAME" \
|
||||
--sse-c AES256 \
|
||||
--sse-c-key "fileb://$KEY_FILE"
|
||||
|
||||
echo "[+] Upload successful! '$REMOTE_NAME' is now stored encrypted in the bucket. 🚀"
|
||||
|
||||
elif [ "$ACTION" == "download" ]; then
|
||||
REMOTE_NAME=$(basename "$TARGET_FILE")
|
||||
LOCAL_DOWNLOAD_PATH="./${REMOTE_NAME}_downloaded"
|
||||
|
||||
echo "[*] Starting SSE-C download of '$REMOTE_NAME' from STACKIT..."
|
||||
aws --endpoint-url "$ENDPOINT" s3 cp "$BUCKET/$REMOTE_NAME" "$LOCAL_DOWNLOAD_PATH" \
|
||||
--sse-c AES256 \
|
||||
--sse-c-key "fileb://$KEY_FILE"
|
||||
|
||||
echo "[+] Download successful! Saved to '$LOCAL_DOWNLOAD_PATH' 🚀"
|
||||
|
||||
else
|
||||
echo "Error: Invalid action '$ACTION'."
|
||||
usage
|
||||
fi
|
||||
Loading…
Add table
Add a link
Reference in a new issue