example(telemetry-router): build hub and spoke architecture for logs
All checks were successful
Default CI / Check for Open TODOs (pull_request) Successful in 28s
Default CI / Secret Scanner (TruffleHog) (pull_request) Successful in 1m3s
Default CI / Pre-Commit Hooks (pull_request) Successful in 1m34s

This commit is contained in:
Mauritz Uphoff 2026-05-29 10:55:06 +02:00
parent 8d6501e44e
commit 359bfde905
9 changed files with 396 additions and 0 deletions

View file

@ -0,0 +1,24 @@
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/stackitcloud/stackit" {
version = "0.96.0"
constraints = ">= 0.95.0"
hashes = [
"h1:NgwbVCV5pfBVMO3xUMop4l5AzvVv3BuBzXpJjgoZfSU=",
"zh:04d309851424a53d3d014dde3b143fc1cdc19fbebf558eb4b927878103f78fb0",
"zh:0dde99e7b343fa01f8eefc378171fb8621bedb20f59157d6cc8e3d46c738105f",
"zh:0ebcdf98a47f301e12925803198320d637552ef57abc49e2a48a009f1ddbf39a",
"zh:176238c057193c9c60c365b83463e758892186fcc2bd14bc9bbf69bf471f1d6b",
"zh:1c514ec6d09ee210ebb813d49b7d3a71b5b9d0b173c743bce9ab937b1e3d303a",
"zh:20433d0dc7e4aa2a806863fc289a2cecb19763624f199babfbe44f22d4d9150f",
"zh:452ceacbe4a1f70c81320b9223f4958c9bc122508c79e86bc97cb9241682c053",
"zh:5f893229f41f8dc2169b5b02785fb2988e8cad2141722a411711182bafefa015",
"zh:69383e27067a6413300d3acbcdad8f890bd187e16630580c09900ba379659284",
"zh:694de24bd05027c3c8b7a7c477973f76cd5a11d7fd38819026b5a0e588698fd9",
"zh:7c7399e3223dd76efb56ca2e3c9435b41bcbaf549839cec36023f801ca5bdcd2",
"zh:8a92b221694c59648d22e2e2a0059015872eff7034ae0ba9eb801fe399644a2c",
"zh:90a8ae716c9bc6c8804a38f7a903c7af7114ce324d0126c64e1447b6d255cdba",
"zh:d29eb17fde9460c5ce3c7a7975eef0ad7fea692eb17fad5e0421952e4d29dbd2",
]
}

View file

@ -0,0 +1,28 @@
# Copyright 2026 Schwarz Digits Cloud GmbH & Co. KG
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_providers {
stackit = {
source = "stackitcloud/stackit"
version = ">=0.95.0"
}
}
}
provider "stackit" {
default_region = var.stackit_region
service_account_key_path = var.stackit_service_account_key_path
enable_beta_resources = true
}

View file

@ -0,0 +1,30 @@
# Copyright 2026 Schwarz Digits Cloud GmbH & Co. KG
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
variable "stackit_org_id" {
type = string
}
variable "stackit_region" {
type = string
default = "eu01"
}
variable "stackit_service_account_key_path" {
type = string
}
variable "stackit_owner_email" {
type = string
}

View file

@ -0,0 +1,43 @@
# Copyright 2026 Schwarz Digits Cloud GmbH & Co. KG
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
resource "stackit_resourcemanager_folder" "this" {
name = "mu-telemetry-router-link-test"
owner_email = var.stackit_owner_email
parent_container_id = var.stackit_org_id
}
resource "stackit_resourcemanager_project" "telemetry_hub" {
parent_container_id = stackit_resourcemanager_folder.this.container_id
name = "telemetry_hub"
owner_email = var.stackit_owner_email
}
resource "stackit_resourcemanager_project" "telemetry_spoke1" {
parent_container_id = stackit_resourcemanager_folder.this.container_id
name = "telemetry_spoke1"
owner_email = var.stackit_owner_email
}
resource "stackit_resourcemanager_project" "telemetry_spoke2" {
parent_container_id = stackit_resourcemanager_folder.this.container_id
name = "telemetry_spoke2"
owner_email = var.stackit_owner_email
}
resource "stackit_resourcemanager_project" "telemetry_spoke3" {
parent_container_id = stackit_resourcemanager_folder.this.container_id
name = "telemetry_spoke3"
owner_email = var.stackit_owner_email
}

View file

@ -0,0 +1,39 @@
# Copyright 2026 Schwarz Digits Cloud GmbH & Co. KG
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
resource "stackit_observability_instance" "this" {
project_id = stackit_resourcemanager_project.telemetry_hub.project_id
name = "telemetry_hub"
plan_name = "Observability-Large-EU01"
alert_config = null
}
resource "stackit_observability_credential" "router_ingest" {
project_id = stackit_resourcemanager_project.telemetry_hub.project_id
instance_id = stackit_observability_instance.this.instance_id
}
resource "stackit_logs_instance" "this" {
project_id = stackit_resourcemanager_project.telemetry_hub.project_id
region = "eu01"
display_name = "telemetry_hub"
retention_days = 30
}
resource "stackit_logs_access_token" "router_ingest" {
project_id = stackit_resourcemanager_project.telemetry_hub.project_id
instance_id = stackit_logs_instance.this.instance_id
display_name = "router-ingest-token"
permissions = ["write"]
}

View file

@ -0,0 +1,99 @@
# Copyright 2026 Schwarz Digits Cloud GmbH & Co. KG
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Create the Telemetry Router Instance in the Hub project
resource "stackit_telemetryrouter_instance" "hub_router" {
project_id = stackit_resourcemanager_project.telemetry_hub.project_id
display_name = "hub-telemetry-router"
description = "Central Telemetry Router for spoke projects and parent folder"
}
# Create an Access Token for the Router
# This token will be used by the links to authenticate with the router
resource "stackit_telemetryrouter_access_token" "hub_router_token" {
project_id = stackit_resourcemanager_project.telemetry_hub.project_id
instance_id = stackit_telemetryrouter_instance.hub_router.instance_id
display_name = "hub-router-link-token"
}
# Link Spoke Project 1 to the Hub Router
resource "stackit_telemetrylink" "spoke1_link" {
resource_type = "project"
resource_id = stackit_resourcemanager_project.telemetry_spoke1.project_id
display_name = "spoke1-to-hub-link"
telemetry_router_id = stackit_telemetryrouter_instance.hub_router.instance_id
access_token = stackit_telemetryrouter_access_token.hub_router_token.access_token
}
# Link Spoke Project 2 to the Hub Router
resource "stackit_telemetrylink" "spoke2_link" {
resource_type = "project"
resource_id = stackit_resourcemanager_project.telemetry_spoke2.project_id
display_name = "spoke2-to-hub-link"
telemetry_router_id = stackit_telemetryrouter_instance.hub_router.instance_id
access_token = stackit_telemetryrouter_access_token.hub_router_token.access_token
}
# Link Spoke Project 3 to the Hub Router
resource "stackit_telemetrylink" "spoke3_link" {
resource_type = "project"
resource_id = stackit_resourcemanager_project.telemetry_spoke3.project_id
display_name = "spoke3-to-hub-link"
telemetry_router_id = stackit_telemetryrouter_instance.hub_router.instance_id
access_token = stackit_telemetryrouter_access_token.hub_router_token.access_token
}
# Link the parent Folder to the Hub Router
# This allows telemetry data from all projects within the folder (if configured) to be routed via the hub router
resource "stackit_telemetrylink" "folder_link" {
resource_type = "folder"
// keep like this
resource_id = stackit_resourcemanager_folder.this.folder_id
display_name = "folder-to-hub-link"
telemetry_router_id = stackit_telemetryrouter_instance.hub_router.instance_id
access_token = stackit_telemetryrouter_access_token.hub_router_token.access_token
}
# Create a Destination for the Router to send filtered logs to the central Observability instance
# We match only logs from the 'critical-app' service
resource "stackit_telemetryrouter_destination" "observability_destination" {
project_id = stackit_resourcemanager_project.telemetry_hub.project_id
instance_id = stackit_telemetryrouter_instance.hub_router.instance_id
display_name = "filtered-obs-dest"
config = {
config_type = "OpenTelemetry"
opentelemetry = {
uri = stackit_observability_instance.this.otlp_http_logs_url
basic_auth = {
username = stackit_observability_credential.router_ingest.username
password = stackit_observability_credential.router_ingest.password
}
}
}
}
# Create a Destination for the Router to send everything NOT filtered to the central Logs instance
resource "stackit_telemetryrouter_destination" "logs_destination" {
project_id = stackit_resourcemanager_project.telemetry_hub.project_id
instance_id = stackit_telemetryrouter_instance.hub_router.instance_id
display_name = "unfiltered-logs-dest"
config = {
config_type = "OpenTelemetry"
opentelemetry = {
# Prepend https:// as the OTLP URI must have a protocol
uri = "https://${stackit_logs_instance.this.ingest_otlp_url}"
bearer_token = stackit_logs_access_token.router_ingest.access_token
}
}
}

View file

@ -0,0 +1,53 @@
# Copyright 2026 Schwarz Digits Cloud GmbH & Co. KG
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
output "telemetry_router_id" {
description = "The ID of the central Telemetry Router"
value = stackit_telemetryrouter_instance.hub_router.instance_id
}
output "telemetry_router_uri" {
description = "The OTLP ingest URI of the central Telemetry Router"
value = stackit_telemetryrouter_instance.hub_router.uri
}
output "spoke1_link_id" {
description = "The ID of the Telemetry Link for Spoke Project 1"
value = stackit_telemetrylink.spoke1_link.id
}
output "spoke2_link_id" {
description = "The ID of the Telemetry Link for Spoke Project 2"
value = stackit_telemetrylink.spoke2_link.id
}
output "spoke3_link_id" {
description = "The ID of the Telemetry Link for Spoke Project 3"
value = stackit_telemetrylink.spoke3_link.id
}
output "folder_link_id" {
description = "The ID of the Telemetry Link for the parent Folder"
value = stackit_telemetrylink.folder_link.id
}
output "observability_logs_ingest_url" {
description = "The OTLP HTTP logs ingest URL for the Observability instance"
value = "https://${stackit_observability_instance.this.otlp_http_logs_url}"
}
output "logs_ingest_url" {
description = "The OTLP ingest URL for the Logs instance"
value = "https://${stackit_logs_instance.this.ingest_otlp_url}"
}

View file

@ -0,0 +1,9 @@
# Maintainers
General maintainers:
- Mauritz Uphoff (mauritz.uphoff@digits.schwarz)
This example is actively maintained. The owner is responsible for reviewing and updating dependencies and functionalities on a monthly basis.
For questions, issues, or feature requests, please email general maintainers.
Please include the BP name and version in your request. We will track your request as an issue.

View file

@ -0,0 +1,71 @@
# Telemetry Router: Hub-and-Spoke Observability Architecture
This example demonstrates a production-ready **Hub-and-Spoke** architecture for centralized telemetry management across multiple STACKIT projects and folders.
## The Goal: Centralized Governance & Intelligent Routing
In large-scale cloud environments, managing observability for dozens or hundreds of projects individually is complex and inefficient. The **STACKIT Telemetry Router** solves this by providing a central "ingestion hub" that can receive, filter, and route data based on your organizational needs.
This example is designed to show customers how to:
1. **Centralize Management**: Use a dedicated "Hub" project to host the core observability infrastructure.
2. **Simplify Onboarding**: Use **Telemetry Links** to connect entire folders or individual projects to the hub router with minimal configuration.
3. **Optimize Costs & Performance**: Implement **Splitter Logic** to route high-value data (e.g., critical application logs) to high-performance observability instances, while sending standard logs to more cost-effective long-term storage.
## Architecture Overview
### Hub-and-Spoke Model
- **The Hub**: A central project hosting the `stackit_telemetryrouter_instance`, a `stackit_observability_instance` (for metrics/critical logs), and a `stackit_logs_instance` (for long-term storage).
- **The Spokes**: Multiple projects (and an entire parent folder) that stream their telemetry data to the Hub.
### Intelligent Routing (The "Splitter")
One of the most powerful features of the Telemetry Router is the ability to route data conditionally using **Destinations**. In this example:
- **Critical Stream**: Any log where `service.name == "critical-app"` is routed to the **Observability Instance**. This allows SRE teams to have high-performance dashboards and alerting for their most important services.
- **Standard Stream**: Everything else (`service.name != "critical-app"`) is routed to the **Logs Instance**. This ensures all data is archived for compliance and debugging without cluttering the premium observability environment.
## Resources Used
| Resource | Purpose |
| :------------------------------------- | :--------------------------------------------------------------------------------- |
| `stackit_telemetryrouter_instance` | The central brain that receives and distributes telemetry data. |
| `stackit_telemetrylink` | Establishes the trust and connection between a Spoke (Project/Folder) and the Hub. |
| `stackit_telemetryrouter_destination` | Defines the routing rules and backend storage (e.g., Observability vs. Logs). |
| `stackit_telemetryrouter_access_token` | Securely authorizes the links to send data to the router. |
## How to Use This Example
### 1. Prerequisites
- A STACKIT Organization ID.
- A Service Account with appropriate permissions.
### 2. Configuration
Create a `terraform.tfvars` file with the following:
```hcl
stackit_org_id = "your-org-uuid"
stackit_owner_email = "owner@example.com"
stackit_service_account_key_path = "path/to/key.json"
```
### 3. Execution
```bash
terraform init
terraform apply
```
## Why This Matters for Customers
- **Central Governance**: Platform teams can define routing and storage policies in one place.
- **Reduced Complexity**: Project owners don't need to manage their own logging/metrics backends.
- **Compliance**: Folder-level linking ensures that every project created within a folder is automatically covered by the organizational observability policy.
- **Data Sovereignty**: Keep your data within your STACKIT environment while maintaining full control over where it is stored.
---
**Note:** The Telemetry Router and Telemetry Link services are currently in **beta**. Ensure `enable_beta_resources = true` is set in your provider configuration.