Aiven Operator for Kubernetes

Manage Aiven services directly within your Kubernetes clusters using the Aiven Kubernetes Operator.

The Aiven Operator for Kubernetes enables you to manage Aiven infrastructure using Kubernetes Custom Resource Definitions (CRDs). This brings Aiven services into your Kubernetes workflow, allowing you to define and manage them alongside your application deployments.

Why use the Kubernetes Operator

Native Kubernetes integration

Manage Aiven services using kubectl and standard Kubernetes workflows

GitOps compatible

Define services in YAML and manage them through version control

Automatic secret management

Connection credentials stored as Kubernetes Secrets

Declarative configuration

Define desired state; the operator handles the rest

Prerequisites

• Aiven account
• Kubernetes cluster (version 1.21 or later) with admin access
• kubectl installed
• Helm 3.x installed
• Authentication token from Aiven Console

Installation

1. Create authentication token

   Generate a token from the Aiven Console under User Profile > Tokens

2. Create Kubernetes Secret

   Store your token as a Kubernetes Secret:

   kubectl create namespace aiven-system
   
   kubectl create secret generic aiven-token \
     --from-literal=token="YOUR_TOKEN_HERE" \
     --namespace aiven-system
   
3. Add Helm repository

   helm repo add aiven https://aiven.github.io/aiven-operator
   helm repo update
   
4. Install the operator

   helm install aiven-operator aiven/aiven-operator \
     --namespace aiven-system \
     --create-namespace
   
5. Verify installation

   kubectl get pods -n aiven-system
   

   You should see the operator pod running:

   NAME                              READY   STATUS    RESTARTS   AGE
   aiven-operator-xxxxxxxxxx-xxxxx   1/1     Running   0          1m
   

Quick start: Deploy PostgreSQL

Create an Aiven for PostgreSQL service using the operator:

1. Create service definition

   Create a file named postgres.yaml:

   postgres.yaml

   apiVersion: aiven.io/v1alpha1
   kind: PostgreSQL
   metadata:
     name: pg-sample
   spec:
     # Authentication token reference
     authSecretRef:
       name: aiven-token
       key: token
     
     # Connection info output
     connInfoSecretTarget:
       name: pg-connection
     
     # Aiven project
     project: my-aiven-project
     
     # Cloud and plan
     cloudName: google-europe-north1
     plan: startup-4
     
     # Maintenance window
     maintenanceWindowDow: sunday
     maintenanceWindowTime: "10:00:00"
     
     # PostgreSQL configuration
     userConfig:
       pg_version: '16'
       public_access:
         pg: true
   
2. Apply the resource

   kubectl apply -f postgres.yaml
   
3. Check service status

   kubectl get postgresqls.aiven.io pg-sample
   

   Wait for the STATE to become RUNNING:

   NAME        PROJECT           PLAN        STATE
   pg-sample   my-aiven-project  startup-4   RUNNING
   
4. Get connection information

   The operator automatically creates a Secret with connection details:

   kubectl get secret pg-connection -o yaml
   

Using the service from a pod

Connect to your Aiven service from within Kubernetes:

pod-example.yaml

apiVersion: v1
kind: Pod
metadata:
  name: postgres-client
spec:
  restartPolicy: Never
  containers:
    - name: postgres
      image: postgres:16-alpine
      command: ['psql', '$(DATABASE_URI)', '-c', 'SELECT version();']
      
      # Inject connection info as environment variables
      envFrom:
        - secretRef:
            name: pg-connection

Apply and check logs:

kubectl apply -f pod-example.yaml
kubectl logs postgres-client

Service examples

Apache Kafka

apiVersion: aiven.io/v1alpha1
kind: Kafka
metadata:
  name: kafka-sample
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  connInfoSecretTarget:
    name: kafka-connection
  
  project: my-aiven-project
  cloudName: google-europe-north1
  plan: business-4
  
  userConfig:
    kafka_version: '3.6'
    kafka:
      auto_create_topics_enable: true
      num_partitions: 3
      default_replication_factor: 2

Kafka Topic

apiVersion: aiven.io/v1alpha1
kind: KafkaTopic
metadata:
  name: orders-topic
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  project: my-aiven-project
  serviceName: kafka-sample
  
  topicName: orders
  partitions: 3
  replication: 2
  
  config:
    retention_ms: 604800000  # 7 days
    compression_type: gzip

MySQL

apiVersion: aiven.io/v1alpha1
kind: MySQL
metadata:
  name: mysql-sample
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  connInfoSecretTarget:
    name: mysql-connection
  
  project: my-aiven-project
  cloudName: aws-us-east-1
  plan: business-4
  
  userConfig:
    mysql_version: '8'
    public_access:
      mysql: true

OpenSearch

apiVersion: aiven.io/v1alpha1
kind: OpenSearch
metadata:
  name: opensearch-sample
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  connInfoSecretTarget:
    name: opensearch-connection
  
  project: my-aiven-project
  cloudName: azure-eastus
  plan: business-4
  
  userConfig:
    opensearch_version: '2'
    public_access:
      opensearch: true

Redis

apiVersion: aiven.io/v1alpha1
kind: Redis
metadata:
  name: redis-sample
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  connInfoSecretTarget:
    name: redis-connection
  
  project: my-aiven-project
  cloudName: google-us-central1
  plan: startup-4
  
  userConfig:
    redis_maxmemory_policy: allkeys-lru
    public_access:
      redis: true

Project management

Create and manage Aiven projects:

apiVersion: aiven.io/v1alpha1
kind: Project
metadata:
  name: my-project
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  # Project will be named 'dev-environment' in Aiven
  project: dev-environment
  
  # Billing group (optional)
  billingGroupId: "your-billing-group-id"
  
  # Project VPC (optional)
  cloudName: google-europe-north1
  
  tags:
    environment: development
    team: platform

Service integrations

Integrate services together:

apiVersion: aiven.io/v1alpha1
kind: ServiceIntegration
metadata:
  name: postgres-to-influx
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  project: my-aiven-project
  integrationType: metrics
  sourceServiceName: pg-sample
  destinationServiceName: influxdb-sample

Advanced configurations

VPC

apiVersion: aiven.io/v1alpha1
kind: ProjectVPC
metadata:
  name: my-vpc
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  project: my-aiven-project
  cloudName: google-europe-north1
  networkCidr: 10.0.0.0/24

Use VPC in a service:

apiVersion: aiven.io/v1alpha1
kind: PostgreSQL
metadata:
  name: pg-in-vpc
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  connInfoSecretTarget:
    name: pg-vpc-connection
  
  project: my-aiven-project
  cloudName: google-europe-north1
  plan: startup-4
  
  # Reference VPC
  projectVPCRef:
    name: my-vpc
  
  userConfig:
    pg_version: '16'

Connection pooling

apiVersion: aiven.io/v1alpha1
kind: ConnectionPool
metadata:
  name: pg-pool
spec:
  authSecretRef:
    name: aiven-token
    key: token
  
  connInfoSecretTarget:
    name: pg-pool-connection
  
  project: my-aiven-project
  serviceName: pg-sample
  databaseName: defaultdb
  username: avnadmin
  poolName: app-pool
  poolSize: 25
  poolMode: transaction

Managing secrets

Connection information

The operator creates Kubernetes Secrets containing:

PostgreSQL

DATABASE_URI: postgres://user:pass@host:port/db
PGHOST: hostname
PGPORT: port
PGDATABASE: database
PGUSER: username
PGPASSWORD: password
PGSSLMODE: require

Kafka

HOST: hostname
PORT: port
ACCESS_CERT: certificate
ACCESS_KEY: private key
CA_CERT: ca certificate

MySQL

HOST: hostname
PORT: port
DATABASE: database
USER: username
PASSWORD: password
MYSQL_URI: mysql://user:pass@host:port/db

Using secrets in deployments

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: app
          image: my-app:latest
          
          # Inject all connection info
          envFrom:
            - secretRef:
                name: pg-connection
          
          # Or select specific keys
          env:
            - name: DB_HOST
              valueFrom:
                secretKeyFor:
                  name: pg-connection
                  key: PGHOST

Monitoring and troubleshooting

Check operator logs

kubectl logs -n aiven-system -l app.kubernetes.io/name=aiven-operator

Check resource status

# List all Aiven resources
kubectl get aiven --all-namespaces

# Get detailed information
kubectl describe postgresql pg-sample

# Check events
kubectl get events --sort-by='.lastTimestamp'

Common issues

Service stuck in REBUILDING

Check the service status in Aiven Console or increase the timeout:

kubectl describe postgresql pg-sample

Look for error messages in the status conditions.

Authentication errors

Verify the token is correct and has necessary permissions:

kubectl get secret aiven-token -o jsonpath='{.data.token}' | base64 -d

Resource not updating

Delete and recreate the resource:

kubectl delete postgresql pg-sample
kubectl apply -f postgres.yaml

Clean up

Remove resources:

1. Delete services

   kubectl delete postgresql pg-sample
   kubectl delete kafka kafka-sample
   

   This also deletes the services in Aiven

2. Uninstall operator

   helm uninstall aiven-operator -n aiven-system
   
3. Remove namespace

   kubectl delete namespace aiven-system
   

GitOps with ArgoCD

Example ArgoCD Application:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: aiven-services
  namespace: argocd
spec:
  project: default
  
  source:
    repoURL: https://github.com/myorg/aiven-services
    targetRevision: main
    path: services
  
  destination:
    server: https://kubernetes.default.svc
    namespace: default
  
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Best practices

Use separate namespaces

Organize services by environment:

kubectl create namespace production
kubectl create namespace staging

kubectl apply -f postgres.yaml -n production
kubectl apply -f postgres-staging.yaml -n staging

Secure token storage

Use external secret managers:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: aiven-token
spec:
  secretStoreRef:
    name: vault
    kind: SecretStore
  target:
    name: aiven-token
  data:
    - secretKey: token
      remoteRef:
        key: aiven/token

Resource naming conventions

Use consistent naming:

metadata:
  name: app-postgres-prod
  labels:
    app: myapp
    component: database
    environment: production

Related resources

• Aiven Operator Documentation
• Aiven Operator GitHub Repository
• API Reference
• Kubernetes Custom Resources