mirror of
https://github.com/external-secrets/external-secrets.git
synced 2024-12-15 17:51:01 +00:00
316 lines
9.6 KiB
Markdown
316 lines
9.6 KiB
Markdown
|
```yaml
|
||
|
---
|
||
|
title: Secret Generators
|
||
|
version: v1alpha1
|
||
|
authors: Christian Hünig, Jan Steffen, Moritz Johner
|
||
|
creation-date: 2022-07-08
|
||
|
status: draft
|
||
|
---
|
||
|
```
|
||
|
|
||
|
# Secret Generators
|
||
|
|
||
|
## Table of Contents
|
||
|
|
||
|
<!-- toc -->
|
||
|
// autogen please
|
||
|
<!-- /toc -->
|
||
|
|
||
|
## Summary
|
||
|
|
||
|
There are use-cases where a secret could be generated by the operator instead of using pre-existing ones from the secret store. For example a MongoDB could be spun up using a password generated by the operator. This password is then used by an application in the cluster to access the database. No human ever needs to read that secret or have to specify it beforehand. A similar feature is available in a project called [VaultOperator](https://github.com/finleap-connect/vaultoperator/) which tries to solve similar problems like this operator just for Hashicorp Vault as a secret store.
|
||
|
|
||
|
## Motivation
|
||
|
|
||
|
The motivation is to boost adoption of zero-trust secrets management
|
||
|
and to make it simple to create secure secrets for machine users
|
||
|
of services running inside clusters (i.e. databases etc).
|
||
|
|
||
|
### Goals
|
||
|
|
||
|
- enable zero-trust secret management for purely technical and internal users
|
||
|
- enable secure passwords by default
|
||
|
- reduce error rates when creating RSA or ECDSA private keys
|
||
|
|
||
|
### Non-Goals
|
||
|
|
||
|
- auto-rotation of secrets
|
||
|
|
||
|
## Proposal
|
||
|
|
||
|
Implement several generators to support not only a number of data types but also generators that interface with external APIs.
|
||
|
|
||
|
### API
|
||
|
|
||
|
We add a new resource group `generators.external-secrets.io`. For each generator we add a new CRD, e.g. for password generation, VaultPKI, ECR, GCR etc.
|
||
|
This custom resource contains all the configuration details to create a secure value.
|
||
|
It can be referenced from a `Kind=ExternalSecret` or `Kind=PushSecret`.
|
||
|
|
||
|
The controller generates the value and passes it onto the `Kind=Secret` with all the values
|
||
|
that have been fetched from the provider.
|
||
|
When used with a `Kind=PushSecret` it will generate the value and push it to the configured provider.
|
||
|
|
||
|
A generator generates a known set of outputs, e.g. `username` and `password`, `private_key` and `certificate` or others.
|
||
|
|
||
|
#### Example CRD
|
||
|
|
||
|
The following custom resource `Kind=VaultPKIBackend` contains all the parameters for issuing a certificate via HashiCorp Vault PKI engine.
|
||
|
|
||
|
```yaml
|
||
|
apiVersion: generators.external-secrets.io/v1alpha1
|
||
|
kind: VaultPKIBackend
|
||
|
metadata:
|
||
|
name: my-vault-pki-creds
|
||
|
spec:
|
||
|
# specific params for generating
|
||
|
commonName: example.com
|
||
|
altNames:
|
||
|
- example.org
|
||
|
- localhost
|
||
|
# ... more opts
|
||
|
auth: {} # service account etc. etc.
|
||
|
```
|
||
|
|
||
|
This resource can be **referenced** from an ExternalSecret or PushSecret.
|
||
|
The ExternalSecret controller generates the secret value and it can be post-processed with
|
||
|
existing features like template or key-rewrite.
|
||
|
|
||
|
```yaml
|
||
|
apiVersion: external-secrets.io/v1beta1
|
||
|
kind: ExternalSecret
|
||
|
metadata:
|
||
|
name: example
|
||
|
spec:
|
||
|
# ...
|
||
|
dataFrom:
|
||
|
|
||
|
# the generator is referenced like here
|
||
|
- sourceRef:
|
||
|
generatorRef:
|
||
|
apiVersion: generators.external-secrets.io/v1alpha1
|
||
|
kind: VaultPKIBackend
|
||
|
name: foobar
|
||
|
```
|
||
|
|
||
|
#### `SecretStoreRef` vs. `SourceRef`
|
||
|
|
||
|
In order to accomodate the generator implementation we need to enhance the `spec.secretStoreRef` functionality
|
||
|
and add `data[].sourceRef` and `dataFrom[].sourceRef` that allows a user to reference generator resources.
|
||
|
This allows us to:
|
||
|
|
||
|
- improve composability and **use multiple SecretStores** within a single ExternalSecret
|
||
|
- use generators from both `data[]` and `dataFrom[]` nodes
|
||
|
|
||
|
If `sourceRef` is not defined we fall back to `spec.secretStoreRef`.
|
||
|
|
||
|
```yaml
|
||
|
spec:
|
||
|
data:
|
||
|
- secretKey: foo
|
||
|
remoteRef: {}
|
||
|
# can point to a SecretStore
|
||
|
# or a generator/generatorRef
|
||
|
sourceRef:
|
||
|
storeRef:
|
||
|
name: foo
|
||
|
dataFrom:
|
||
|
# allow using multiple stores from a single ExternalSecret
|
||
|
- find: { ... }
|
||
|
sourceRef:
|
||
|
storeRef:
|
||
|
name: "foo"
|
||
|
- find: { ... }
|
||
|
sourceRef:
|
||
|
storeRef:
|
||
|
name: "bar"
|
||
|
```
|
||
|
|
||
|
Specifying more than one attribute within a sourceRef is not supported and must be rejected.
|
||
|
|
||
|
```
|
||
|
sourceRef:
|
||
|
storeRef: {}
|
||
|
generatorRef: {} # ILLEGAL!
|
||
|
```
|
||
|
|
||
|
<details>
|
||
|
<summary>PushSecret example</summary>
|
||
|
|
||
|
```yaml
|
||
|
apiVersion: external-secrets.io/v1alpha1
|
||
|
kind: PushSecret
|
||
|
metadata:
|
||
|
name: "hello-world"
|
||
|
spec:
|
||
|
secretStoreRefs: [] # ...
|
||
|
selector:
|
||
|
# we can consider supporting multiple generators per PushSecret
|
||
|
# but need to take care of key collision
|
||
|
generatorRef:
|
||
|
apiVersion: generators.external-secrets.io/v1alpha1
|
||
|
kind: VaultPKIBackend
|
||
|
name: foobar
|
||
|
data: {} # ...
|
||
|
```
|
||
|
|
||
|
</details>
|
||
|
|
||
|
<details>
|
||
|
<summary>Password Generator Example</summary>
|
||
|
|
||
|
```yaml
|
||
|
apiVersion: generators.external-secrets.io/v1alpha1
|
||
|
kind: Password
|
||
|
metadata:
|
||
|
name: foo
|
||
|
spec:
|
||
|
characterSet: "..." # TBD
|
||
|
length: 42
|
||
|
---
|
||
|
apiVersion: external-secrets.io/v1beta1
|
||
|
kind: ExternalSecret
|
||
|
metadata:
|
||
|
name: example
|
||
|
spec:
|
||
|
# ...
|
||
|
dataFrom:
|
||
|
- generatorRef:
|
||
|
apiVersion: generators.external-secrets.io/v1alpha1
|
||
|
kind: Password
|
||
|
name: foo
|
||
|
```
|
||
|
|
||
|
</details>
|
||
|
|
||
|
<details>
|
||
|
<summary>Example: multiple generators with key rewrite</summary>
|
||
|
|
||
|
```yaml
|
||
|
apiVersion: external-secrets.io/v1beta1
|
||
|
kind: ExternalSecret
|
||
|
metadata:
|
||
|
name: example
|
||
|
spec:
|
||
|
# ...
|
||
|
dataFrom:
|
||
|
# using multiple generators
|
||
|
# this will produce the following values:
|
||
|
# foo-username
|
||
|
# foo-password
|
||
|
# bar-username
|
||
|
# bar-password
|
||
|
- generatorRef:
|
||
|
apiVersion: generators.external-secrets.io/v1alpha1
|
||
|
kind: ECR
|
||
|
name: foo
|
||
|
rewrite:
|
||
|
- regexp:
|
||
|
source: .+
|
||
|
target: foo-$1
|
||
|
- generatorRef:
|
||
|
apiVersion: generators.external-secrets.io/v1alpha1
|
||
|
kind: ECR
|
||
|
name: bar
|
||
|
rewrite:
|
||
|
- regexp:
|
||
|
source: .+
|
||
|
target: bar-$1
|
||
|
```
|
||
|
|
||
|
</details>
|
||
|
|
||
|
### Composability and extensibility
|
||
|
|
||
|
We can maintain a set of in-tree generators (e.g. for password, binary, hash, alphanumeric, VaultPKI, VaultAWS, ECR, GCR, ACR, Harbor...) but that doesn't provide an extensible interface for users.
|
||
|
We can consider the following two approaches (possibly more) that provide extensibility:
|
||
|
|
||
|
> Note: This section is just an outlook and nothing that we aim to implement right away.
|
||
|
> We will re-iterate on this feature and see if we need to make the
|
||
|
> generator implementation more extensible.
|
||
|
|
||
|
#### (1) HTTP/gRPC interface
|
||
|
|
||
|
We can provide an in-tree generator that interfaces through a well-known protocol (e.g. HTTP webhook or gRPC).
|
||
|
|
||
|
```yaml
|
||
|
apiVersion: generators.external-secrets.io/v1alpha1
|
||
|
kind: HTTPWebhook
|
||
|
metadata:
|
||
|
name: example
|
||
|
spec:
|
||
|
# similar to webhook provider
|
||
|
# see: https://external-secrets.io/v0.5.8/provider-webhook/
|
||
|
url: "http://httpbin.org/get?parameter={{ .remoteRef.key }}"
|
||
|
result:
|
||
|
jsonPath: "$.args.parameter"
|
||
|
headers:
|
||
|
Content-Type: application/json
|
||
|
Authorization: Basic {{ print .auth.username ":" .auth.password | b64enc }}
|
||
|
secrets:
|
||
|
- name: auth
|
||
|
secretRef:
|
||
|
name: webhook-credentials
|
||
|
```
|
||
|
|
||
|
Pros:
|
||
|
|
||
|
- simple to extend from a user perspective
|
||
|
- simple simple to add from external-secrets perspective
|
||
|
|
||
|
Cons:
|
||
|
|
||
|
- prone to network errors
|
||
|
- high coupling of secret sync and generator
|
||
|
|
||
|
#### (2) with an intermediate GenerateRequest
|
||
|
|
||
|
This is very similar to [cert-manager's external issuer API](https://cert-manager.io/docs/contributing/external-issuers/) that allows users to implement their own controller to issue certifiates.
|
||
|
We maintain some core generator controllers and leave the implementation of others up to the user.
|
||
|
|
||
|
```
|
||
|
┌───────────────────┐ ┌──────────────────────┐
|
||
|
│ │ 1 │ │ 2 ┌───────────────┐
|
||
|
│ ExternalSecret/ │ create │ Kind:GenerateRequest │ generate │ │
|
||
|
│ PushSecret ├────────►│ generatorRef: │◄───────────┤ FooGenerator │
|
||
|
│ Controller │◄────────┤ kind: Foo │ │ Controller │
|
||
|
│ │ result │ name: bar │ │ │
|
||
|
└─────────┬─────────┘ 3 │ result: │ └───────────────┘
|
||
|
│ │ username: foo │
|
||
|
4 │ │ password: bar │
|
||
|
create │ │ │
|
||
|
│ └──────────────────────┘
|
||
|
│
|
||
|
▼
|
||
|
┌──────────────────┐
|
||
|
│ │
|
||
|
│ Kind: Secret │
|
||
|
│ data: │
|
||
|
│ username: foo │
|
||
|
│ password: bar │
|
||
|
│ │
|
||
|
└──────────────────┘
|
||
|
```
|
||
|
|
||
|
Pros:
|
||
|
|
||
|
- loose coupling of generator and secret sync workflows
|
||
|
|
||
|
Cons:
|
||
|
|
||
|
- very high complexity
|
||
|
- hard to extend from a user perspective
|
||
|
|
||
|
### Behavior
|
||
|
|
||
|
#### with ExternalSecret
|
||
|
|
||
|
Every time an ExternalSecret reconciles the generator will produce a new set of output values.
|
||
|
Hence, the secrets are rotated. The frequency can be controlled with `refreshInterval`.
|
||
|
With `refreshInterval=0` the Secret will be created exactly once and will not rotate.
|
||
|
|
||
|
#### with PushSecret
|
||
|
|
||
|
The generator produces an output once which is pushed to the provider. If a secret already exists in that place it can not be overridden.
|
||
|
|
||
|
---
|