1
0
Fork 0
mirror of https://github.com/external-secrets/external-secrets.git synced 2024-12-15 17:51:01 +00:00
external-secrets/design/005-secret-generator-group.md

316 lines
9.6 KiB
Markdown
Raw Normal View History

```yaml
---
title: Secret Generators
version: v1alpha1
authors: Christian Hünig, Jan Steffen, Moritz Johner
creation-date: 2022-07-08
status: draft
---
```
# Secret Generators
## Table of Contents
<!-- toc -->
// autogen please
<!-- /toc -->
## Summary
There are use-cases where a secret could be generated by the operator instead of using pre-existing ones from the secret store. For example a MongoDB could be spun up using a password generated by the operator. This password is then used by an application in the cluster to access the database. No human ever needs to read that secret or have to specify it beforehand. A similar feature is available in a project called [VaultOperator](https://github.com/finleap-connect/vaultoperator/) which tries to solve similar problems like this operator just for Hashicorp Vault as a secret store.
## Motivation
The motivation is to boost adoption of zero-trust secrets management
and to make it simple to create secure secrets for machine users
of services running inside clusters (i.e. databases etc).
### Goals
- enable zero-trust secret management for purely technical and internal users
- enable secure passwords by default
- reduce error rates when creating RSA or ECDSA private keys
### Non-Goals
- auto-rotation of secrets
## Proposal
Implement several generators to support not only a number of data types but also generators that interface with external APIs.
### API
We add a new resource group `generators.external-secrets.io`. For each generator we add a new CRD, e.g. for password generation, VaultPKI, ECR, GCR etc.
This custom resource contains all the configuration details to create a secure value.
It can be referenced from a `Kind=ExternalSecret` or `Kind=PushSecret`.
The controller generates the value and passes it onto the `Kind=Secret` with all the values
that have been fetched from the provider.
When used with a `Kind=PushSecret` it will generate the value and push it to the configured provider.
A generator generates a known set of outputs, e.g. `username` and `password`, `private_key` and `certificate` or others.
#### Example CRD
The following custom resource `Kind=VaultPKIBackend` contains all the parameters for issuing a certificate via HashiCorp Vault PKI engine.
```yaml
apiVersion: generators.external-secrets.io/v1alpha1
kind: VaultPKIBackend
metadata:
name: my-vault-pki-creds
spec:
# specific params for generating
commonName: example.com
altNames:
- example.org
- localhost
# ... more opts
auth: {} # service account etc. etc.
```
This resource can be **referenced** from an ExternalSecret or PushSecret.
The ExternalSecret controller generates the secret value and it can be post-processed with
existing features like template or key-rewrite.
```yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: example
spec:
# ...
dataFrom:
# the generator is referenced like here
- sourceRef:
generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: VaultPKIBackend
name: foobar
```
#### `SecretStoreRef` vs. `SourceRef`
In order to accomodate the generator implementation we need to enhance the `spec.secretStoreRef` functionality
and add `data[].sourceRef` and `dataFrom[].sourceRef` that allows a user to reference generator resources.
This allows us to:
- improve composability and **use multiple SecretStores** within a single ExternalSecret
- use generators from both `data[]` and `dataFrom[]` nodes
If `sourceRef` is not defined we fall back to `spec.secretStoreRef`.
```yaml
spec:
data:
- secretKey: foo
remoteRef: {}
# can point to a SecretStore
# or a generator/generatorRef
sourceRef:
storeRef:
name: foo
dataFrom:
# allow using multiple stores from a single ExternalSecret
- find: { ... }
sourceRef:
storeRef:
name: "foo"
- find: { ... }
sourceRef:
storeRef:
name: "bar"
```
Specifying more than one attribute within a sourceRef is not supported and must be rejected.
```
sourceRef:
storeRef: {}
generatorRef: {} # ILLEGAL!
```
<details>
<summary>PushSecret example</summary>
```yaml
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
name: "hello-world"
spec:
secretStoreRefs: [] # ...
selector:
# we can consider supporting multiple generators per PushSecret
# but need to take care of key collision
generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: VaultPKIBackend
name: foobar
data: {} # ...
```
</details>
<details>
<summary>Password Generator Example</summary>
```yaml
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
metadata:
name: foo
spec:
characterSet: "..." # TBD
length: 42
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: example
spec:
# ...
dataFrom:
- generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
name: foo
```
</details>
<details>
<summary>Example: multiple generators with key rewrite</summary>
```yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: example
spec:
# ...
dataFrom:
# using multiple generators
# this will produce the following values:
# foo-username
# foo-password
# bar-username
# bar-password
- generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: ECR
name: foo
rewrite:
- regexp:
source: .+
target: foo-$1
- generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: ECR
name: bar
rewrite:
- regexp:
source: .+
target: bar-$1
```
</details>
### Composability and extensibility
We can maintain a set of in-tree generators (e.g. for password, binary, hash, alphanumeric, VaultPKI, VaultAWS, ECR, GCR, ACR, Harbor...) but that doesn't provide an extensible interface for users.
We can consider the following two approaches (possibly more) that provide extensibility:
> Note: This section is just an outlook and nothing that we aim to implement right away.
> We will re-iterate on this feature and see if we need to make the
> generator implementation more extensible.
#### (1) HTTP/gRPC interface
We can provide an in-tree generator that interfaces through a well-known protocol (e.g. HTTP webhook or gRPC).
```yaml
apiVersion: generators.external-secrets.io/v1alpha1
kind: HTTPWebhook
metadata:
name: example
spec:
# similar to webhook provider
# see: https://external-secrets.io/v0.5.8/provider-webhook/
url: "http://httpbin.org/get?parameter={{ .remoteRef.key }}"
result:
jsonPath: "$.args.parameter"
headers:
Content-Type: application/json
Authorization: Basic {{ print .auth.username ":" .auth.password | b64enc }}
secrets:
- name: auth
secretRef:
name: webhook-credentials
```
Pros:
- simple to extend from a user perspective
- simple simple to add from external-secrets perspective
Cons:
- prone to network errors
- high coupling of secret sync and generator
#### (2) with an intermediate GenerateRequest
This is very similar to [cert-manager's external issuer API](https://cert-manager.io/docs/contributing/external-issuers/) that allows users to implement their own controller to issue certifiates.
We maintain some core generator controllers and leave the implementation of others up to the user.
```
┌───────────────────┐ ┌──────────────────────┐
│ │ 1 │ │ 2 ┌───────────────┐
│ ExternalSecret/ │ create │ Kind:GenerateRequest │ generate │ │
│ PushSecret ├────────►│ generatorRef: │◄───────────┤ FooGenerator │
│ Controller │◄────────┤ kind: Foo │ │ Controller │
│ │ result │ name: bar │ │ │
└─────────┬─────────┘ 3 │ result: │ └───────────────┘
│ │ username: foo │
4 │ │ password: bar │
create │ │ │
│ └──────────────────────┘
┌──────────────────┐
│ │
│ Kind: Secret │
│ data: │
│ username: foo │
│ password: bar │
│ │
└──────────────────┘
```
Pros:
- loose coupling of generator and secret sync workflows
Cons:
- very high complexity
- hard to extend from a user perspective
### Behavior
#### with ExternalSecret
Every time an ExternalSecret reconciles the generator will produce a new set of output values.
Hence, the secrets are rotated. The frequency can be controlled with `refreshInterval`.
With `refreshInterval=0` the Secret will be created exactly once and will not rotate.
#### with PushSecret
The generator produces an output once which is pushed to the provider. If a secret already exists in that place it can not be overridden.
---