mirror of
https://github.com/external-secrets/external-secrets.git
synced 2024-12-14 11:57:59 +00:00
🗺️ Design for dataFrom key rewrite (#1188)
* design for dataFrom key rewrite Signed-off-by: Gustavo Carvalho <gustavo.carvalho@container-solutions.com> * Update design/004-datafrom-key-rewrite.md Co-authored-by: Moritz Johner <moolen@users.noreply.github.com> * Update design/004-datafrom-key-rewrite.md Co-authored-by: Moritz Johner <moolen@users.noreply.github.com> * Adding drawbacks and examples. Changing CRDs to have block Signed-off-by: Gustavo Carvalho <gustavo.carvalho@container-solutions.com> Co-authored-by: Moritz Johner <moolen@users.noreply.github.com>
This commit is contained in:
Gustavo Fernandes de Carvalho
GitHub
parent
7eff8db532
commit
4db9017d2e
3 changed files with 172 additions and 0 deletions
172
design/004-datafrom-key-rewrite.md
Normal file
172
design/004-datafrom-key-rewrite.md
Normal file
|
|
@ -0,0 +1,172 @@
|
|||
```yaml
|
||||
---
|
||||
title: dataFrom key rewrite
|
||||
version: v1alpha1
|
||||
authors:
|
||||
creation-date: 2022-05-25
|
||||
status: draft
|
||||
---
|
||||
```
|
||||
|
||||
# Datafrom Key Rewrite
|
||||
|
||||
## Table of Contents
|
||||
|
||||
<!-- toc -->
|
||||
// autogen please
|
||||
<!-- /toc -->
|
||||
|
||||
|
||||
## Summary
|
||||
DataFrom key rewrite aims to give users the possibility to rewrite secret keys before manipulating into a template. This feature is specially useful with the addition of `dataFrom.find` introduced in v0.5.0, as a way to allow users to rewrite secret keys in batches, without knowing the full content of the key.
|
||||
|
||||
## Motivation
|
||||
It is hard to administer key creation without having weird names that convolutes with the path. It is natural to have a feature to allow secret Keys manipulation both from the lifecycle of the secret management and as part of the templating feature.
|
||||
|
||||
### Goals
|
||||
- CRD Design for dataFrom key rewrite
|
||||
|
||||
### Non-Goals
|
||||
Do not implement secret values rewrite - this should be done with templating - nor handle initial secret value conversion (see [#920](https://github.com/external-secrets/external-secrets/issues/920)).
|
||||
|
||||
### Terminology
|
||||
- Secret Key: the kubernetes secret key, aka the `string` in `map[string][]byte` in the codebase used to render Kubernetes Secrets.
|
||||
|
||||
## Proposal
|
||||
|
||||
Add a rewrite step after applying either `GetSecretMap` or `GetAllSecrets`, but before applying any conversion logic. This allows us to apply individual mappings to different `dataFrom` groups so they can be composed well together in combination with `data` (see example below).
|
||||
|
||||
### User Stories
|
||||
1. As an user I want to be able to remove paths from my Secret Keys
|
||||
2. As an user I want to be able to customize secret keys to have more meaningful value to the tenants consuming it.
|
||||
3. As an user I want external-secrets to tell me if there is any rewrite conflict and not to overwrite secrets information.
|
||||
|
||||
### API
|
||||
Proposed CRD changes:
|
||||
|
||||
```yaml
|
||||
apiVersion: external-secrets.io/v1beta1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: sample
|
||||
namespace: default
|
||||
spec:
|
||||
refreshInterval: 1m
|
||||
target:
|
||||
name: foobar
|
||||
secretStoreRef:
|
||||
name: secretstore-sample
|
||||
kind: SecretStore
|
||||
dataFrom:
|
||||
- extract:
|
||||
key: /foo/bar
|
||||
rewrite:
|
||||
- regexp:
|
||||
source: my-known-key
|
||||
target: new_key_name
|
||||
- regexp:
|
||||
source: my-unknown-(.*)
|
||||
target: unknown-$1
|
||||
- find:
|
||||
conversionStrategy: Unicode
|
||||
path: "my/path/"
|
||||
name:
|
||||
regexp: "my-pattern"
|
||||
rewrite:
|
||||
- regexp:
|
||||
source: my/path/(.*)
|
||||
target: $1
|
||||
- regexp:
|
||||
source: creds/(db|system)/(.*)
|
||||
target: $2-$1
|
||||
- find:
|
||||
conversionStrategy: Default
|
||||
path: "my/path/"
|
||||
tags:
|
||||
env: dev
|
||||
rewrite:
|
||||
- regexp:
|
||||
source: my/path/(.*)
|
||||
target: dev-$1
|
||||
|
||||
```
|
||||
|
||||
### Behavior
|
||||
After applying `GetAllSecrets` or `GetSecretMap`, a rewrite logic is applied by using regular expressions capture groups. This will convert the Secret Keys according to a given standard, possibly even up to a well known key for the `GetSecretMap`. After the key rewrite is done, the `ConversionStrategy` should be applied, and then the new collection of Secret Keys (`map[string][]byte`) can be rendered to template.
|
||||
|
||||
#### Examples
|
||||
|
||||
* adding a prefix/suffix - will transform `my-secret` in `my-preffix-my-secret-my-suffix`
|
||||
```
|
||||
rewrite:
|
||||
- regexp:
|
||||
source: (.*)
|
||||
target: my-preffix-$1-my-suffix
|
||||
```
|
||||
* removing a prefix/suffix - will transform `my-preffix-my-secret-my-suffix` into `my-secret`
|
||||
```
|
||||
rewrite:
|
||||
- regexp:
|
||||
source: my-preffix-(.*)-my-suffix
|
||||
target: $1
|
||||
```
|
||||
* Replacing characters - will transform `my/path/reader_db.creds-webapp` into `my-path-reader-db-creds-webapp`
|
||||
```
|
||||
rewrite:
|
||||
- regexp:
|
||||
source: \.|-|_|/ #standardizing separation with '-'
|
||||
target: -
|
||||
|
||||
```
|
||||
|
||||
* Removing path from the key name - will transform `my/path/reader-db-creds-webapp` into `reader-db-creds-webapp`
|
||||
```
|
||||
- find:
|
||||
conversionStrategy: Default
|
||||
path: "my/path/"
|
||||
tags:
|
||||
env: dev
|
||||
rewrite:
|
||||
- regexp:
|
||||
source: my/path/(.*)
|
||||
target: $1
|
||||
###
|
||||
```
|
||||
|
||||
* Combining operations - will transform `my/path/reader-db-creds-webapp` into `db-creds-reader`
|
||||
```
|
||||
- find:
|
||||
conversionStrategy: Default
|
||||
path: "my/path/"
|
||||
tags:
|
||||
env: dev
|
||||
rewrite:
|
||||
- regexp:
|
||||
source: my/path/(?P<secret>.*)
|
||||
target: $secret
|
||||
- regexp:
|
||||
source: (?U)(?P<role>.*)-(?P<app>.*)-webapp
|
||||
target: $app-$role
|
||||
```
|
||||
|
||||
|
||||
### Drawbacks
|
||||
|
||||
It would not be trivial to replace a specific set of existing characters for new characters with this strategy. This could be implemented with a `replace` method afterwards.
|
||||
|
||||
There are also some known limitations to golang regexp library, which implementes RE2. Some of the known limitations include:
|
||||
* Lack of ability to do lookaheads or lookbehinds
|
||||
* Lack of negation expressions
|
||||
* Lack of support to conditionl branches.
|
||||
* Lack of support to possessive repetitions.
|
||||
|
||||
A list of compatibility and known limitations considering other commonly used regexp frameworks (such as PCRE and PERL) are listed here https://github.com/google/re2/wiki/Syntax
|
||||
|
||||
### Acceptance Criteria
|
||||
+ If multiple keys have the same name an error should happen to the user referencing the original Secret Keys.
|
||||
+ A rewrite respects the regular expression limits
|
||||
+ A rewrite is compatible with `dataFrom.extract` and `dataFrom.find`
|
||||
+ It should be possible to remove paths with the `rewrite` operation.
|
||||
|
||||
## Alternatives
|
||||
Add provider-specific ConversionStrategies, which would be painfully hard to maintain.
|
||||
Loading…
Reference in a new issue