feat: 1password add support for tags and configurable PushSecret vault (#4173)

2024-12-14 11:57:59 +00:00 · 2024-12-10 08:53:36 +01:00 · 2024-12-10 08:53:36 +01:00 · 34f526f134
commit 34f526f134
parent 867185fe4e
5 changed files with 204 additions and 14 deletions
--- a/docs/provider/1password-automation.md
+++ b/docs/provider/1password-automation.md
@ -3,9 +3,11 @@
 External Secrets Operator integrates with [1Password Secrets Automation](https://1password.com/products/secrets/) for secret management.

 ### Important note about this documentation
+
 _**The 1Password API calls the entries in vaults 'Items'. These docs use the same term.**_

 ### Behavior
+
 * How an Item is equated to an ExternalSecret:
    * `remoteRef.key` is equated to an Item's Title
    * `remoteRef.property` is equated to:
@ -28,6 +30,7 @@ _**The 1Password API calls the entries in vaults 'Items'. These docs use the sam
    * `find.tags` are not supported at this time.

 ### Prerequisites
+
 * 1Password requires running a 1Password Connect Server to which the API requests will be made.
    * External Secrets does not run this server. See [Deploy a Connect Server](#deploy-a-connect-server).
    * One Connect Server is needed per 1Password Automation Environment.
@ -35,6 +38,7 @@ _**The 1Password API calls the entries in vaults 'Items'. These docs use the sam
 * 1Password Connect Server version 1.5.6 or higher.

 ### Setup Authentication
+
 _Authentication requires a `1password-credentials.json` file provided to the Connect Server, and a related 'Access Token' for the client in this provider to authenticate to that Connect Server. Both of these are generated by 1Password._

 1. Setup an Automation Environment [at 1Password.com](https://support.1password.com/secrets-automation/), or [via the op CLI](https://github.com/1Password/connect/blob/a0a5f3d92e68497098d9314721335a7bb68a3b2d/README.md#create-server-and-access-token).
@ -58,6 +62,7 @@ _Authentication requires a `1password-credentials.json` file provided to the Con
 ```

 ### Deploy a Connect Server
+
 * Follow the remaining instructions in the [Quick Start guide](https://github.com/1Password/connect/blob/a0a5f3d92e68497098d9314721335a7bb68a3b2d/README.md#quick-start).
    * Deploy at minimum a Deployment and Service for a Connect Server, to go along with the Secret for the Server created in the [Setup Authentication section](#setup-authentication).
 * The Service's name will be referenced in SecretStores/ClusterSecretStores.
@ -65,15 +70,20 @@ _Authentication requires a `1password-credentials.json` file provided to the Con
 * Unencrypted secret values are passed over the connection between the Operator and the Connect Server. **Encrypting the connection is recommended.**

 ### Creating Compatible 1Password Items
+
 _Also see [examples below](#examples) for matching SecretStore and ExternalSecret specs._
+
 #### Manually (Password type)
+
 1. Click the plus button to create a new Password type Item.
 1. Change the title to what you want `remoteRef.key` to be.
 1. Set what you want `remoteRef.property` to be in the field sections where is says 'label', and values where it says 'new field'.
 1. Click the 'Save' button.

 ![create-password-screenshot](../pictures/screenshot_1password_create_password.png)
+
 #### Manually (Document type)
+
 * Click the plus button to create a new Document type Item.
 * Choose the file to upload and upload it.
 * Change the title to match `remoteRef.key`
@ -81,7 +91,9 @@ _Also see [examples below](#examples) for matching SecretStore and ExternalSecre
 * Click the 'Save' button.

 ![create-document-screenshot](../pictures/screenshot_1password_create_document.png)
+
 #### Scripting (Password type with op [CLI](https://developer.1password.com/docs/cli/v1/get-started/))
+
 * Create `file.json` with the following contents, swapping in your keys and values. Note: `section.name`'s and `section.title`'s values are ignored by the Operator, but cannot be empty for the `op` CLI
    ```json
       {
@ -126,10 +138,13 @@ _Also see [examples below](#examples) for matching SecretStore and ExternalSecre
      }
    ```
 * Run `op item create --template file.json`
+
 #### Scripting (Document type)
+
 * Unfortunately the `op` CLI doesn't seem to support uploading multiple files to the same Item, and the current Go lib has a [bug](https://github.com/1Password/connect-sdk-go/issues/45). `op` can be used to create a Document type Item with one file in it, but for now it's necessary to add multiple files to the same Document via the GUI.

 #### In-built field labeled `password` on Password type Items
+
 * TL;DR if you need a field labeled `password`, use the in-built one rather than the one in a fields Section.

 ![password-field-example](../pictures/screenshot_1password_password_field.png)
@ -139,6 +154,7 @@ _Also see [examples below](#examples) for matching SecretStore and ExternalSecre
 * The in-built `password` field is not otherwise special for the purposes of ExternalSecrets. It can be ignored when not in use.

 ### Examples
+
 Examples of using the `my-env-config` and `my-cert` Items [seen above](#manually-password-type).

 * Note: with this configuration a 1Password Item titled `my-env-config` is correlated to a ExternalSecret named `my-env-config` that results in a Kubernetes secret named `my-env-config`, all with matching names for the key/value pairs. This is a way to increase comprehensibility.
@ -153,10 +169,13 @@ Examples of using the `my-env-config` and `my-cert` Items [seen above](#manually
 ```

 ### Additional Notes
+
 #### General
+
 * It's intuitive to use Document type Items for Kubernetes secrets mounted as files, and Password type Items for ones that will be mounted as environment variables, but either can be used for either. It comes down to what's more convenient.

 #### Why no version history
+
 * 1Password only supports version history on their in-built `password` field. Therefore, implementing version history in this provider would require one Item in 1Password per `remoteRef` in an ExternalSecret. Additionally `remoteRef.property` would be pointless/unusable.
 * For example, a Kubernetes secret with 15 keys (say, used in `envFrom`,) would require 15 Items in the 1Password vault, instead of 15 Fields in 1 Item. This would quickly get untenable for more than a few secrets, because:
    * All Items would have to have unique names which means `secretKey` couldn't match the Item name the `remoteRef` is targeting.
@ -165,11 +184,13 @@ Examples of using the `my-env-config` and `my-cert` Items [seen above](#manually
 * To support new and old versions of a secret value at the same time, create a new Item in 1Password with the new value, and point some ExternalSecrets at a time to the new Item.

 #### Keeping misconfiguration from working
+
 * One instance of the ExternalSecrets Operator _can_ work with many Connect Server instances, but it may not be the best approach.
 * With one Operator instance per Connect Server instance, namespaces and RBAC can be used to improve security posture, and perhaps just as importantly, it's harder to misconfigure something and have it work (supply env A's secret values to env B for example.)
 * You can run as many 1Password Connect Servers as you need security boundaries to help protect against accidental misconfiguration.

 #### Patching ExternalSecrets with Kustomize
+
 * An overlay can provide a SecretStore specific to that overlay, and then use JSON6902 to patch all the ExternalSecrets coming from base to point to that SecretStore. Here's an example `overlays/staging/kustomization.yaml`:
    ```yaml
    ---
@ -189,3 +210,15 @@ Examples of using the `my-env-config` and `my-cert` Items [seen above](#manually
          path: /spec/secretStoreRef/name
          value: staging
    ```
+
+### Push Secret
+
+To push a secret from Kubernetes cluster and create it as a secret in 1Password, a `Kind=PushSecret` resource is needed.
+
+Updating the vault on an existing PushSecret is currently not supported. To update the vault, create a new PushSecret with the updated vault.
+
+```yaml
+{% include '1password-push-secret.yaml' %}
+```
+
+Then it will create an item in onepassword `op://staging/1pw-secret-name/password` equal to `my-secret`.
--- a/docs/snippets/1password-push-secret.yaml
+++ b/docs/snippets/1password-push-secret.yaml
@ -0,0 +1,32 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: source-secret
+stringData:
+  source-key: "my-secret"
+---
+apiVersion: external-secrets.io/v1alpha1
+kind: PushSecret
+metadata:
+  name: pushsecret-example # Customisable
+spec:
+  deletionPolicy: Delete
+  refreshInterval: 1h
+  secretStoreRefs:
+    - name: 1password
+      kind: ClusterSecretStore
+  selector:
+    secret:
+      name: source-secret # Source Kubernetes secret
+  data:
+    - match:
+        secretKey: source-key # Source Kubernetes secret key to be pushed
+        remoteRef:
+          remoteKey: 1pw-secret-name # 1Password item/secret name
+          property: password         # (Optional) 1Password field type, default password
+      metadata:
+        apiVersion: kubernetes.external-secrets.io/v1alpha1
+        kind: PushSecretMetadata
+        spec:
+          vault: staging            # Optional the vault the secret is going to be pushed to, defaults to the first defined vault in the (Cluster)SecretStore
+          tags: ["tag1", "tag2"]    # Optional metadata to be pushed with the secret
--- a/docs/snippets/1password-secret-store.yaml
+++ b/docs/snippets/1password-secret-store.yaml
@ -6,7 +6,7 @@ metadata:
 spec:
  provider:
    onepassword:
-      connectHost: https://onepassword-connect-staging
+      connectHost: https://onepassword-connect-staging:8080
      vaults:
        staging: 1  # look in this vault first
        shared: 2   # next look in here. error if not found
--- a/pkg/provider/onepassword/onepassword.go
+++ b/pkg/provider/onepassword/onepassword.go
@ -32,6 +32,7 @@ import (
 	esv1beta1 "github.com/external-secrets/external-secrets/apis/externalsecrets/v1beta1"
 	"github.com/external-secrets/external-secrets/pkg/find"
 	"github.com/external-secrets/external-secrets/pkg/utils"
+	"github.com/external-secrets/external-secrets/pkg/utils/metadata"
 	"github.com/external-secrets/external-secrets/pkg/utils/resolvers"
 )

@ -57,11 +58,12 @@ const (
 	errCreateItem            = "error creating 1Password Item: %w"
 	errDeleteItem            = "error deleting 1Password Item: %w"
 	// custom error messages.
-	errKeyNotFoundMsg       = "key not found in 1Password Vaults"
-	errNoVaultsMsg          = "no vaults found"
-	errExpectedOneItemMsg   = "expected one 1Password Item matching"
-	errExpectedOneFieldMsg  = "expected one 1Password ItemField matching"
-	errExpectedOneFieldMsgF = "%w: '%s' in '%s', got %d"
+	errKeyNotFoundMsg             = "key not found in 1Password Vaults"
+	errNoVaultsMsg                = "no vaults found"
+	errMetadataVaultNotinProvider = "metadata vault '%s' not in provider vaults"
+	errExpectedOneItemMsg         = "expected one 1Password Item matching"
+	errExpectedOneFieldMsg        = "expected one 1Password ItemField matching"
+	errExpectedOneFieldMsgF       = "%w: '%s' in '%s', got %d"

 	documentCategory = "DOCUMENT"
 	fieldPrefix      = "field"
@ -87,6 +89,11 @@ type ProviderOnePassword struct {
 	client connect.Client
 }

+type PushSecretMetadataSpec struct {
+	Tags  []string `json:"tags,omitempty"`
+	Vault string   `json:"vault,omitempty"`
+}
+
 // https://github.com/external-secrets/external-secrets/issues/644
 var (
 	_ esv1beta1.SecretsClient = &ProviderOnePassword{}
@ -222,18 +229,40 @@ const (

 // createItem creates a new item in the first vault. If no vaults exist, it returns an error.
 func (provider *ProviderOnePassword) createItem(val []byte, ref esv1beta1.PushSecretData) error {
-	// Get the first vault
-	sortedVaults := sortVaults(provider.vaults)
-	if len(sortedVaults) == 0 {
-		return ErrNoVaults
+	// Get the metadata
+	metadata, err := metadata.ParseMetadataParameters[PushSecretMetadataSpec](ref.GetMetadata())
+	if err != nil {
+		return fmt.Errorf("failed to parse push secret metadata: %w", err)
 	}
-	vaultID := sortedVaults[0]
+
+	// Check if there is a vault is specified in the metadata
+	vaultID := ""
+	if metadata != nil && metadata.Spec.Vault != "" {
+		// check if metadata.Spec.Vault is in provider.vaults
+		if _, ok := provider.vaults[metadata.Spec.Vault]; !ok {
+			return fmt.Errorf(errMetadataVaultNotinProvider, metadata.Spec.Vault)
+		}
+		vaultID = metadata.Spec.Vault
+	} else {
+		// Get the first vault from the provider
+		sortedVaults := sortVaults(provider.vaults)
+		if len(sortedVaults) == 0 {
+			return ErrNoVaults
+		}
+		vaultID = sortedVaults[0]
+	}
+
 	// Get the label
 	label := ref.GetProperty()
 	if label == "" {
 		label = passwordLabel
 	}

+	var tags []string
+	if metadata != nil && metadata.Spec.Tags != nil {
+		tags = metadata.Spec.Tags
+	}
+
 	// Create the item
 	item := &onepassword.Item{
 		Title:    ref.GetRemoteKey(),
@ -244,9 +273,10 @@ func (provider *ProviderOnePassword) createItem(val []byte, ref esv1beta1.PushSe
 		Fields: []*onepassword.ItemField{
 			generateNewItemField(label, string(val)),
 		},
+		Tags: tags,
 	}

-	_, err := provider.client.CreateItem(item, vaultID)
+	_, err = provider.client.CreateItem(item, vaultID)
 	return err
 }

@ -317,6 +347,14 @@ func (provider *ProviderOnePassword) PushSecret(ctx context.Context, secret *cor
 		label = passwordLabel
 	}

+	metadata, err := metadata.ParseMetadataParameters[PushSecretMetadataSpec](ref.GetMetadata())
+	if err != nil {
+		return fmt.Errorf("failed to parse push secret metadata: %w", err)
+	}
+	if metadata != nil && metadata.Spec.Tags != nil {
+		providerItem.Tags = metadata.Spec.Tags
+	}
+
 	providerItem.Fields, err = updateFieldValue(providerItem.Fields, label, string(val))
 	if err != nil {
 		return fmt.Errorf(errUpdateItem, err)
--- a/pkg/provider/onepassword/onepassword_test.go
+++ b/pkg/provider/onepassword/onepassword_test.go
@ -16,6 +16,7 @@ package onepassword

 import (
 	"context"
+	"encoding/json"
 	"errors"
 	"fmt"
 	"reflect"
@ -30,6 +31,7 @@ import (
 	esv1beta1 "github.com/external-secrets/external-secrets/apis/externalsecrets/v1beta1"
 	esmeta "github.com/external-secrets/external-secrets/apis/meta/v1"
 	"github.com/external-secrets/external-secrets/pkg/provider/onepassword/fake"
+	"github.com/external-secrets/external-secrets/pkg/utils/metadata"
 )

 const (
@ -1539,6 +1541,7 @@ type fakeRef struct {
 	key       string
 	prop      string
 	secretKey string
+	metadata  *apiextensionsv1.JSON
 }

 func (f fakeRef) GetRemoteKey() string {
@ -1554,7 +1557,7 @@ func (f fakeRef) GetSecretKey() string {
 }

 func (f fakeRef) GetMetadata() *apiextensionsv1.JSON {
-	return nil
+	return f.metadata
 }

 func validateItem(t *testing.T, expectedItem, actualItem *onepassword.Item) {
@ -1574,9 +1577,20 @@ func TestProviderOnePasswordCreateItem(t *testing.T) {
 		ref                esv1beta1.PushSecretData
 	}
 	const vaultName = "vault1"
+	const fallbackVaultName = "vault2"

 	thridPartyErr := errors.New("third party error")

+	metadata := &metadata.PushSecretMetadata[PushSecretMetadataSpec]{
+		APIVersion: metadata.APIVersion,
+		Kind:       metadata.Kind,
+		Spec: PushSecretMetadataSpec{
+			Tags:  []string{"tag1", "tag2"},
+			Vault: fallbackVaultName,
+		},
+	}
+	metadataRaw, _ := json.Marshal(metadata)
+
 	testCases := []testCase{
 		{
 			setupNote: "standard create",
@ -1587,7 +1601,8 @@ func TestProviderOnePasswordCreateItem(t *testing.T) {
 			},
 			expectedErr: nil,
 			vaults: map[string]int{
-				vaultName: 1,
+				vaultName:         1,
+				fallbackVaultName: 2,
 			},
 			createValidateFunc: func(t *testing.T, item *onepassword.Item, s string) (*onepassword.Item, error) {
 				validateItem(t, &onepassword.Item{
@ -1666,6 +1681,36 @@ func TestProviderOnePasswordCreateItem(t *testing.T) {
 				return nil, thridPartyErr
 			},
 		},
+		{
+			setupNote: "valid metadata overrides",
+			val:       []byte("testing"),
+			ref: fakeRef{
+				key:  "another",
+				prop: "property",
+				metadata: &apiextensionsv1.JSON{
+					Raw: metadataRaw,
+				},
+			},
+			vaults: map[string]int{
+				vaultName:         1,
+				fallbackVaultName: 2,
+			},
+			expectedErr: nil,
+			createValidateFunc: func(t *testing.T, item *onepassword.Item, s string) (*onepassword.Item, error) {
+				validateItem(t, &onepassword.Item{
+					Title:    "another",
+					Category: onepassword.Server,
+					Vault: onepassword.ItemVault{
+						ID: fallbackVaultName,
+					},
+					Fields: []*onepassword.ItemField{
+						generateNewItemField("property", "testing"),
+					},
+					Tags: []string{"tag1", "tag2"},
+				}, item)
+				return item, nil
+			},
+		},
 	}
 	provider := &ProviderOnePassword{}
 	for _, tc := range testCases {
@ -2050,6 +2095,16 @@ func TestProviderOnePasswordPushSecret(t *testing.T) {
 			ID: vaultName,
 		}
 	)
+
+	metadata := &metadata.PushSecretMetadata[PushSecretMetadataSpec]{
+		APIVersion: metadata.APIVersion,
+		Kind:       metadata.Kind,
+		Spec: PushSecretMetadataSpec{
+			Tags: []string{"tag1", "tag2"},
+		},
+	}
+	metadataRaw, _ := json.Marshal(metadata)
+
 	testCases := []testCase{
 		{
 			vaults: map[string]int{
@ -2198,6 +2253,38 @@ func TestProviderOnePasswordPushSecret(t *testing.T) {
 				},
 			},
 		},
+		{
+			setupNote:   "create item with metadata overwrites success",
+			expectedErr: nil,
+			val: &corev1.Secret{Data: map[string][]byte{
+				key1: []byte("testing"),
+			}},
+			ref: fakeRef{
+				key:       key1,
+				prop:      "prop",
+				secretKey: key1,
+				metadata: &apiextensionsv1.JSON{
+					Raw: metadataRaw,
+				},
+			},
+			vaults: map[string]int{
+				vaultName: 1,
+			},
+			createValidateFunc: func(item *onepassword.Item, s string) (*onepassword.Item, error) {
+				validateItem(t, &onepassword.Item{
+					Title:    key1,
+					Category: onepassword.Server,
+					Vault: onepassword.ItemVault{
+						ID: vaultName,
+					},
+					Fields: []*onepassword.ItemField{
+						generateNewItemField("prop", "testing"),
+					},
+					Tags: []string{"tag1", "tag2"},
+				}, item)
+				return item, nil
+			},
+		},
 	}
 	provider := &ProviderOnePassword{}
 	for _, tc := range testCases {