mirrors/kyverno
mirrors/kyverno
1
0
Fork 0
mirror of https://github.com/kyverno/kyverno.git synced 2025-03-31 03:45:17 +00:00
Code Activity

Merge branch 'master' of github.com:nirmata/kyverno into fix-875

Browse source 
* 'master' of github.com:nirmata/kyverno: (201 commits)
  update tag v1.1.6-rc2 (#887)
  Bugfix/update docs for deny rules (#886)
  message fix
  corrected message in generating resources in readme
  Bugfix/fix install repo (#885)
  Bugfix/1.1.6 adjust resync and cleanup unused (#884)
  middelware removed (#882)
  fix typo
  fix namespace lister
  Added readiness and liveness prob  (#874)
  set ownerReference in pv update
  suppress violation on evicted pod
  - fix violations re-create on the same resource - skip background processing if a resource is to be deleted
  suppress violation on evicted pod
  - fix violations re-create on the same resource - skip background processing if a resource is to be deleted
  Bugfix/659 support wildcards for namespaces (#871)
  resource limit added to kyverno defination and kyvrno chart
  update cpu  request limit and remove cpu limit
  remove roles
  remove unused code
  ...
This commit is contained in:
Yuvraj 2020-05-28 13:08:56 -07:00
commit db419c61b8
173 changed files with 6890 additions and 3410 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
.codeclimate.yml Normal file
View file

@ -0,0 +1,18 @@
engines:
govet:
enabled: true
golint:
enabled: false
gofmt:
enabled: true
ratings:
paths:
- "**.go"
exclude_paths:
- documentation/
- definitions
- gh-pages
- samples
- scripts

3
.gitignore vendored
View file

@ -5,3 +5,6 @@ gh-pages/public
_output
coverage.txt
.idea
cmd/initContainer/kyvernopre
cmd/kyverno/kyverno
cmd/cli/kubectl-kyverno/kyverno

17
.golangci.yml Normal file
View file

@ -0,0 +1,17 @@
linters:
enable:
- gosec
- errcheck
- gosimple
- bodyclose
- staticcheck
disable:
- ineffassign
- deadcode
- unused
- structcheck
run:
skip-files:
- ".+_test.go"
- ".+_test_.+.go"

5
.travis.yml
View file

@ -2,6 +2,11 @@ language: go
go:
- "1.13"
cache:
directories:
- $HOME/.cache/go-build
- $GOPATH/pkg/mod
# safelist
branches:
only:

8
CONTRIBUTING.md
View file

@ -1 +1,7 @@
See: https://github.com/nirmata/kyverno#contributing
# Contributing to Kyverno
## Code Style
We follow the community provided standard [code structure](https://github.com/golang-standards/project-layout).
See : https://github.com/nirmata/kyverno#contributing

9
Makefile
View file

@ -8,9 +8,9 @@ GIT_BRANCH := $(shell git branch | grep \* | cut -d ' ' -f2)
GIT_HASH := $(GIT_BRANCH)/$(shell git log -1 --pretty=format:"%H")
TIMESTAMP := $(shell date '+%Y-%m-%d_%I:%M:%S%p')
REGISTRY=index.docker.io
REGISTRY?=index.docker.io
REPO=$(REGISTRY)/nirmata/kyverno
IMAGE_TAG=$(GIT_VERSION)
IMAGE_TAG?=$(GIT_VERSION)
GOOS ?= $(shell go env GOOS)
PACKAGE ?=github.com/nirmata/kyverno
LD_FLAGS="-s -w -X $(PACKAGE)/pkg/version.BuildVersion=$(GIT_VERSION) -X $(PACKAGE)/pkg/version.BuildHash=$(GIT_HASH) -X $(PACKAGE)/pkg/version.BuildTime=$(TIMESTAMP)"
@ -53,6 +53,11 @@ docker-push-initContainer:
.PHONY: docker-build-kyverno docker-tag-repo-kyverno docker-push-kyverno
KYVERNO_PATH := cmd/kyverno
KYVERNO_IMAGE := kyverno
local:
go build -ldflags=$(LD_FLAGS) $(PWD)/$(KYVERNO_PATH)
go build -ldflags=$(LD_FLAGS) $(PWD)/$(CLI_PATH)
kyverno:
GOOS=$(GOOS) go build -o $(PWD)/$(KYVERNO_PATH)/kyverno -ldflags=$(LD_FLAGS) $(PWD)/$(KYVERNO_PATH)/main.go

193
README.md
View file

@ -6,7 +6,7 @@
Kyverno is a policy engine designed for Kubernetes.
Kubernetes supports declarative validation, mutation, and generation of resource configurations using policies written as Kubernetes resources.
Kyverno supports declarative validation, mutation, and generation of resource configurations using policies written as Kubernetes resources.
Kyverno can be used to scan existing workloads for best practices, or can be used to enforce best practices by blocking or mutating API requests.Kyverno allows cluster adminstrators to manage environment specific configurations independently of workload configurations and enforce configuration best practices for their clusters.
@ -26,7 +26,7 @@ Policy enforcement is captured using Kubernetes events. Kyverno also reports pol
This policy requires that all pods have CPU and memory resource requests and limits:
````yaml
```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
@ -35,108 +35,145 @@ spec:
# `enforce` blocks the request. `audit` reports violations
validationFailureAction: enforce
rules:
- name: check-pod-resources
match:
resources:
kinds:
- Pod
validate:
message: "CPU and memory resource requests and limits are required"
pattern:
spec:
containers:
# 'name: *' selects all containers in the pod
- name: "*"
resources:
limits:
# '?' requires 1 alphanumeric character and '*' means that there can be 0 or more characters.
# Using them together e.g. '?*' requires at least one character.
memory: "?*"
cpu: "?*"
requests:
memory: "?*"
cpu: "?*"
````
- name: check-pod-resources
match:
resources:
kinds:
- Pod
validate:
message: "CPU and memory resource requests and limits are required"
pattern:
spec:
containers:
# 'name: *' selects all containers in the pod
- name: "*"
resources:
limits:
# '?' requires 1 alphanumeric character and '*' means that
# there can be 0 or more characters. Using them together
# e.g. '?*' requires at least one character.
memory: "?*"
cpu: "?*"
requests:
memory: "?*"
cpu: "?*"
```
This policy prevents users from changing default network policies:
```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: deny-netpol-changes
spec:
validationFailureAction: enforce
background: false
rules:
- name: check-netpol-updates
match:
resources:
kinds:
- NetworkPolicy
name:
- *-default
exclude:
clusterRoles:
- cluster-admin
validate:
message: "Changing default network policies is not allowed"
deny: {}
```
### 2. Mutating resources
This policy sets the imagePullPolicy to Always if the image tag is latest:
````yaml
```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: set-image-pull-policy
spec:
rules:
- name: set-image-pull-policy
match:
resources:
kinds:
- Pod
mutate:
overlay:
spec:
containers:
# match images which end with :latest
- (image): "*:latest"
# set the imagePullPolicy to "Always"
imagePullPolicy: "Always"
````
- name: set-image-pull-policy
match:
resources:
kinds:
- Pod
mutate:
overlay:
spec:
containers:
# match images which end with :latest
- (image): "*:latest"
# set the imagePullPolicy to "Always"
imagePullPolicy: "Always"
```
### 3. Generating resources
This policy sets the Zookeeper and Kafka connection strings for all namespaces with a label key 'kafka'.
This policy sets the Zookeeper and Kafka connection strings for all namespaces.
````yaml
```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: "zk-kafka-address"
spec:
rules:
- name: "zk-kafka-address"
match:
resources:
kinds:
- Namespace
generate:
kind: ConfigMap
name: zk-kafka-address
# generate the resource in the new namespace
namespace: "{{request.object.metadata.name}}"
data:
- name: "zk-kafka-address"
match:
resources:
kinds:
- Namespace
generate:
kind: ConfigMap
name: zk-kafka-address
# generate the resource in the new namespace
namespace: "{{request.object.metadata.name}}"
data:
ZK_ADDRESS: "192.168.10.10:2181,192.168.10.11:2181,192.168.10.12:2181"
KAFKA_ADDRESS: "192.168.10.13:9092,192.168.10.14:9092,192.168.10.15:9092"
````
kind: ConfigMap
data:
ZK_ADDRESS: "192.168.10.10:2181,192.168.10.11:2181,192.168.10.12:2181"
KAFKA_ADDRESS: "192.168.10.13:9092,192.168.10.14:9092,192.168.10.15:9092"
```
### 4. More examples
Refer to a list of curated of ***[sample policies](/samples/README.md)*** that can be applied to your cluster.
**For more examples, refer to a list of curated of **_[sample policies](/samples/README.md)_** that can be applied to your cluster.**
## Documentation
* [Getting Started](documentation/installation.md)
* [Writing Policies](documentation/writing-policies.md)
* [Validate Resources](documentation/writing-policies-validate.md)
* [Mutate Resources](documentation/writing-policies-mutate.md)
* [Generate Resources](documentation/writing-policies-generate.md)
* [Variable Substitution](documentation/writing-policies-variables.md)
* [Preconditions](documentation/writing-policies-preconditions.md)
* [Auto-Generation of Pod Controller Policies](documentation/writing-policies-autogen.md)
* [Background Processing](documentation/writing-policies-background.md)
* [Testing Policies](documentation/testing-policies.md)
* [Policy Violations](documentation/policy-violations.md)
* [Kyverno CLI](documentation/kyverno-cli.md)
* [Sample Policies](/samples/README.md)
- [Getting Started](documentation/installation.md)
- [Writing Policies](documentation/writing-policies.md)
- [Selecting Resources](/documentation/writing-policies-match-exclude.md)
- [Validating Resources](documentation/writing-policies-validate.md)
- [Mutating Resources](documentation/writing-policies-mutate.md)
- [Generating Resources](documentation/writing-policies-generate.md)
- [Variable Substitution](documentation/writing-policies-variables.md)
- [Preconditions](documentation/writing-policies-preconditions.md)
- [Auto-Generation of Pod Controller Policies](documentation/writing-policies-autogen.md)
- [Background Processing](documentation/writing-policies-background.md)
- [Testing Policies](documentation/testing-policies.md)
- [Policy Violations](documentation/policy-violations.md)
- [Kyverno CLI](documentation/kyverno-cli.md)
- [Sample Policies](/samples/README.md)
## Presentations and Articles
- [Introducing Kyverno - blog post](https://nirmata.com/2019/07/11/managing-kubernetes-configuration-with-policies/)
- [CNCF Video and Slides](https://www.cncf.io/webinars/how-to-keep-your-clusters-safe-and-healthy/)
- [10 Kubernetes Best Practices - blog post](https://thenewstack.io/10-kubernetes-best-practices-you-can-easily-apply-to-your-clusters/)
- [VMware Code Meetup Video](https://www.youtube.com/watch?v=mgEmTvLytb0)
- [Virtual Rejekts Video](https://www.youtube.com/watch?v=caFMtSg4A6I)
- [TGIK Video](https://www.youtube.com/watch?v=ZE4Zu9WQET4&list=PL7bmigfV0EqQzxcNpmcdTJ9eFRPBe-iZa&index=18&t=0s)
## License
[Apache License 2.0](https://github.com/nirmata/kyverno/blob/master/LICENSE)
## Alternatives
### Open Policy Agent
@ -155,20 +192,20 @@ Refer to a list of curated of ***[sample policies](/samples/README.md)*** that c
Tools like [Kustomize](https://github.com/kubernetes-sigs/kustomize) can be used to manage variations in configurations outside of clusters. There are several advantages to this approach when used to produce variations of the same base configuration. However, such solutions cannot be used to validate or enforce configurations.
## Roadmap
See [Milestones](https://github.com/nirmata/kyverno/milestones) and [Issues](https://github.com/nirmata/kyverno/issues).
## Getting help
* For feature requests and bugs, file an [issue](https://github.com/nirmata/kyverno/issues).
* For discussions or questions, join our [Kubernetes Slack channel #kyverno](https://app.slack.com/client/T09NY5SBT/CLGR9BJU9) or the [mailing list](https://groups.google.com/forum/#!forum/kyverno)
- For feature requests and bugs, file an [issue](https://github.com/nirmata/kyverno/issues).
- For discussions or questions, join the **#kyverno** channel on the [Kubernetes Slack](https://kubernetes.slack.com/) or the [mailing list](https://groups.google.com/forum/#!forum/kyverno)
## Contributing
Thanks for your interest in contributing!
* Please review and agree to abide with the [Code of Conduct](/CODE_OF_CONDUCT.md) before contributing.
* See the [Wiki](https://github.com/nirmata/kyverno/wiki) for developer documentation.
* Browse through the [open issues](https://github.com/nirmata/kyverno/issues)
- Please review and agree to abide with the [Code of Conduct](/CODE_OF_CONDUCT.md) before contributing.
- We encourage all contributions and encourage you to read our [contribution guidelines](./CONTRIBUTING.md).
- See the [Wiki](https://github.com/nirmata/kyverno/wiki) for developer documentation.
- Browse through the [open issues](https://github.com/nirmata/kyverno/issues)

0
data/swaggerDoc.go → api/swaggerDoc.go
View file

19
charts/kyverno/Chart.yaml Normal file
View file

@ -0,0 +1,19 @@
apiVersion: v2
name: kyverno
version: 0.0.1
appVersion: v1.1.5
description: Kubernetes Native Policy Management
keywords:
- kubernetes
- nirmata
- policy agent
- validating webhook
- admissions controller
home: https://kyverno.io/
sources:
- https://github.com/nirmata/kyverno
maintainers:
- name: Nirmata
url: https://kyverno.io/
engine: gotpl
kubeVersion: ">=1.10.0-0"

104
charts/kyverno/README.md Normal file
View file

@ -0,0 +1,104 @@
# kyverno
[Kyverno](https://kyverno.io) is a Kubernetes Native Policy Management engine. It allows you to
* Manage policies as Kubernetes resources.
* Validate, mutate, and generate configurations.
* Select resources based on labels and wildcards.
* View policy enforcement as events.
* Detect policy violations for existing resources.
## TL;DR;
```console
$ helm install --create-namespace -n kyverno kyverno ./charts/kyverno
```
## Introduction
This chart bootstraps a Kyverno deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.
## Installing the Chart
Kyverno makes assumptions about naming of namespaces and resources. Therefore, the chart must be installed with the default release name `kyverno` (default if --name is omitted) and in the namespace 'kyverno':
```console
$ helm install --namespace kyverno kyverno ./charts/kyverno
```
Note that Helm by default expects the namespace to already exist before running helm install. If you want Helm to create the namespace, add --create-namespace to the command.
The command deploys kyverno on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
## Uninstalling the Chart
To uninstall/delete the `kyverno` deployment:
```console
$ helm delete -n kyverno kyverno
```
The command removes all the Kubernetes components associated with the chart and deletes the release.
## Configuration
The following table lists the configurable parameters of the kyverno chart and their default values.
Parameter | Description | Default
--- | --- | ---
`affinity` | node/pod affinities | `nil`
`createSelfSignedCert` | generate a self signed cert and certificate authority. Kyverno defaults to using kube-controller-manager CA-signed certificate or existing cert secret if false. | `false`
`config.existingConfig` | existing Kubernetes configmap to use for the resource filters configuration | `nil`
`config.resourceFilters` | list of filter of resource types to be skipped by kyverno policy engine. See [documentation](https://github.com/nirmata/kyverno/blob/master/documentation/installation.md#filter-kubernetes-resources-that-admission-webhook-should-not-process) for details | `["[Event,*,*]","[*,kube-system,*]","[*,kube-public,*]","[*,kube-node-lease,*]","[Node,*,*]","[APIService,*,*]","[TokenReview,*,*]","[SubjectAccessReview,*,*]","[*,kyverno,*]"]`
`extraArgs` | list of extra arguments to give the binary | `[]`
`fullnameOverride` | override the expanded name of the chart | `nil`
`generatecontrollerExtraResources` | extra resource type Kyverno is allowed to generate | `[]`
`image.pullPolicy` | Image pull policy | `IfNotPresent`
`image.pullSecrets` | Specify image pull secrets | `[]` (does not add image pull secrets to deployed pods)
`image.repository` | Image repository | `nirmata/kyverno`
`image.tag` | Image tag | `nil`
`initImage.pullPolicy` | Init image pull policy | `nil`
`initImage.repository` | Init image repository | `nirmata/kyvernopre`
`initImage.tag` | Init image tag | `nil`
`livenessProbe` | liveness probe configuration | `{}`
`nameOverride` | override the name of the chart | `nil`
`nodeSelector` | node labels for pod assignment | `{}`
`podAnnotations` | annotations to add to each pod | `{}`
`podLabels` | additional labels to add to each pod | `{}`
`podSecurityContext` | security context for the pod | `{}`
`priorityClassName` | priorityClassName | `nil`
`rbac.create` | create cluster roles, cluster role bindings, and service account | `true`
`rbac.serviceAccount.create` | create a service account | `true`
`rbac.serviceAccount.name` | the service account name | `nil`
`rbac.serviceAccount.annotations` | annotations for the service account | `{}`
`readinessProbe` | readiness probe configuration | `{}`
`replicaCount` | desired number of pods | `1`
`resources` | pod resource requests & limits | `{}`
`service.annotations` | annotations to add to the service | `{}`
`service.nodePort` | node port | `nil`
`service.port` | port for the service | `443`
`service.type` | type of service | `ClusterIP`
`tolerations` | list of node taints to tolerate | `[]`
`securityContext` | security context configuration | `{}`
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
```console
$ helm install --namespace kyverno kyverno ./charts/kyverno \
--set=image.tag=v0.0.2,resources.limits.cpu=200m
```
Alternatively, a YAML file that specifies the values for the above parameters can be provided while installing the chart. For example,
```console
$ helm install --namespace kyverno kyverno ./charts/kyverno -f values.yaml
```
> **Tip**: You can use the default [values.yaml](values.yaml)
## TLS Configuration
If `createSelfSignedCert` is `true`, Helm will take care of the steps of creating an external self-signed certificate describe in option 2 of the [installation documentation](https://github.com/nirmata/kyverno/blob/master/documentation/installation.md#option-2-use-your-own-ca-signed-certificate)
If `createSelfSignedCert` is `false`, Kyverno will generate a pair using the kube-controller-manager., or you can provide your own TLS CA and signed-key pair and create the secret yourself as described in the documentation.

468
charts/kyverno/crds/crds.yaml Normal file
View file

@ -0,0 +1,468 @@
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: clusterpolicies.kyverno.io
spec:
group: kyverno.io
versions:
- name: v1
served: true
storage: true
scope: Cluster
names:
kind: ClusterPolicy
plural: clusterpolicies
singular: clusterpolicy
shortNames:
- cpol
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
status: {}
spec:
required:
- rules
properties:
# default values to be handled by user
validationFailureAction:
type: string
enum:
- enforce # blocks the resorce api-reques if a rule fails.
- audit # allows resource creation and reports the failed validation rules as violations. Default
background:
type: boolean
rules:
type: array
items:
type: object
required:
- name
- match
properties:
name:
type: string
match:
type: object
required:
- resources
properties:
roles:
type: array
items:
type: string
clusterRoles:
type: array
items:
type: string
subjects:
type: array
items:
type: object
required:
- kind
- name
properties:
kind:
type: string
apiGroup:
type: string
name:
type: string
namespace:
type: string
resources:
type: object
minProperties: 1
properties:
kinds:
type: array
items:
type: string
name:
type: string
namespaces:
type: array
items:
type: string
selector:
properties:
matchLabels:
type: object
additionalProperties:
type: string
matchExpressions:
type: array
items:
type: object
required:
- key
- operator
properties:
key:
type: string
operator:
type: string
values:
type: array
items:
type: string
exclude:
type: object
properties:
roles:
type: array
items:
type: string
clusterRoles:
type: array
items:
type: string
subjects:
type: array
items:
type: object
required:
- kind
- name
properties:
kind:
type: string
apiGroup:
type: string
name:
type: string
namespace:
type: string
resources:
type: object
properties:
kinds:
type: array
items:
type: string
name:
type: string
namespaces:
type: array
items:
type: string
selector:
properties:
matchLabels:
type: object
additionalProperties:
type: string
matchExpressions:
type: array
items:
type: object
required:
- key
- operator
properties:
key:
type: string
operator:
type: string
values:
type: array
items:
type: string
preconditions:
type: array
items:
type: object
required:
- key # can be of any type
- operator # typed
- value # can be of any type
mutate:
type: object
properties:
overlay:
AnyValue: {}
patches:
type: array
items:
type: object
required:
- path
- op
properties:
path:
type: string
op:
type: string
enum:
- add
- replace
- remove
value:
AnyValue: {}
validate:
type: object
properties:
message:
type: string
pattern:
AnyValue: {}
anyPattern:
AnyValue: {}
deny:
properties:
conditions:
type: array
items:
type: object
required:
- key # can be of any type
- operator # typed
- value # can be of any type
properties:
operator:
type: string
enum:
- Equal
- Equals
- NotEqual
- NotEquals
key:
type: string
value:
type: string
generate:
type: object
required:
- kind
- name
properties:
kind:
type: string
name:
type: string
namespace:
type: string
clone:
type: object
required:
- namespace
- name
properties:
namespace:
type: string
name:
type: string
data:
AnyValue: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: clusterpolicyviolations.kyverno.io
spec:
group: kyverno.io
versions:
- name: v1
served: true
storage: true
scope: Cluster
names:
kind: ClusterPolicyViolation
plural: clusterpolicyviolations
singular: clusterpolicyviolation
shortNames:
- cpolv
subresources:
status: {}
additionalPrinterColumns:
- name: Policy
type: string
description: The policy that resulted in the violation
JSONPath: .spec.policy
- name: ResourceKind
type: string
description: The resource kind that cause the violation
JSONPath: .spec.resource.kind
- name: ResourceName
type: string
description: The resource name that caused the violation
JSONPath: .spec.resource.name
- name: Age
type: date
JSONPath: .metadata.creationTimestamp
validation:
openAPIV3Schema:
properties:
spec:
required:
- policy
- resource
- rules
properties:
policy:
type: string
resource:
type: object
required:
- kind
- name
properties:
kind:
type: string
name:
type: string
rules:
type: array
items:
type: object
required:
- name
- type
- message
properties:
name:
type: string
type:
type: string
message:
type: string
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: policyviolations.kyverno.io
spec:
group: kyverno.io
versions:
- name: v1
served: true
storage: true
scope: Namespaced
names:
kind: PolicyViolation
plural: policyviolations
singular: policyviolation
shortNames:
- polv
subresources:
status: {}
additionalPrinterColumns:
- name: Policy
type: string
description: The policy that resulted in the violation
JSONPath: .spec.policy
- name: ResourceKind
type: string
description: The resource kind that cause the violation
JSONPath: .spec.resource.kind
- name: ResourceName
type: string
description: The resource name that caused the violation
JSONPath: .spec.resource.name
- name: Age
type: date
JSONPath: .metadata.creationTimestamp
validation:
openAPIV3Schema:
properties:
spec:
required:
- policy
- resource
- rules
properties:
policy:
type: string
resource:
type: object
required:
- kind
- name
properties:
kind:
type: string
name:
type: string
rules:
type: array
items:
type: object
required:
- name
- type
- message
properties:
name:
type: string
type:
type: string
message:
type: string
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: generaterequests.kyverno.io
spec:
group: kyverno.io
versions:
- name: v1
served: true
storage: true
scope: Namespaced
names:
kind: GenerateRequest
plural: generaterequests
singular: generaterequest
shortNames:
- gr
subresources:
status: {}
additionalPrinterColumns:
- name: Policy
type: string
description: The policy that resulted in the violation
JSONPath: .spec.policy
- name: ResourceKind
type: string
description: The resource kind that cause the violation
JSONPath: .spec.resource.kind
- name: ResourceName
type: string
description: The resource name that caused the violation
JSONPath: .spec.resource.name
- name: ResourceNamespace
type: string
description: The resource namespace that caused the violation
JSONPath: .spec.resource.namespace
- name: status
type : string
description: Current state of generate request
JSONPath: .status.state
- name: Age
type: date
JSONPath: .metadata.creationTimestamp
validation:
openAPIV3Schema:
properties:
spec:
required:
- policy
- resource
properties:
policy:
type: string
resource:
type: object
required:
- kind
- name
properties:
kind:
type: string
name:
type: string
namespace:
type: string

62
charts/kyverno/templates/_helpers.tpl Normal file
View file

@ -0,0 +1,62 @@
{{/* vim: set filetype=mustache: */}}
{{/* Expand the name of the chart. */}}
{{- define "kyverno.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "kyverno.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{/* Create chart name and version as used by the chart label. */}}
{{- define "kyverno.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/* Helm required labels */}}
{{- define "kyverno.labels" -}}
app.kubernetes.io/name: {{ template "kyverno.name" . }}
helm.sh/chart: {{ template "kyverno.chart" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}
{{/* matchLabels */}}
{{- define "kyverno.matchLabels" -}}
app.kubernetes.io/name: {{ template "kyverno.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}
{{/* Get the config map name. */}}
{{- define "kyverno.configMapName" -}}
{{- printf "%s" (default (include "kyverno.fullname" .) .Values.config.existingConfig) -}}
{{- end -}}
{{/* Create the name of the service to use */}}
{{- define "kyverno.serviceName" -}}
{{- printf "%s-svc" (include "kyverno.fullname" .) | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/* Create the name of the service account to use */}}
{{- define "kyverno.serviceAccountName" -}}
{{- if .Values.rbac.serviceAccount.create -}}
{{ default (include "kyverno.fullname" .) .Values.rbac.serviceAccount.name }}
{{- else -}}
{{ default "default" .Values.rbac.serviceAccount.name }}
{{- end -}}
{{- end -}}

147
charts/kyverno/templates/clusterrole.yaml Normal file
View file

@ -0,0 +1,147 @@
{{- if .Values.rbac.create }}
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: {{ template "kyverno.fullname" . }}:policyviolations
rules:
- apiGroups: ["kyverno.io"]
resources:
- policyviolations
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ template "kyverno.fullname" . }}:webhook
rules:
# Dynamic creation of webhooks, events & certs
- apiGroups:
- '*'
resources:
- events
- mutatingwebhookconfigurations
- validatingwebhookconfigurations
- certificatesigningrequests
- certificatesigningrequests/approval
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- certificates.k8s.io
resources:
- certificatesigningrequests
- certificatesigningrequests/approval
- certificatesigningrequests/status
resourceNames:
- kubernetes.io/legacy-unknown
verbs:
- create
- delete
- get
- update
- watch
- apiGroups:
- certificates.k8s.io
resources:
- signers
resourceNames:
- kubernetes.io/legacy-unknown
verbs:
- approve
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ template "kyverno.fullname" . }}:userinfo
rules:
# get the roleRef for incoming api-request user
- apiGroups:
- "*"
resources:
- rolebindings
- clusterrolebindings
- configmaps
verbs:
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ template "kyverno.fullname" . }}:customresources
rules:
# Kyverno CRs
- apiGroups:
- '*'
resources:
- clusterpolicies
- clusterpolicies/status
- clusterpolicyviolations
- clusterpolicyviolations/status
- policyviolations
- policyviolations/status
- generaterequests
- generaterequests/status
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ template "kyverno.fullname" . }}:policycontroller
rules:
# background processing, identify all existing resources
- apiGroups:
- '*'
resources:
- '*'
verbs:
- get
- list
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ template "kyverno.fullname" . }}:generatecontroller
rules:
# process generate rules to generate resources
- apiGroups:
- "*"
resources:
- namespaces
- networkpolicies
- secrets
- configmaps
- resourcequotas
- limitranges
- clusterroles
- rolebindings
- clusterrolebindings
{{- range .Values.generatecontrollerExtraResources }}
- {{ . }}
{{- end }}
verbs:
- create
- update
- delete
- get
# dynamic watches on trigger resources for generate rules
# re-evaluate the policy if the resource is updated
- apiGroups:
- '*'
resources:
- namespaces
verbs:
- watch
{{- end }}

66
charts/kyverno/templates/clusterrolebinding.yaml Normal file
View file

@ -0,0 +1,66 @@
{{- if .Values.rbac.create }}
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ template "kyverno.fullname" . }}:webhook
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ template "kyverno.fullname" . }}:webhook
subjects:
- kind: ServiceAccount
name: {{ template "kyverno.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ template "kyverno.fullname" . }}:userinfo
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ template "kyverno.fullname" . }}:userinfo
subjects:
- kind: ServiceAccount
name: {{ template "kyverno.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ template "kyverno.fullname" . }}:customresources
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ template "kyverno.fullname" . }}:customresources
subjects:
- kind: ServiceAccount
name: {{ template "kyverno.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ template "kyverno.fullname" . }}:policycontroller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ template "kyverno.fullname" . }}:policycontroller
subjects:
- kind: ServiceAccount
name: {{ template "kyverno.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ template "kyverno.fullname" . }}:generatecontroller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ template "kyverno.fullname" . }}:generatecontroller
subjects:
- kind: ServiceAccount
name: {{ template "kyverno.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
{{- end }}

10
charts/kyverno/templates/configmap.yaml Normal file
View file

@ -0,0 +1,10 @@
{{- if (not .Values.config.existingConfig) }}
apiVersion: v1
kind: ConfigMap
metadata:
labels: {{ include "kyverno.labels" . | nindent 4 }}
name: {{ template "kyverno.configMapName" . }}
data:
# resource types to be skipped by kyverno policy engine
resourceFilters: {{ join "" .Values.config.resourceFilters | quote }}
{{- end -}}

65
charts/kyverno/templates/deployment.yaml Normal file
View file

@ -0,0 +1,65 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "kyverno.fullname" . }}
labels: {{ include "kyverno.labels" . | nindent 4 }}
spec:
selector:
matchLabels: {{ include "kyverno.matchLabels" . | nindent 6 }}
replicas: {{ .Values.replicaCount }}
template:
metadata:
labels: {{ include "kyverno.labels" . | nindent 8 }}
{{- range $key, $value := .Values.podLabels }}
{{ $key }}: {{ $value }}
{{- end }}
{{- with .Values.podAnnotations }}
annotations: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
spec:
{{- with .Values.image.pullSecrets }}
imagePullSecrets: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- with .Values.podSecurityContext }}
securityContext: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
serviceAccountName: {{ template "kyverno.serviceAccountName" . }}
{{- if .Values.priorityClassName }}
priorityClassName: {{ .Values.priorityClassName | quote }}
{{- end }}
initContainers:
- name: kyverno-pre
image: {{ .Values.initImage.repository }}:{{ default .Chart.AppVersion (default .Values.image.tag .Values.initImage.tag) }}
imagePullPolicy: {{ default .Values.image.pullPolicy .Values.initImage.pullPolicy }}
containers:
- name: kyverno
image: {{ .Values.image.repository }}:{{ default .Chart.AppVersion .Values.image.tag }}
imagePullPolicy: {{ .Values.image.pullPolicy }}
{{- with .Values.extraArgs }}
args: {{ tpl (toYaml .) $ | nindent 12 }}
{{- end }}
{{- with .Values.resources }}
resources: {{ tpl (toYaml .) $ | nindent 12 }}
{{- end }}
ports:
- containerPort: 443
name: https
protocol: TCP
env:
- name: INIT_CONFIG
value: {{ template "kyverno.configMapName" . }}
{{- with .Values.livenessProbe }}
livenessProbe: {{ tpl (toYaml .) $ | nindent 12 }}
{{- end }}
{{- with .Values.readinessProbe }}
readinessProbe: {{ tpl (toYaml .) $ | nindent 12 }}
{{- end }}

23
charts/kyverno/templates/secret.yaml Normal file
View file

@ -0,0 +1,23 @@
{{- if .Values.createSelfSignedCert }}
{{- $ca := .ca | default (genCA (printf "*.%s.svc" .Release.Namespace) 1024) -}}
{{- $cert := genSignedCert (printf "%s.%s.svc" (include "kyverno.serviceName" .) .Release.Namespace) nil nil 1024 $ca -}}
apiVersion: v1
kind: Secret
metadata:
name: {{ template "kyverno.serviceName" . }}.{{ .Release.Namespace }}.svc.kyverno-tls-ca
labels: {{ include "kyverno.labels" . | nindent 4 }}
data:
rootCA.crt: {{ $ca.Cert | b64enc }}
---
apiVersion: v1
kind: Secret
metadata:
name: {{ template "kyverno.serviceName" . }}.{{ .Release.Namespace }}.svc.kyverno-tls-pair
labels: {{ include "kyverno.labels" . | nindent 4 }}
annotations:
self-signed-cert: "true"
type: kubernetes.io/tls
data:
tls.key: {{ $cert.Key | b64enc }}
tls.crt: {{ $cert.Cert | b64enc }}
{{- end -}}

19
charts/kyverno/templates/service.yaml Normal file
View file

@ -0,0 +1,19 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "kyverno.serviceName" . }}
labels: {{ include "kyverno.labels" . | nindent 4 }}
{{- with .Values.service.annotations }}
annotations: {{ tpl (toYaml .) $ | nindent 4 }}
{{- end }}
spec:
ports:
- port: {{ .Values.service.port }}
targetPort: https
protocol: TCP
name: https
{{- if and (eq .Values.service.type "NodePort") (not (empty .Values.service.nodePort)) }}
nodePort: {{ .Values.service.nodePort }}
{{- end }}
selector: {{ include "kyverno.matchLabels" . | nindent 4 }}
type: {{ .Values.service.type }}

10
charts/kyverno/templates/serviceaccount.yaml Normal file
View file

@ -0,0 +1,10 @@
{{- if .Values.rbac.create }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ template "kyverno.serviceAccountName" . }}
labels: {{ include "kyverno.labels" . | nindent 4 }}
{{- if .Values.rbac.serviceAccount.annotations }}
annotations: {{ toYaml .Values.rbac.serviceAccount.annotations | nindent 4 }}
{{- end }}
{{- end }}

125
charts/kyverno/values.yaml Normal file
View file

@ -0,0 +1,125 @@
nameOverride:
fullnameOverride:
rbac:
create: true
serviceAccount:
create: true
name:
annotations: {}
# example.com/annotation: value
image:
repository: nirmata/kyverno
# Defaults to appVersion in Chart.yaml if omitted
tag:
pullPolicy: IfNotPresent
pullSecrets: []
# - secretName
initImage:
repository: nirmata/kyvernopre
# If initImage.tag is missing, defaults to image.tag
tag:
# If initImage.pullPolicy is missing, defaults to image.pullPolicy
pullPolicy:
# No pull secrets just for initImage; just add to image.pullSecrets
replicaCount: 1
podLabels: {}
# example.com/label: foo
podAnnotations: {}
# example.com/annotation: foo
podSecurityContext: {}
affinity: {}
nodeSelector: {}
tolerations: []
extraArgs: []
# - --fqdn-as-cn
# - --webhooktimeout=4
resources:
limits:
memory: 128Mi
requests:
cpu: 100m
memory: 50Mi
## Liveness Probe. The block is directly forwarded into the deployment, so you can use whatever livenessProbe configuration you want.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
##
livenessProbe:
httpGet:
path: /health/liveness
port: 443
scheme: HTTPS
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 2
successThreshold: 1
## Readiness Probe. The block is directly forwarded into the deployment, so you can use whatever readinessProbe configuration you want.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
##
readinessProbe:
httpGet:
path: /health/readiness
port: 443
scheme: HTTPS
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
successThreshold: 1
# TODO(mbarrien): Should we just list all resources for the
# generatecontroller in here rather than having defaults hard-coded?
generatecontrollerExtraResources:
# - ResourceA
# - ResourceB
config:
# resource types to be skipped by kyverno policy engine
# Make sure to surround each entry in quotes so that it doesn't get parsed
# as a nested YAML list. These are joined together without spaces in the configmap
resourceFilters:
- "[Event,*,*]"
- "[*,kube-system,*]"
- "[*,kube-public,*]"
- "[*,kube-node-lease,*]"
- "[Node,*,*]"
- "[APIService,*,*]"
- "[TokenReview,*,*]"
- "[SubjectAccessReview,*,*]"
- "[*,kyverno,*]"
# Or give the name of an existing config map (ignores default/provided resourceFilters)
existingConfig:
# existingConfig: init-config
service:
port: 443
type: ClusterIP
# Only used if service.type is NodePort
nodePort:
## Provide any additional annotations which may be required. This can be used to
## set the LoadBalancer service type to internal only.
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
##
annotations: {}
# Kyverno requires a certificate key pair and corresponding certificate authority
# to properly register its webhooks. This can be done in one of 3 ways:
# 1) Use kube-controller-manager to generate a CA-signed certificate (preferred)
# 2) Provide your own CA and cert.
# In this case, you will need to create a certificate with a specific name and data structure.
# As long as you follow the naming scheme, it will be automatically picked up.
# kyverno-svc.(namespace).svc.kyverno-tls-ca (with data entry named rootCA.crt)
# kyverno-svc.kyverno.svc.kyverno-tls-key-pair (with data entries named tls.key and tls.crt)
# 3) Let Helm generate a self signed cert, by setting createSelfSignedCert true
# If letting Kyverno create its own CA or providing your own, make createSelfSignedCert is false
createSelfSignedCert: false

97
cmd/initContainer/main.go
View file

@ -5,23 +5,26 @@ package main
import (
"flag"
"fmt"
"os"
"regexp"
"strconv"
"sync"
"time"
"github.com/golang/glog"
"github.com/nirmata/kyverno/pkg/config"
client "github.com/nirmata/kyverno/pkg/dclient"
"github.com/nirmata/kyverno/pkg/signal"
"github.com/nirmata/kyverno/pkg/utils"
"k8s.io/apimachinery/pkg/api/errors"
rest "k8s.io/client-go/rest"
clientcmd "k8s.io/client-go/tools/clientcmd"
"k8s.io/klog"
"k8s.io/klog/klogr"
"sigs.k8s.io/controller-runtime/pkg/log"
)
var (
kubeconfig string
setupLog = log.Log.WithName("setup")
)
const (
@ -30,26 +33,38 @@ const (
)
func main() {
defer glog.Flush()
klog.InitFlags(nil)
log.SetLogger(klogr.New())
// arguments
flag.StringVar(&kubeconfig, "kubeconfig", "", "Path to a kubeconfig. Only required if out-of-cluster.")
if err := flag.Set("v", "2"); err != nil {
klog.Fatalf("failed to set log level: %v", err)
}
flag.Parse()
// os signal handler
stopCh := signal.SetupSignalHandler()
// create client config
clientConfig, err := createClientConfig(kubeconfig)
if err != nil {
glog.Fatalf("Error building kubeconfig: %v\n", err)
setupLog.Error(err, "Failed to build kubeconfig")
os.Exit(1)
}
// DYNAMIC CLIENT
// - client for all registered resources
client, err := client.NewClient(clientConfig, 10*time.Second, stopCh)
client, err := client.NewClient(clientConfig, 15*time.Minute, stopCh, log.Log)
if err != nil {
glog.Fatalf("Error creating client: %v\n", err)
setupLog.Error(err, "Failed to create client")
os.Exit(1)
}
// Exit for unsupported version of kubernetes cluster
// https://github.com/nirmata/kyverno/issues/700
// - supported from v1.12.7+
isVersionSupported(client)
if !utils.HigherThanKubernetesVersion(client, log.Log, 1, 12, 7) {
os.Exit(1)
}
requests := []request{
// Resource
@ -78,53 +93,46 @@ func main() {
for err := range merge(done, stopCh, p1, p2) {
if err != nil {
failure = true
glog.Errorf("failed to cleanup: %v", err)
log.Log.Error(err, "failed to cleanup resource")
}
}
// if there is any failure then we fail process
if failure {
glog.Errorf("failed to cleanup webhook configurations")
log.Log.Info("failed to cleanup webhook configurations")
os.Exit(1)
}
}
func init() {
// arguments
flag.StringVar(&kubeconfig, "kubeconfig", "", "Path to a kubeconfig. Only required if out-of-cluster.")
flag.Set("logtostderr", "true")
flag.Set("stderrthreshold", "WARNING")
flag.Set("v", "2")
flag.Parse()
}
func removeWebhookIfExists(client *client.Client, kind string, name string) error {
logger := log.Log.WithName("removeExistingWebhook").WithValues("kind", kind, "name", name)
var err error
// Get resource
_, err = client.GetResource(kind, "", name)
if errors.IsNotFound(err) {
glog.V(4).Infof("%s(%s) not found", name, kind)
logger.V(4).Info("resource not found")
return nil
}
if err != nil {
glog.Errorf("failed to get resource %s(%s)", name, kind)
logger.Error(err, "failed to get resource")
return err
}
// Delete resource
err = client.DeleteResource(kind, "", name, false)
if err != nil {
glog.Errorf("failed to delete resource %s(%s)", name, kind)
logger.Error(err, "failed to delete resource")
return err
}
glog.Infof("cleaned up resource %s(%s)", name, kind)
logger.Info("removed the resource")
return nil
}
func createClientConfig(kubeconfig string) (*rest.Config, error) {
logger := log.Log
if kubeconfig == "" {
glog.Info("Using in-cluster configuration")
logger.Info("Using in-cluster configuration")
return rest.InClusterConfig()
}
glog.Infof("Using configuration from '%s'", kubeconfig)
logger.Info(fmt.Sprintf("Using configuration from '%s'", kubeconfig))
return clientcmd.BuildConfigFromFlags("", kubeconfig)
}
@ -163,6 +171,7 @@ func gen(done <-chan struct{}, stopCh <-chan struct{}, requests ...request) <-ch
// processes the requests
func process(client *client.Client, done <-chan struct{}, stopCh <-chan struct{}, requests <-chan request) <-chan error {
logger := log.Log.WithName("process")
out := make(chan error)
go func() {
defer close(out)
@ -170,10 +179,10 @@ func process(client *client.Client, done <-chan struct{}, stopCh <-chan struct{}
select {
case out <- removeWebhookIfExists(client, req.kind, req.name):
case <-done:
println("done process")
logger.Info("done")
return
case <-stopCh:
println("shutting down process")
logger.Info("shutting down")
return
}
}
@ -183,6 +192,7 @@ func process(client *client.Client, done <-chan struct{}, stopCh <-chan struct{}
// waits for all processes to be complete and merges result
func merge(done <-chan struct{}, stopCh <-chan struct{}, processes ...<-chan error) <-chan error {
logger := log.Log.WithName("merge")
var wg sync.WaitGroup
out := make(chan error)
// gets the output from each process
@ -192,10 +202,10 @@ func merge(done <-chan struct{}, stopCh <-chan struct{}, processes ...<-chan err
select {
case out <- err:
case <-done:
println("done merge")
logger.Info("done")
return
case <-stopCh:
println("shutting down merge")
logger.Info("shutting down")
return
}
}
@ -213,32 +223,3 @@ func merge(done <-chan struct{}, stopCh <-chan struct{}, processes ...<-chan err
}()
return out
}
func isVersionSupported(client *client.Client) {
serverVersion, err := client.DiscoveryClient.GetServerVersion()
if err != nil {
glog.Fatalf("Failed to get kubernetes server version: %v\n", err)
}
exp := regexp.MustCompile(`v(\d*).(\d*).(\d*)`)
groups := exp.FindAllStringSubmatch(serverVersion.String(), -1)
if len(groups) != 1 || len(groups[0]) != 4 {
glog.Fatalf("Failed to extract kubernetes server version: %v.err %v\n", serverVersion, err)
}
// convert string to int
// assuming the version are always intergers
major, err := strconv.Atoi(groups[0][1])
if err != nil {
glog.Fatalf("Failed to extract kubernetes major server version: %v.err %v\n", serverVersion, err)
}
minor, err := strconv.Atoi(groups[0][2])
if err != nil {
glog.Fatalf("Failed to extract kubernetes minor server version: %v.err %v\n", serverVersion, err)
}
sub, err := strconv.Atoi(groups[0][3])
if err != nil {
glog.Fatalf("Failed to extract kubernetes sub minor server version:%v. err %v\n", serverVersion, err)
}
if major <= 1 && minor <= 12 && sub < 7 {
glog.Fatalf("Unsupported kubernetes server version %s. Kyverno is supported from version v1.12.7+", serverVersion)
}
}

190
cmd/kyverno/main.go
View file

@ -3,11 +3,12 @@ package main
import (
"context"
"flag"
"fmt"
"os"
"time"
"github.com/nirmata/kyverno/pkg/openapi"
"github.com/golang/glog"
"github.com/nirmata/kyverno/pkg/checker"
kyvernoclient "github.com/nirmata/kyverno/pkg/client/clientset/versioned"
kyvernoinformer "github.com/nirmata/kyverno/pkg/client/informers/externalversions"
@ -18,7 +19,6 @@ import (
generatecleanup "github.com/nirmata/kyverno/pkg/generate/cleanup"
"github.com/nirmata/kyverno/pkg/policy"
"github.com/nirmata/kyverno/pkg/policystatus"
"github.com/nirmata/kyverno/pkg/policystore"
"github.com/nirmata/kyverno/pkg/policyviolation"
"github.com/nirmata/kyverno/pkg/signal"
"github.com/nirmata/kyverno/pkg/utils"
@ -27,8 +27,13 @@ import (
"github.com/nirmata/kyverno/pkg/webhooks"
webhookgenerate "github.com/nirmata/kyverno/pkg/webhooks/generate"
kubeinformers "k8s.io/client-go/informers"
"k8s.io/klog"
"k8s.io/klog/klogr"
log "sigs.k8s.io/controller-runtime/pkg/log"
)
const resyncPeriod = 15 * time.Minute
var (
kubeconfig string
serverIP string
@ -38,87 +43,96 @@ var (
// will be removed in future and the configuration will be set only via configmaps
filterK8Resources string
// User FQDN as CSR CN
fqdncn bool
fqdncn bool
setupLog = log.Log.WithName("setup")
)
func main() {
defer glog.Flush()
version.PrintVersionInfo()
// cleanUp Channel
cleanUp := make(chan struct{})
// handle os signals
stopCh := signal.SetupSignalHandler()
// CLIENT CONFIG
clientConfig, err := config.CreateClientConfig(kubeconfig)
if err != nil {
glog.Fatalf("Error building kubeconfig: %v\n", err)
klog.InitFlags(nil)
log.SetLogger(klogr.New())
flag.StringVar(&filterK8Resources, "filterK8Resources", "", "k8 resource in format [kind,namespace,name] where policy is not evaluated by the admission webhook. example --filterKind \"[Deployment, kyverno, kyverno]\" --filterKind \"[Deployment, kyverno, kyverno],[Events, *, *]\"")
flag.IntVar(&webhookTimeout, "webhooktimeout", 3, "timeout for webhook configurations")
flag.StringVar(&kubeconfig, "kubeconfig", "", "Path to a kubeconfig. Only required if out-of-cluster.")
flag.StringVar(&serverIP, "serverIP", "", "IP address where Kyverno controller runs. Only required if out-of-cluster.")
flag.StringVar(&runValidationInMutatingWebhook, "runValidationInMutatingWebhook", "", "Validation will also be done using the mutation webhook, set to 'true' to enable. Older kubernetes versions do not work properly when a validation webhook is registered.")
if err := flag.Set("v", "2"); err != nil {
setupLog.Error(err, "failed to set log level")
os.Exit(1)
}
// KYVENO CRD CLIENT
// Generate CSR with CN as FQDN due to https://github.com/nirmata/kyverno/issues/542
flag.BoolVar(&fqdncn, "fqdn-as-cn", false, "use FQDN as Common Name in CSR")
flag.Parse()
version.PrintVersionInfo(log.Log)
cleanUp := make(chan struct{})
stopCh := signal.SetupSignalHandler()
clientConfig, err := config.CreateClientConfig(kubeconfig, log.Log)
if err != nil {
setupLog.Error(err, "Failed to build kubeconfig")
os.Exit(1)
}
// KYVERNO CRD CLIENT
// access CRD resources
// - Policy
// - PolicyViolation
pclient, err := kyvernoclient.NewForConfig(clientConfig)
if err != nil {
glog.Fatalf("Error creating client: %v\n", err)
setupLog.Error(err, "Failed to create client")
os.Exit(1)
}
// DYNAMIC CLIENT
// - client for all registered resources
// - invalidate local cache of registered resource every 10 seconds
client, err := dclient.NewClient(clientConfig, 10*time.Second, stopCh)
client, err := dclient.NewClient(clientConfig, 5*time.Minute, stopCh, log.Log)
if err != nil {
glog.Fatalf("Error creating client: %v\n", err)
setupLog.Error(err, "Failed to create client")
os.Exit(1)
}
// CRD CHECK
// - verify if the CRD for Policy & PolicyViolation are available
if !utils.CRDInstalled(client.DiscoveryClient) {
glog.Fatalf("Required CRDs unavailable")
if !utils.CRDInstalled(client.DiscoveryClient, log.Log) {
setupLog.Error(fmt.Errorf("CRDs not installed"), "Failed to access Kyverno CRDs")
os.Exit(1)
}
// KUBERNETES CLIENT
kubeClient, err := utils.NewKubeClient(clientConfig)
if err != nil {
glog.Fatalf("Error creating kubernetes client: %v\n", err)
setupLog.Error(err, "Failed to create kubernetes client")
os.Exit(1)
}
// TODO(shuting): To be removed for v1.2.0
utils.CleanupOldCrd(client)
// TODO: To be removed for v1.2.0
utils.CleanupOldCrd(client, log.Log)
// KUBERNETES RESOURCES INFORMER
// watches namespace resource
// - cache resync time: 10 seconds
kubeInformer := kubeinformers.NewSharedInformerFactoryWithOptions(
kubeClient,
10*time.Second)
// KUBERNETES Dynamic informer
// - cahce resync time: 10 seconds
kubedynamicInformer := client.NewDynamicSharedInformerFactory(10 * time.Second)
kubeInformer := kubeinformers.NewSharedInformerFactoryWithOptions(kubeClient, resyncPeriod)
kubedynamicInformer := client.NewDynamicSharedInformerFactory(resyncPeriod)
// WERBHOOK REGISTRATION CLIENT
webhookRegistrationClient := webhookconfig.NewWebhookRegistrationClient(
clientConfig,
client,
serverIP,
int32(webhookTimeout))
int32(webhookTimeout),
log.Log)
// Resource Mutating Webhook Watcher
lastReqTime := checker.NewLastReqTime()
lastReqTime := checker.NewLastReqTime(log.Log.WithName("LastReqTime"))
rWebhookWatcher := webhookconfig.NewResourceWebhookRegister(
lastReqTime,
kubeInformer.Admissionregistration().V1beta1().MutatingWebhookConfigurations(),
kubeInformer.Admissionregistration().V1beta1().ValidatingWebhookConfigurations(),
webhookRegistrationClient,
runValidationInMutatingWebhook,
log.Log.WithName("ResourceWebhookRegister"),
)
// KYVERNO CRD INFORMER
// watches CRD resources:
// - Policy
// - PolicyVolation
// - cache resync time: 10 seconds
pInformer := kyvernoinformer.NewSharedInformerFactoryWithOptions(
pclient,
10*time.Second)
pInformer := kyvernoinformer.NewSharedInformerFactoryWithOptions(pclient, resyncPeriod)
// Configuration Data
// dynamically load the configuration from configMap
@ -127,21 +141,22 @@ func main() {
configData := config.NewConfigData(
kubeClient,
kubeInformer.Core().V1().ConfigMaps(),
filterK8Resources)
// Policy meta-data store
policyMetaStore := policystore.NewPolicyStore(pInformer.Kyverno().V1().ClusterPolicies())
filterK8Resources,
log.Log.WithName("ConfigData"),
)
// EVENT GENERATOR
// - generate event with retry mechanism
egen := event.NewEventGenerator(
eventGenerator := event.NewEventGenerator(
client,
pInformer.Kyverno().V1().ClusterPolicies())
pInformer.Kyverno().V1().ClusterPolicies(),
log.Log.WithName("EventGenerator"))
// Policy Status Handler - deals with all logic related to policy status
statusSync := policystatus.NewSync(
pclient,
policyMetaStore)
pInformer.Kyverno().V1().ClusterPolicies().Lister(),)
// POLICY VIOLATION GENERATOR
// -- generate policy violation
@ -149,29 +164,34 @@ func main() {
client,
pInformer.Kyverno().V1().ClusterPolicyViolations(),
pInformer.Kyverno().V1().PolicyViolations(),
statusSync.Listener)
statusSync.Listener,
log.Log.WithName("PolicyViolationGenerator"),
)
// POLICY CONTROLLER
// - reconciliation policy and policy violation
// - process policy on existing resources
// - status aggregator: receives stats when a policy is applied
// & updates the policy status
pc, err := policy.NewPolicyController(pclient,
// - status aggregator: receives stats when a policy is applied & updates the policy status
policyCtrl, err := policy.NewPolicyController(pclient,
client,
pInformer.Kyverno().V1().ClusterPolicies(),
pInformer.Kyverno().V1().ClusterPolicyViolations(),
pInformer.Kyverno().V1().PolicyViolations(),
configData,
egen,
eventGenerator,
pvgen,
policyMetaStore,
rWebhookWatcher)
rWebhookWatcher,
kubeInformer.Core().V1().Namespaces(),
log.Log.WithName("PolicyController"),
)
if err != nil {
glog.Fatalf("error creating policy controller: %v\n", err)
setupLog.Error(err, "Failed to create policy controller")
os.Exit(1)
}
// GENERATE REQUEST GENERATOR
grgen := webhookgenerate.NewGenerator(pclient, stopCh)
grgen := webhookgenerate.NewGenerator(pclient, stopCh, log.Log.WithName("GenerateRequestGenerator"))
// GENERATE CONTROLLER
// - applies generate rules on resources based on generate requests created by webhook
@ -180,11 +200,13 @@ func main() {
client,
pInformer.Kyverno().V1().ClusterPolicies(),
pInformer.Kyverno().V1().GenerateRequests(),
egen,
eventGenerator,
pvgen,
kubedynamicInformer,
statusSync.Listener,
log.Log.WithName("GenerateController"),
)
// GENERATE REQUEST CLEANUP
// -- cleans up the generate requests that have not been processed(i.e. state = [Pending, Failed]) for more than defined timeout
grcc := generatecleanup.NewController(
@ -193,12 +215,14 @@ func main() {
pInformer.Kyverno().V1().ClusterPolicies(),
pInformer.Kyverno().V1().GenerateRequests(),
kubedynamicInformer,
log.Log.WithName("GenerateCleanUpController"),
)
// CONFIGURE CERTIFICATES
tlsPair, err := client.InitTLSPemPair(clientConfig, fqdncn)
if err != nil {
glog.Fatalf("Failed to initialize TLS key/certificate pair: %v\n", err)
setupLog.Error(err, "Failed to initialize TLS key/certificate pair")
os.Exit(1)
}
// WEBHOOK REGISTRATION
@ -207,13 +231,20 @@ func main() {
// resource webhook confgiuration is generated dynamically in the webhook server and policy controller
// based on the policy resources created
if err = webhookRegistrationClient.Register(); err != nil {
glog.Fatalf("Failed registering Admission Webhooks: %v\n", err)
setupLog.Error(err, "Failed to register Admission webhooks")
os.Exit(1)
}
openAPIController, err := openapi.NewOpenAPIController()
if err != nil {
setupLog.Error(err, "Failed to create openAPIController")
os.Exit(1)
}
// Sync openAPI definitions of resources
openApiSync := openapi.NewCRDSync(client)
openAPISync := openapi.NewCRDSync(client, openAPIController)
// WEBHOOOK
// WEBHOOK
// - https server to provide endpoints called based on rules defined in Mutating & Validation webhook configuration
// - reports the results based on the response from the policy engine:
// -- annotations on resources with update details on mutation JSON patches
@ -226,18 +257,23 @@ func main() {
pInformer.Kyverno().V1().ClusterPolicies(),
kubeInformer.Rbac().V1().RoleBindings(),
kubeInformer.Rbac().V1().ClusterRoleBindings(),
egen,
eventGenerator,
webhookRegistrationClient,
statusSync.Listener,
configData,
policyMetaStore,
pvgen,
grgen,
rWebhookWatcher,
cleanUp)
cleanUp,
log.Log.WithName("WebhookServer"),
openAPIController,
)
if err != nil {
glog.Fatalf("Unable to create webhook server: %v\n", err)
setupLog.Error(err, "Failed to create webhook server")
os.Exit(1)
}
// Start the components
pInformer.Start(stopCh)
kubeInformer.Start(stopCh)
@ -245,14 +281,13 @@ func main() {
go grgen.Run(1)
go rWebhookWatcher.Run(stopCh)
go configData.Run(stopCh)
go policyMetaStore.Run(stopCh)
go pc.Run(1, stopCh)
go egen.Run(1, stopCh)
go policyCtrl.Run(3, stopCh)
go eventGenerator.Run(1, stopCh)
go grc.Run(1, stopCh)
go grcc.Run(1, stopCh)
go pvgen.Run(1, stopCh)
go statusSync.Run(1, stopCh)
go openApiSync.Run(1, stopCh)
openAPISync.Run(1, stopCh)
// verifys if the admission control is enabled and active
// resync: 60 seconds
@ -269,23 +304,12 @@ func main() {
defer func() {
cancel()
}()
// cleanup webhookconfigurations followed by webhook shutdown
server.Stop(ctx)
// resource cleanup
// remove webhook configurations
<-cleanUp
glog.Info("successful shutdown of kyverno controller")
}
func init() {
flag.StringVar(&filterK8Resources, "filterK8Resources", "", "k8 resource in format [kind,namespace,name] where policy is not evaluated by the admission webhook. example --filterKind \"[Deployment, kyverno, kyverno]\" --filterKind \"[Deployment, kyverno, kyverno],[Events, *, *]\"")
flag.IntVar(&webhookTimeout, "webhooktimeout", 3, "timeout for webhook configurations")
flag.StringVar(&kubeconfig, "kubeconfig", "", "Path to a kubeconfig. Only required if out-of-cluster.")
flag.StringVar(&serverIP, "serverIP", "", "IP address where Kyverno controller runs. Only required if out-of-cluster.")
flag.StringVar(&runValidationInMutatingWebhook, "runValidationInMutatingWebhook", "", "Validation will also be done using the mutation webhook, set to 'true' to enable. Older kubernetes versions do not work properly when a validation webhook is registered.")
// Generate CSR with CN as FQDN due to https://github.com/nirmata/kyverno/issues/542
flag.BoolVar(&fqdncn, "fqdn-as-cn", false, "use FQDN as Common Name in CSR")
config.LogDefaultFlags()
flag.Parse()
setupLog.Info("Kyverno shutdown successful")
}

96
definitions/install.yaml
View file

@ -20,6 +20,7 @@ spec:
validation:
openAPIV3Schema:
properties:
status: {}
spec:
required:
- rules
@ -69,7 +70,7 @@ spec:
type: string
name:
type: string
Namespace:
namespace:
type: string
resources:
type: object
@ -109,8 +110,6 @@ spec:
type: string
exclude:
type: object
required:
- resources
properties:
roles:
type: array
@ -134,7 +133,7 @@ spec:
type: string
name:
type: string
Namespace:
namespace:
type: string
resources:
type: object
@ -211,6 +210,28 @@ spec:
AnyValue: {}
anyPattern:
AnyValue: {}
deny:
properties:
conditions:
type: array
items:
type: object
required:
- key # can be of any type
- operator # typed
- value # can be of any type
properties:
operator:
type: string
enum:
- Equal
- Equals
- NotEqual
- NotEquals
key:
type: string
value:
type: string
generate:
type: object
required:
@ -221,6 +242,8 @@ spec:
type: string
name:
type: string
namespace:
type: string
clone:
type: object
required:
@ -566,6 +589,28 @@ rules:
- patch
- update
- watch
- apiGroups:
- certificates.k8s.io
resources:
- certificatesigningrequests
- certificatesigningrequests/approval
- certificatesigningrequests/status
resourceNames:
- kubernetes.io/legacy-unknown
verbs:
- create
- delete
- get
- update
- watch
- apiGroups:
- certificates.k8s.io
resources:
- signers
resourceNames:
- kubernetes.io/legacy-unknown
verbs:
- approve
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
@ -621,6 +666,7 @@ rules:
verbs:
- get
- list
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
@ -661,7 +707,7 @@ metadata:
namespace: kyverno
data:
# resource types to be skipped by kyverno policy engine
resourceFilters: "[Event,*,*][*,kube-system,*][*,kube-public,*][*,kube-node-lease,*][Node,*,*][APIService,*,*][TokenReview,*,*][SubjectAccessReview,*,*][*,kyverno,*]"
resourceFilters: "[Event,*,*][*,kube-system,*][*,kube-public,*][*,kube-node-lease,*][Node,*,*][APIService,*,*][TokenReview,*,*][SubjectAccessReview,*,*][*,kyverno,*][Binding,*,*][ReplicaSet,*,*]"
---
apiVersion: apps/v1
kind: Deployment
@ -683,16 +729,44 @@ spec:
serviceAccountName: kyverno-service-account
initContainers:
- name: kyverno-pre
image: nirmata/kyvernopre:v1.1.4-rc1
image: nirmata/kyvernopre:v1.1.6-rc2
containers:
- name: kyverno
image: nirmata/kyverno:v1.1.4-rc1
image: nirmata/kyverno:v1.1.6-rc2
imagePullPolicy: Always
args:
- "--filterK8Resources=[Event,*,*][*,kube-system,*][*,kube-public,*][*,kube-node-lease,*][Node,*,*][APIService,*,*][TokenReview,*,*][SubjectAccessReview,*,*][*,kyverno,*]"
# customize webhook timout
# - "--webhooktimeout=4"
- "--filterK8Resources=[Event,*,*][*,kube-system,*][*,kube-public,*][*,kube-node-lease,*][Node,*,*][APIService,*,*][TokenReview,*,*][SubjectAccessReview,*,*][*,kyverno,*][Binding,*,*][ReplicaSet,*,*]"
# customize webhook timeout
#- "--webhooktimeout=4"
- "-v=2"
ports:
- containerPort: 443
env:
- name: INIT_CONFIG
value: init-config
value: init-config
resources:
requests:
memory: "50Mi"
cpu: "100m"
limits:
memory: "128Mi"
livenessProbe:
httpGet:
path: /health/liveness
port: 443
scheme: HTTPS
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 4
successThreshold: 1
readinessProbe:
httpGet:
path: /health/readiness
port: 443
scheme: HTTPS
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 4
successThreshold: 1

29
definitions/install_debug.yaml
View file

@ -20,6 +20,7 @@ spec:
validation:
openAPIV3Schema:
properties:
status: {}
spec:
required:
- rules
@ -109,8 +110,6 @@ spec:
type: string
exclude:
type: object
required:
- resources
properties:
roles:
type: array
@ -134,7 +133,7 @@ spec:
type: string
name:
type: string
Namespace:
namespace:
type: string
resources:
type: object
@ -211,6 +210,28 @@ spec:
AnyValue: {}
anyPattern:
AnyValue: {}
deny:
properties:
conditions:
type: array
items:
type: object
required:
- key # can be of any type
- operator # typed
- value # can be of any type
properties:
operator:
type: string
enum:
- Equal
- Equals
- NotEqual
- NotEquals
key:
type: string
value:
type: string
generate:
type: object
required:
@ -221,6 +242,8 @@ spec:
type: string
name:
type: string
namespace:
type: string
clone:
type: object
required:

28
documentation/installation.md
View file

@ -4,7 +4,7 @@
The Kyverno policy engine runs as an admission webhook and requires a CA-signed certificate and key to setup secure TLS communication with the kube-apiserver (the CA can be self-signed).
There are 2 ways to configure the secure communications link between Kyverno and the kube-apiserver:
There are 2 ways to configure the secure communications link between Kyverno and the kube-apiserver.
## Option 1: Use kube-controller-manager to generate a CA-signed certificate
@ -18,7 +18,7 @@ To install Kyverno in a cluster that supports certificate signing, run the follo
kubectl create -f https://github.com/nirmata/kyverno/raw/master/definitions/install.yaml
````
Note that the above command will install the last released (stable) version of Kyverno. If you want to install the latest version, you can edit the `install.yaml` and update the image tag.
Note that the above command will install the last released (stable) version of Kyverno. If you want to install the latest version, you can edit the [install.yaml] and update the image tag.
To check the Kyverno controller status, run the command:
@ -133,7 +133,7 @@ subjects:
### 4. Install Kyverno
To install a specific version, change the image tag with git tag in `install.yaml`.
To install a specific version, download [install.yaml] and then change the image tag.
e.g., change image tag from `latest` to the specific tag `v1.0.0`.
>>>
@ -141,10 +141,10 @@ e.g., change image tag from `latest` to the specific tag `v1.0.0`.
containers:
- name: kyverno
# image: nirmata/kyverno:latest
image: nirmata/kyverno:v0.3.0
image: nirmata/kyverno:v1.0.0
````sh
kubectl create -f https://github.com/nirmata/kyverno/raw/master/definitions/install.yaml
kubectl create -f ./install.yaml
````
To check the Kyverno controller status, run the command:
@ -168,7 +168,7 @@ Here is a script that generates a self-signed CA, a TLS certificate-key pair, an
# Configure a namespace admin to access policy violations
During Kyverno installation, it creates a ClusterRole `kyverno:policyviolations` which has the `list,get,watch` operation on resource `policyviolations`. To grant access to a namespace admin, configure the following YAML file then apply to the cluster.
During Kyverno installation, it creates a ClusterRole `kyverno:policyviolations` which has the `list,get,watch` operations on resource `policyviolations`. To grant access to a namespace admin, configure the following YAML file then apply to the cluster.
- Replace `metadata.namespace` with namespace of the admin
- Configure `subjects` field to bind admin's role to the ClusterRole `policyviolation`
@ -200,16 +200,16 @@ subjects:
To build Kyverno in a development environment see: https://github.com/nirmata/kyverno/wiki/Building
To run controller in this mode you should prepare TLS key/certificate pair for debug webhook, then start controller with kubeconfig and the server address.
To run controller in this mode you should prepare a TLS key/certificate pair for debug webhook, then start controller with kubeconfig and the server address.
1. Run `scripts/deploy-controller-debug.sh --service=localhost --serverIP=<server_IP>`, where <server_IP> is the IP address of the host where controller runs. This scripts will generate TLS certificate for debug webhook server and register this webhook in the cluster. Also it registers CustomResource Policy.
1. Run `scripts/deploy-controller-debug.sh --service=localhost --serverIP=<server_IP>`, where <server_IP> is the IP address of the host where controller runs. This scripts will generate a TLS certificate for debug webhook server and register this webhook in the cluster. It also registers a CustomResource policy.
2. Start the controller using the following command: `sudo kyverno --kubeconfig=~/.kube/config --serverIP=<server_IP>`
# Filter kuberenetes resources that admission webhook should not process
The admission webhook checks if a policy is applicable on all admission requests. The kubernetes kinds that are not be processed can be filtered by adding the configmap named `init-config` in namespace `kyverno` and specifying the resources to be filtered under `data.resourceFilters`
# Filter Kubernetes resources that admission webhook should not process
The admission webhook checks if a policy is applicable on all admission requests. The Kubernetes kinds that are not be processed can be filtered by adding a `ConfigMap` in namespace `kyverno` and specifying the resources to be filtered under `data.resourceFilters`. The default name of this `ConfigMap` is `init-config` but can be changed by modifying the value of the environment variable `INIT_CONFIG` in the kyverno deployment dpec. `data.resourceFilters` must be a sequence of one or more `[<Kind>,<Namespace>,<Name>]` entries with `*` as wildcard. Thus, an item `[Node,*,*]` means that admissions of `Node` in any namespace and with any name will be ignored.
THe confimap is picked from the envenvironment variable `INIT_CONFIG` passed to the kyverno deployment spec. The resourceFilters configuration can be updated dynamically at runtime.
By default we have specified Nodes, Events, APIService & SubjectAccessReview as the kinds to be skipped in the default configuration [install.yaml].
```
apiVersion: v1
@ -222,9 +222,11 @@ data:
resourceFilters: "[Event,*,*][*,kube-system,*][*,kube-public,*][*,kube-node-lease,*][Node,*,*][APIService,*,*][TokenReview,*,*][SubjectAccessReview,*,*][*,kyverno,*]"
```
By default we have specified Nodes, Events, APIService & SubjectAccessReview as the kinds to be skipped in the default configmap
[install.yaml](https://github.com/nirmata/kyverno/raw/master/definitions/install.yaml).
To modify the `ConfigMap`, either directly edit the `ConfigMap` `init-config` in the default configuration [install.yaml] and redeploy it or modify the `ConfigMap` use `kubectl`. Changes to the `ConfigMap` through `kubectl` will automatically be picked up at runtime.
---
<small>*Read Next >> [Writing Policies](/documentation/writing-policies.md)*</small>
[install.yaml]: https://github.com/nirmata/kyverno/raw/master/definitions/install.yaml

8
documentation/kyverno-cli.md
View file

@ -16,6 +16,14 @@ make cli
mv ./cmd/cli/kubectl-kyverno/kyverno /usr/local/bin/kyverno
```
## Install via AUR (archlinux)
You can install the kyverno cli via your favourite AUR helper (e.g. [yay](https://github.com/Jguer/yay))
```
yay -S kyverno-git
```
## Commands
#### Version

16
documentation/testing-policies.md
View file

@ -3,23 +3,23 @@
# Testing Policies
The resources definitions for testing are located in [/test](/test) directory. Each test contains a pair of files: one is the resource definition, and the second is the kyverno policy for this definition.
The resources definitions for testing are located in the [test](/test) directory. Each test contains a pair of files: one is the resource definition, and the second is the Kyverno policy for this definition.
## Test using kubectl
To do this you should [install kyverno to the cluster](/documentation/installation.md).
To do this you should [install Kyverno to the cluster](installation.md).
For example, to test the simplest kyverno policy for ConfigMap, create the policy and then the resource itself via kubectl:
For example, to test the simplest Kyverno policy for `ConfigMap`, create the policy and then the resource itself via `kubectl`:
````bash
cd test/ConfigMap
kubectl create -f policy-CM.yaml
kubectl create -f CM.yaml
cd test
kubectl create -f policy/policy-CM.yaml
kubectl create -f resources/CM.yaml
````
Then compare the original resource definition in CM.yaml with the actual one:
Then compare the original resource definition in `CM.yaml` with the actual one:
````bash
kubectl get -f CM.yaml -o yaml
kubectl get -f resources/CM.yaml -o yaml
````
## Test using Kyverno CLI

2
documentation/writing-policies-autogen.md
View file

@ -2,7 +2,7 @@
# Auto Generating Rules for Pod Controllers
Writing policies on pods helps address all pod creation flows. However, when pod cotrollers are used pod level policies result in errors not being reported when the pod controller object is created.
Writing policies on pods helps address all pod creation flows. However, when pod controllers are used, pod level policies result in errors not being reported when the pod controller object is created.
Kyverno solves this issue by supporting automatic generation of policy rules for pod controllers from a rule written for a pod.

8
documentation/writing-policies-generate.md
View file

@ -1,6 +1,6 @@
<small>*[documentation](/README.md#documentation) / [Writing Policies](/documentation/writing-policies.md) / Generate Resources*</small>
# Generate Resources
# Generating Resources
The ```generate``` rule can used to create additional resources when a new resource is created. This is useful to create supporting resources, such as new role bindings for a new namespace.
@ -48,8 +48,8 @@ spec:
````
In this example new namespaces will receive 2 new resources after its creation:
* A ConfigMap cloned from default/config-template.
* A Secret with values DB_USER and DB_PASSWORD, and label ```purpose: mongo```.
* A `ConfigMap` cloned from `default/config-template`.
* A `Secret` with values `DB_USER` and `DB_PASSWORD`, and label `purpose: mongo`.
## Example 2
@ -81,7 +81,7 @@ spec:
policyname: "default"
````
In this example new namespaces will receive a NetworkPolicy that default denies all inbound and outbound traffic.
In this example new namespaces will receive a `NetworkPolicy` that by default denies all inbound and outbound traffic.
---

93
documentation/writing-policies-match-exclude.md Normal file
View file

@ -0,0 +1,93 @@
<small>*[documentation](/README.md#documentation) / Writing Policies / Match & Exclude *</small>
# Match & Exclude
The `match` and `exclude` filters control which resources policies are applied to.
The match / exclude clauses have the same structure, and can each contain the following elements:
* resources: select resources by name, namespaces, kinds, and label selectors.
* subjects: select users, user groups, and service accounts
* roles: select namespaced roles
* clusterroles: select cluster wide roles
At least one element must be specified in a `match` block. The `kind` attribute is optional, but if it's not specified the policy rule will only be applicable to metatdata that is common across all resources kinds.
When Kyverno receives an admission controller request, i.e. a validation or mutation webhook, it first checks to see if the resource and user information matches or should be excluded from processing. If both checks pass, then the rule logic to mutate, validate, or generate resources is applied.
The following YAML provides an example for a match clause.
````yaml
apiVersion : kyverno.io/v1
kind : ClusterPolicy
metadata :
name : policy
spec :
# 'enforce' to block resource request if any rules fail
# 'audit' to allow resource request on failure of rules, but create policy violations to report them
validationFailureAction: enforce
# Each policy has a list of rules applied in declaration order
rules:
# Rules must have a unique name
- name: "check-pod-controller-labels"
# Each rule matches specific resource described by "match" field.
match:
resources:
kinds: # Required, list of kinds
- Deployment
- StatefulSet
name: "mongo*" # Optional, a resource name is optional. Name supports wildcards (* and ?)
namespaces: # Optional, list of namespaces. Supports wildcards (* and ?)
- "dev*"
- test
selector: # Optional, a resource selector is optional. Values support wildcards (* and ?)
matchLabels:
app: mongodb
matchExpressions:
- {key: tier, operator: In, values: [database]}
# Optional, subjects to be matched
subjects:
- kind: User
name: mary@somecorp.com
# Optional, roles to be matched
roles:
# Optional, clusterroles to be matched
clusterroles: cluster-admin
...
````
All`match` and `exclude` element must be satisfied for the resource to be selected as a candidate for the policy rule. In other words, the match and exclude conditions are evaluated using a logical AND operation.
Here is an example of a rule that matches all pods, excluding pods created by using the `cluster-admin` cluster role.
````yaml
spec:
rules:
name: "match-pods-except-admin"
match:
resources:
kinds:
- Pod
exclude:
clusterroles: cluster-admin
````
This rule that matches all pods, excluding pods in the `kube-system` namespace.
````yaml
spec:
rules:
name: "match-pods-except-admin"
match:
resources:
kinds:
- Pod
exclude:
namespaces:
- "kube-system"
````
---
<small>*Read Next >> [Validate Resources](/documentation/writing-policies-validate.md)*</small>

67
documentation/writing-policies-mutate.md
View file

@ -1,10 +1,10 @@
<small>*[documentation](/README.md#documentation) / [Writing Policies](/documentation/writing-policies.md) / Mutate Resources*</small>
# Mutate Resources
# Mutating Resources
The ```mutate``` rule can be used to add, replace, or delete elements in matching resources. A mutate rule can be written as a JSON Patch or as an overlay.
By using a ```patch``` in the (JSONPatch - RFC 6902)[http://jsonpatch.com/] format, you can make precise changes to the resource being created. Using an ```overlay``` is convenient for describing the desired state of the resource.
By using a ```patch``` in the [JSONPatch - RFC 6902](http://jsonpatch.com/) format, you can make precise changes to the resource being created. Using an ```overlay``` is convenient for describing the desired state of the resource.
Resource mutation occurs before validation, so the validation rules should not contradict the changes performed by the mutation section.
@ -20,38 +20,41 @@ A JSON Patch rule provides an alternate way to mutate resources.
With Kyverno, the add and replace have the same behavior i.e. both operations will add or replace the target element.
This patch adds an init container to all deployments.
This patch policy adds, or replaces, entries in a `ConfigMap` with the name `config-game` in any namespace.
````yaml
apiVersion : kyverno.io/v1
kind : ClusterPolicy
metadata :
name : policy-v1
name : policy-generate-cm
spec :
rules:
- name: "add-init-secrets"
- name: pCM1
match:
resources:
kinds:
- Deployment
name: "config-game"
kinds :
- ConfigMap
mutate:
overlay:
spec:
template:
spec:
initContainers:
- name: init-secrets
image: nirmata.io/kube-vault-client:v2
patches:
- path: "/data/ship.properties"
op: add
value: |
type=starship
owner=utany.corp
- path : "/data/newKey1"
op : add
value : newValue1
````
Here is the example of a patch that removes a label from the secret:
````yaml
apiVersion : kyverno.io/v1
kind : ClusterPolicy
metadata :
name : policy-remove-label
spec :
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: policy-remove-label
spec:
rules:
- name: "Remove unwanted label"
match:
@ -72,14 +75,14 @@ A mutation overlay describes the desired form of resource. The existing resource
The overlay cannot be used to delete values in a resource: use **patches** for this purpose.
The following mutation overlay will add (or replace) the memory request and limit to 10Gi for every Pod with a label ```memory: high```:
The following mutation overlay will add (or replace) the memory request and limit to 10Gi for every Pod with a label `memory: high`:
````yaml
apiVersion : kyverno.io/v1
kind : ClusterPolicy
metadata :
name : policy-change-memory-limit
spec :
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: policy-change-memory-limit
spec:
rules:
- name: "Set hard memory limit to 2Gi"
match:
@ -117,7 +120,7 @@ spec:
- name: "Add IP to subsets"
match:
resources:
kinds :
kinds:
- Endpoints
mutate:
overlay:
@ -148,14 +151,14 @@ The **anchors** values support **wildcards**:
A `conditional anchor` evaluates to `true` if the anchor tag exists and if the value matches the specified value. Processing stops if a tag does not exist or when the value does not match. Once processing stops, any child elements or any remaining siblings in a list, will not be processed.
For example, this overlay will add or replace the value 6443 for the port field, for all ports with a name value that starts with "secure":
For example, this overlay will add or replace the value `6443` for the `port` field, for all ports with a name value that starts with "secure":
````yaml
apiVersion: kyverno.io/v1
kind : ClusterPolicy
metadata :
name : policy-set-port
spec :
kind: ClusterPolicy
metadata:
name: policy-set-port
spec:
rules:
- name: "Set port"
match:
@ -174,7 +177,7 @@ If the anchor tag value is an object or array, the entire object or array must m
### Add if not present anchor
A variation of an anchor, is to add a field value if it is not already defined. This is done by using the `add anchor` (short for `add if not present anchor`) with the notation ````+(...)```` for the tag.
A variation of an anchor, is to add a field value if it is not already defined. This is done by using the `add anchor` (short for `add if not present anchor`) with the notation `+(...)` for the tag.
An `add anchor` is processed as part of applying the mutation. Typically, every non-anchor tag-value is applied as part of the mutation. If the `add anchor` is set on a tag, the tag and value are only applied if they do not exist in the resource.

80
documentation/writing-policies-validate.md
View file

@ -1,9 +1,15 @@
<small>*[documentation](/README.md#documentation) / [Writing Policies](/documentation/writing-policies.md) / Validate Resources*</small>
# Validate Resources
# Validating Resources and Requests
A validation rule is expressed as an overlay pattern that expresses the desired configuration. Resource configurations must match fields and expressions defined in the pattern to pass the validation rule. The following rules are followed when processing the overlay pattern:
A validation rule can be used to validate resources or to deny API requests based on other information.
To validate resource data, define a [pattern](#patterns) in the validation rule. To deny certain API requests define a [deny](#deny-rules) element in the validation rule along a set of conditions that control when to allow or deny the request.
## Patterns
A validation rule that checks resource data is defined as an overlay pattern that provides the desired configuration. Resource configurations must match fields and expressions defined in the pattern to pass the validation rule. The following rules are followed when processing the overlay pattern:
1. Validation will fail if a field is defined in the pattern and if the field does not exist in the configuration.
2. Undefined fields are treated as wildcards.
@ -14,7 +20,6 @@ A validation rule is expressed as an overlay pattern that expresses the desired
7. The validation of siblings is performed only when one of the field values matches the value defined in the pattern. You can use the parenthesis operator to explictly specify a field value that must be matched. This allows writing rules like 'if fieldA equals X, then fieldB must equal Y'.
8. Validation of child values is only performed if the parent matches the pattern.
## Patterns
### Wildcards
1. `*` - matches zero or more alphanumeric characters
@ -33,7 +38,7 @@ A validation rule is expressed as an overlay pattern that expresses the desired
There is no operator for `equals` as providing a field value in the pattern requires equality to the value.
## Anchors
### Anchors
Anchors allow conditional processing (i.e. "if-then-else) and other logical checks in validation patterns. The following types of anchors are supported:
@ -45,7 +50,7 @@ Anchors allow conditional processing (i.e. "if-then-else) and other logical chec
| Existence | ^() | Works on the list/array type only. If at least one element in the list satisfies the pattern. In contrast, a conditional anchor would validate that all elements in the list match the pattern. <br/>e.g. At least one container with image nginx:latest must exist. <br/>&nbsp;&nbsp;&nbsp;&nbsp;^(containers):<br/>&nbsp;&nbsp;&nbsp;&nbsp;- image: nginx:latest<br/> |
| Negation | X() | The tag cannot be specified. The value of the tag is not evaulated. <br/>e.g. Hostpath tag cannot be defined.<br/>&nbsp;&nbsp;&nbsp;&nbsp;X(hostPath):<br/> |
## Anchors and child elements
### Anchors and child elements
Child elements are handled differently for conditional and equality anchors.
@ -77,17 +82,17 @@ For equality anchors, a child element is considered to be part of the "then" cla
This is read as "If a hostPath volume exists, then the path must not be equal to /var/run/docker.sock".
## Examples
### Validation Pattern Examples
The following rule prevents the creation of Deployment, StatefuleSet and DaemonSet resources without label 'app' in selector:
````yaml
apiVersion : kyverno.io/v1
kind : ClusterPolicy
metadata :
name : validation-example
spec :
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: validation-example
spec:
rules:
- name: check-label
match:
@ -113,18 +118,18 @@ spec :
````
### Existence anchor: at least one
#### Existence anchor: at least one
A variation of an anchor, is to check that in a list of elements at least one element exists that matches the patterm. This is done by using the ^(...) notation for the field.
For example, this pattern will check that at least one container has memory requests and limits defined and that the request is less than the limit:
````yaml
apiVersion : kyverno.io/v1
kind : ClusterPolicy
metadata :
name : validation-example2
spec :
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: validation-example2
spec:
rules:
- name: check-memory_requests_link_in_yaml_relative
match:
@ -147,7 +152,7 @@ spec :
memory: "2048Mi"
````
### Logical OR across validation patterns
#### Logical OR across validation patterns
In some cases content can be defined at a different level. For example, a security context can be defined at the Pod or Container level. The validation rule should pass if either one of the conditions is met.
@ -190,5 +195,44 @@ Additional examples are available in [samples](/samples/README.md)
The `validationFailureAction` attribute controls processing behaviors when the resource is not compliant with the policy. If the value is set to `enforce` resource creation or updates are blocked when the resource does not comply, and when the value is set to `audit` a policy violation is reported but the resource creation or update is allowed.
## Deny rules
In addition to applying patterns to check resources, a validate rule can `deny` a request based on a set of conditions. This is useful for applying fine grained access controls that cannot be performed using Kubernetes RBAC.
For example, the policy below denies `delete requests` for objects with the label `app.kubernetes.io/managed-by: kyverno` and for all users who do not have the `cluster-admin` role.
As the example shows, you can use `match` and `exclude` to select when the rule should be applied and then use additional conditions in the `deny` declaration to apply fine-grained controls.
Note that the `validationFailureAction` must be set to `enforce` to block the request.
```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: multi-tenancy
spec:
validationFailureAction: enforce
background: false
rules:
- name: block-deletes-for-kyverno-resources
match:
resources:
selector:
matchLabels:
app.kubernetes.io/managed-by: kyverno
exclude:
clusterRoles:
- cluster-admin
validate:
message: "Deleting {{request.oldObject.kind}}/{{request.oldObject.metadata.name}} is not allowed"
deny:
conditions:
- key: "{{request.operation}}"
operator: Equals
value: "DELETE"
```
Learn more about using [variables](writing-policies-variables.md) and [conditions](writing-policies-preconditions.md) in upcoming sections.
---
<small>*Read Next >> [Mutate Resources](/documentation/writing-policies-mutate.md)*</small>

4
documentation/writing-policies-variables.md
View file

@ -14,9 +14,9 @@ The following data is available for use in context:
Kyverno automatically creates a few useful variables:
- `serviceAccountName` : the last part of a service account i.e. without the suffix `system:serviceaccount:<namespace>:` and stores the userName. For example, when processing a request from `system:serviceaccount:nirmata:user1` Kyverno will store the value `user1` in the variable `serviceAccountName`.
- `serviceAccountName` : the "userName" which is last part of a service account i.e. without the prefix `system:serviceaccount:<namespace>:`. For example, when processing a request from `system:serviceaccount:nirmata:user1` Kyverno will store the value `user1` in the variable `serviceAccountName`.
- `serviceAccountNamespace` : the `namespace` portion of the serviceAccount. For example, when processing a request from `system:serviceaccount:nirmata:user1` Kyverno will store `nirmata` in the variable `serviceAccountNamespace`.
- `serviceAccountNamespace` : the "namespace" part of the serviceAccount. For example, when processing a request from `system:serviceaccount:nirmata:user1` Kyverno will store `nirmata` in the variable `serviceAccountNamespace`.
## Examples

56
documentation/writing-policies.md
View file

@ -8,59 +8,9 @@ The following picture shows the structure of a Kyverno Policy:
Each Kyverno policy contains one or more rules. Each rule has a `match` clause, an optional `exclude` clause, and one of a `mutate`, `validate`, or `generate` clause.
The match / exclude clauses have the same structure, and can contain the following elements:
* resources: select resources by name, namespaces, kinds, and label selectors.
* subjects: select users, user groups, and service accounts
* roles: select namespaced roles
* clusterroles: select cluster wide roles
When Kyverno receives an admission controller request, i.e. a validation or mutation webhook, it first checks to see if the resource and user information matches or should be excluded from processing. If both checks pass, then the rule logic to mutate, validate, or generate resources is applied.
The following YAML provides an example for a match clause.
````yaml
apiVersion : kyverno.io/v1
kind : ClusterPolicy
metadata :
name : policy
spec :
# 'enforce' to block resource request if any rules fail
# 'audit' to allow resource request on failure of rules, but create policy violations to report them
validationFailureAction: enforce
# Each policy has a list of rules applied in declaration order
rules:
# Rules must have a unique name
- name: "check-pod-controller-labels"
# Each rule matches specific resource described by "match" field.
match:
resources:
kinds: # Required, list of kinds
- Deployment
- StatefulSet
name: "mongo*" # Optional, a resource name is optional. Name supports wildcards (* and ?)
namespaces: # Optional, list of namespaces. Supports wildcards (* and ?)
- "dev*"
- test
selector: # Optional, a resource selector is optional. Values support wildcards (* and ?)
matchLabels:
app: mongodb
matchExpressions:
- {key: tier, operator: In, values: [database]}
# Optional, subjects to be matched
subjects:
- kind: User
name: mary@somecorp.com
# Optional, roles to be matched
roles:
# Optional, clusterroles to be matched
clusterroles: cluster-admin
...
````
Each rule can validate, mutate, or generate configurations of matching resources. A rule definition can contain only a single **mutate**, **validate**, or **generate** child node. These actions are applied to the resource in described order: mutation, validation and then generation.
Each rule can validate, mutate, or generate configurations of matching resources. A rule definition can contain only a single **mutate**, **validate**, or **generate** child node.
These actions are applied to the resource in described order: mutation, validation and then generation.
---
<small>*Read Next >> [Validate Resources](/documentation/writing-policies-validate.md)*</small>
<small>*Read Next >> [Selecting Resources](/documentation/writing-policies-match-exclude.md)*</small>

35
go.mod
View file

@ -5,39 +5,40 @@ go 1.13
require (
github.com/cenkalti/backoff v2.2.1+incompatible
github.com/evanphx/json-patch v4.5.0+incompatible
github.com/gogo/protobuf v1.3.1 // indirect
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/go-logr/logr v0.1.0
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7 // indirect
github.com/googleapis/gnostic v0.3.1
github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 // indirect
github.com/hashicorp/golang-lru v0.5.3 // indirect
github.com/imdario/mergo v0.3.8 // indirect
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
github.com/json-iterator/go v1.1.9 // indirect
github.com/julienschmidt/httprouter v1.3.0
github.com/minio/minio v0.0.0-20200114012931-30922148fbb5
github.com/ory/go-acc v0.1.0 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/ory/go-acc v0.2.1 // indirect
github.com/rogpeppe/godef v1.1.2 // indirect
github.com/spf13/cobra v0.0.5
github.com/spf13/pflag v1.0.5 // indirect
github.com/tevino/abool v0.0.0-20170917061928-9b9efcf221b5
golang.org/x/crypto v0.0.0-20200109152110-61a87790db17 // indirect
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 // indirect
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d // indirect
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1 // indirect
golang.org/x/time v0.0.0-20191024005414-555d28b269f0 // indirect
google.golang.org/appengine v1.6.5 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.2.7
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/yaml.v2 v2.2.8
gotest.tools v2.2.0+incompatible
k8s.io/api v0.0.0-20190409021203-6e4e0e4f393b
k8s.io/apimachinery v0.0.0-20190404173353-6a84e37a896d
k8s.io/cli-runtime v0.0.0-20191004110135-b9eb767d2e1a
k8s.io/client-go v11.0.1-0.20190516230509-ae8359b20417+incompatible
k8s.io/klog v1.0.0 // indirect
k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
k8s.io/api v0.17.4
k8s.io/apimachinery v0.17.4
k8s.io/cli-runtime v0.17.4
k8s.io/client-go v0.17.4
k8s.io/klog v1.0.0
k8s.io/kube-openapi v0.0.0-20200121204235-bf4fb3bd569c
k8s.io/utils v0.0.0-20200109141947-94aeca20bf09 // indirect
sigs.k8s.io/kustomize v2.0.3+incompatible // indirect
sigs.k8s.io/controller-runtime v0.5.0
)
// Added for go1.13 migration https://github.com/golang/go/issues/32805
replace github.com/gorilla/rpc v1.2.0+incompatible => github.com/gorilla/rpc v1.2.0
replace (
github.com/gorilla/rpc v1.2.0+incompatible => github.com/gorilla/rpc v1.2.0
k8s.io/code-generator => k8s.io/code-generator v0.0.0-20200306081859-6a048a382944
k8s.io/component-base => k8s.io/component-base v0.0.0-20190612130303-4062e14deebe
)

202
go.sum
View file

@ -1,3 +1,5 @@
9fans.net/go v0.0.0-20181112161441-237454027057 h1:OcHlKWkAMJEF1ndWLGxp5dnJQkYM/YImUOvsBoz6h5E=
9fans.net/go v0.0.0-20181112161441-237454027057/go.mod h1:diCsxrliIURU9xsYtjCp5AbpQKqdhKmf0ujWDUSkfoY=
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
@ -15,6 +17,13 @@ github.com/Azure/azure-pipeline-go v0.2.1/go.mod h1:UGSo8XybXnIGZ3epmeBw7Jdz+HiU
github.com/Azure/azure-storage-blob-go v0.8.0/go.mod h1:lPI3aLPpuLTeUwh1sViKXFxwl2B6teiRqI0deQUvsw0=
github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8=
github.com/Azure/go-autorest v11.7.1+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
github.com/Azure/go-autorest/autorest v0.9.0/go.mod h1:xyHB1BMZT0cuDHU7I0+g046+BFDTQ8rEZB0s4Yfa6bI=
github.com/Azure/go-autorest/autorest/adal v0.5.0/go.mod h1:8Z9fGy2MpX0PvDjB1pEgQTmVqjGhiHBW7RJJEciWzS0=
github.com/Azure/go-autorest/autorest/date v0.1.0/go.mod h1:plvfp3oPSKwf2DNjlBjWF/7vwR+cUD/ELuzDCXwHUVA=
github.com/Azure/go-autorest/autorest/mocks v0.1.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0=
github.com/Azure/go-autorest/autorest/mocks v0.2.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0=
github.com/Azure/go-autorest/logger v0.1.0/go.mod h1:oExouG+K6PryycPJfVSxi/koC6LSNgds39diKLz7Vrc=
github.com/Azure/go-autorest/tracing v0.5.0/go.mod h1:r/s2XiOKccPW3HrqB+W0TQzfbtp2fGCgRFtBroKn4Dk=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/DATA-DOG/go-sqlmock v1.3.3/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM=
@ -26,21 +35,29 @@ github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5/go.mod h1:lmUJ/7eu/Q8
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/PuerkitoBio/purell v1.0.0 h1:0GoNN3taZV6QI81IXgCbxMyEaJDXMSIjArYBCYzVVvs=
github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
github.com/PuerkitoBio/purell v1.1.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
github.com/PuerkitoBio/purell v1.1.1 h1:WEQqlqaGbrPkxLJWfBwQmfEAE1Z7ONdDLqrN38tNFfI=
github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2 h1:JCHLVE3B+kJde7bIEo5N4J+ZbLhp0J1Fs+ulyRws4gE=
github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 h1:d+Bc7a5rLufV/sSk/8dngufqelfh6jnri85riMAaF/M=
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
github.com/Shopify/sarama v1.24.1/go.mod h1:fGP8eQ6PugKEI0iUETYYtnP6d1pH/bdDMTel1X5ajsU=
github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
github.com/StackExchange/wmi v0.0.0-20190523213315-cbe66965904d/go.mod h1:3eOhrUMpNV+6aFIbp5/iudMxNCF27Vw2OZgy4xEx0Fg=
github.com/agnivade/levenshtein v1.0.1/go.mod h1:CURSv5d9Uaml+FovSIICkLbAUZ9S4RqaHDIsdSBg7lM=
github.com/ajg/form v0.0.0-20160822230020-523a5da1a92f/go.mod h1:uL1WgH+h2mgNtvBq0339dVnzXdBETtL2LeUXaIv25UY=
github.com/alecthomas/participle v0.2.1/go.mod h1:SW6HZGeZgSIpcUWX3fXpfZhuaWHnmoD5KCVaqSaNTkk=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/aliyun/aliyun-oss-go-sdk v0.0.0-20190307165228-86c17b95fcd5/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
github.com/armon/go-metrics v0.0.0-20190430140413-ec5e00d3c878/go.mod h1:3AMJUQhVx52RsWOnlkpikZr01T/yAVN2gn0861vByNg=
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
github.com/aws/aws-sdk-go v1.20.21/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go v1.23.19/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-xray-sdk-go v0.9.4/go.mod h1:XtMKdBQfpVut+tJEwI7+dJFRxxRdxHDyVNp2tHXRq04=
@ -50,6 +67,7 @@ github.com/beevik/ntp v0.2.0/go.mod h1:hIHWr+l3+/clUnF44zdK+CWW7fO8dR5cIylAQ76NR
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
github.com/blang/semver v3.5.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4=
github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps=
github.com/cenkalti/backoff v2.1.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM=
@ -64,6 +82,7 @@ github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cockroachdb/apd v1.1.0/go.mod h1:8Sl8LxpKi29FqWXR16WEFZRNSz3SoPzUzeMeY4+DwBQ=
github.com/cockroachdb/cockroach-go v0.0.0-20181001143604-e0a95dfd547c/go.mod h1:XGLbWH/ujMcbPbhZq52Nv6UrCghb1yGn//133kEsvDk=
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI=
github.com/codegangsta/negroni v1.0.0/go.mod h1:v0y3T5G7Y1UlFfyxFn/QLRU4a2EuNau2iZY63YTKWo0=
github.com/containerd/continuity v0.0.0-20181203112020-004b46473808/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
@ -73,11 +92,16 @@ github.com/coreos/bbolt v1.3.3/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkE
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/etcd v3.3.12+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/coreos/pkg v0.0.0-20180108230652-97fdf19511ea/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/davecgh/go-spew v0.0.0-20151105211317-5215b55f46b2/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
@ -85,9 +109,13 @@ github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
github.com/djherbis/atime v1.0.0/go.mod h1:5W+KBIuTwVGcqjIfaTwt+KSYX1o6uep8dtevevQP/f8=
github.com/docker/docker v0.7.3-0.20190327010347-be7ac8be2ae0/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-units v0.3.3/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96/go.mod h1:Qh8CwZgvJUkLughtfhJv5dyTYa91l1fOUCrgjqmcifM=
github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/dustin/go-humanize v0.0.0-20180713052910-9f541cc9db5d/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/eapache/go-resiliency v1.1.0/go.mod h1:kFI+JgMyC7bLPUVY133qvEBtVayf5mFgVsvEsIPBvNs=
@ -96,9 +124,14 @@ github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21/go.mod h1
github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
github.com/eclipse/paho.mqtt.golang v1.2.0/go.mod h1:H9keYFcgq3Qr5OUJm/JZI/i6U7joQ8SYLhZwfeOo6Ts=
github.com/elazarl/go-bindata-assetfs v1.0.0/go.mod h1:v+YaWX3bdea5J/mo8dSETolEo7R71Vk1u8bnjau5yw4=
github.com/elazarl/goproxy v0.0.0-20170405201442-c4fc26588b6e/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc=
github.com/elazarl/goproxy v0.0.0-20181003060214-f58a169a71a5/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc=
github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633 h1:H2pdYOb3KQ1/YsqVWoWNLQO+fusocsw354rqGTZtAgw=
github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
github.com/emicklei/go-restful v2.9.5+incompatible h1:spTtZBk5DYEvbxMVutUuTyh1Ao2r4iyvLdACqsl/Ljk=
github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
github.com/evanphx/json-patch v0.0.0-20190203023257-5858425f7550/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch v4.5.0+incompatible h1:ouOWdg56aJriqS0huScTkVXPC5IcNrDCXZ6OoTAWu7M=
github.com/evanphx/json-patch v4.5.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
@ -113,21 +146,68 @@ github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2H
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32 h1:Mn26/9ZMNWSw9C9ERFA1PUxfmGpolnw2v0bKOREu5ew=
github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32/go.mod h1:GIjDIg/heH5DOkXY3YJ/wNhfHsQHoXGjl8G8amsYQ1I=
github.com/globalsign/mgo v0.0.0-20180905125535-1ca0a4f7cbcb/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q=
github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q=
github.com/go-bindata/go-bindata v3.1.1+incompatible/go.mod h1:xK8Dsgwmeed+BBsSy2XTopBn/8uK2HWuGSnA11C3Joo=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-ldap/ldap v3.0.2+incompatible/go.mod h1:qfd9rJvER9Q0/D/Sqn1DfHRoBp40uXYvFoEVrNEPqRc=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-logr/logr v0.1.0 h1:M1Tv3VzNlEHg6uyACnRdtrploV2P7wZqH8BoQMtz0cg=
github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas=
github.com/go-logr/zapr v0.1.0/go.mod h1:tabnROwaDl0UNxkVeFRbY8bwB37GwRv0P8lg6aAiEnk=
github.com/go-ole/go-ole v1.2.4/go.mod h1:XCwSNxSkXRo4vlyPy93sltvi/qJq0jqQhjqQNIwKuxM=
github.com/go-openapi/analysis v0.0.0-20180825180245-b006789cd277/go.mod h1:k70tL6pCuVxPJOHXQ+wIac1FUrvNkHolPie/cLEU6hI=
github.com/go-openapi/analysis v0.17.0/go.mod h1:IowGgpVeD0vNm45So8nr+IcQ3pxVtpRoBWb8PVZO0ik=
github.com/go-openapi/analysis v0.18.0/go.mod h1:IowGgpVeD0vNm45So8nr+IcQ3pxVtpRoBWb8PVZO0ik=
github.com/go-openapi/analysis v0.19.2/go.mod h1:3P1osvZa9jKjb8ed2TPng3f0i/UY9snX6gxi44djMjk=
github.com/go-openapi/analysis v0.19.5/go.mod h1:hkEAkxagaIvIP7VTn8ygJNkd4kAYON2rCu0v0ObL0AU=
github.com/go-openapi/errors v0.17.0/go.mod h1:LcZQpmvG4wyF5j4IhA73wkLFQg+QJXOQHVjmcZxhka0=
github.com/go-openapi/errors v0.18.0/go.mod h1:LcZQpmvG4wyF5j4IhA73wkLFQg+QJXOQHVjmcZxhka0=
github.com/go-openapi/errors v0.19.2/go.mod h1:qX0BLWsyaKfvhluLejVpVNwNRdXZhEbTA4kxxpKBC94=
github.com/go-openapi/jsonpointer v0.0.0-20160704185906-46af16f9f7b1 h1:wSt/4CYxs70xbATrGXhokKF1i0tZjENLOo1ioIO13zk=
github.com/go-openapi/jsonpointer v0.0.0-20160704185906-46af16f9f7b1/go.mod h1:+35s3my2LFTysnkMfxsJBAMHj/DoqoB9knIWoYG/Vk0=
github.com/go-openapi/jsonpointer v0.17.0/go.mod h1:cOnomiV+CVVwFLk0A/MExoFMjwdsUdVpsRhURCKh+3M=
github.com/go-openapi/jsonpointer v0.18.0/go.mod h1:cOnomiV+CVVwFLk0A/MExoFMjwdsUdVpsRhURCKh+3M=
github.com/go-openapi/jsonpointer v0.19.2/go.mod h1:3akKfEdA7DF1sugOqz1dVQHBcuDBPKZGEoHC/NkiQRg=
github.com/go-openapi/jsonpointer v0.19.3 h1:gihV7YNZK1iK6Tgwwsxo2rJbD1GTbdm72325Bq8FI3w=
github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
github.com/go-openapi/jsonreference v0.0.0-20160704190145-13c6e3589ad9 h1:tF+augKRWlWx0J0B7ZyyKSiTyV6E1zZe+7b3qQlcEf8=
github.com/go-openapi/jsonreference v0.0.0-20160704190145-13c6e3589ad9/go.mod h1:W3Z9FmVs9qj+KR4zFKmDPGiLdk1D9Rlm7cyMvf57TTg=
github.com/go-openapi/jsonreference v0.17.0/go.mod h1:g4xxGn04lDIRh0GJb5QlpE3HfopLOL6uZrK/VgnsK9I=
github.com/go-openapi/jsonreference v0.18.0/go.mod h1:g4xxGn04lDIRh0GJb5QlpE3HfopLOL6uZrK/VgnsK9I=
github.com/go-openapi/jsonreference v0.19.2/go.mod h1:jMjeRr2HHw6nAVajTXJ4eiUwohSTlpa0o73RUL1owJc=
github.com/go-openapi/jsonreference v0.19.3 h1:5cxNfTy0UVC3X8JL5ymxzyoUZmo8iZb+jeTWn7tUa8o=
github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8=
github.com/go-openapi/loads v0.17.0/go.mod h1:72tmFy5wsWx89uEVddd0RjRWPZm92WRLhf7AC+0+OOU=
github.com/go-openapi/loads v0.18.0/go.mod h1:72tmFy5wsWx89uEVddd0RjRWPZm92WRLhf7AC+0+OOU=
github.com/go-openapi/loads v0.19.0/go.mod h1:72tmFy5wsWx89uEVddd0RjRWPZm92WRLhf7AC+0+OOU=
github.com/go-openapi/loads v0.19.2/go.mod h1:QAskZPMX5V0C2gvfkGZzJlINuP7Hx/4+ix5jWFxsNPs=
github.com/go-openapi/loads v0.19.4/go.mod h1:zZVHonKd8DXyxyw4yfnVjPzBjIQcLt0CCsn0N0ZrQsk=
github.com/go-openapi/runtime v0.0.0-20180920151709-4f900dc2ade9/go.mod h1:6v9a6LTXWQCdL8k1AO3cvqx5OtZY/Y9wKTgaoP6YRfA=
github.com/go-openapi/runtime v0.19.0/go.mod h1:OwNfisksmmaZse4+gpV3Ne9AyMOlP1lt4sK4FXt0O64=
github.com/go-openapi/runtime v0.19.4/go.mod h1:X277bwSUBxVlCYR3r7xgZZGKVvBd/29gLDlFGtJ8NL4=
github.com/go-openapi/spec v0.0.0-20160808142527-6aced65f8501 h1:C1JKChikHGpXwT5UQDFaryIpDtyyGL/CR6C2kB7F1oc=
github.com/go-openapi/spec v0.0.0-20160808142527-6aced65f8501/go.mod h1:J8+jY1nAiCcj+friV/PDoE1/3eeccG9LYBs0tYvLOWc=
github.com/go-openapi/spec v0.17.0/go.mod h1:XkF/MOi14NmjsfZ8VtAKf8pIlbZzyoTvZsdfssdxcBI=
github.com/go-openapi/spec v0.18.0/go.mod h1:XkF/MOi14NmjsfZ8VtAKf8pIlbZzyoTvZsdfssdxcBI=
github.com/go-openapi/spec v0.19.2/go.mod h1:sCxk3jxKgioEJikev4fgkNmwS+3kuYdJtcsZsD5zxMY=
github.com/go-openapi/spec v0.19.3 h1:0XRyw8kguri6Yw4SxhsQA/atC88yqrk0+G4YhI2wabc=
github.com/go-openapi/spec v0.19.3/go.mod h1:FpwSN1ksY1eteniUU7X0N/BgJ7a4WvBFVA8Lj9mJglo=
github.com/go-openapi/strfmt v0.17.0/go.mod h1:P82hnJI0CXkErkXi8IKjPbNBM6lV6+5pLP5l494TcyU=
github.com/go-openapi/strfmt v0.18.0/go.mod h1:P82hnJI0CXkErkXi8IKjPbNBM6lV6+5pLP5l494TcyU=
github.com/go-openapi/strfmt v0.19.0/go.mod h1:+uW+93UVvGGq2qGaZxdDeJqSAqBqBdl+ZPMF/cC8nDY=
github.com/go-openapi/strfmt v0.19.3/go.mod h1:0yX7dbo8mKIvc3XSKp7MNfxw4JytCfCD6+bY1AVL9LU=
github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87 h1:zP3nY8Tk2E6RTkqGYrarZXuzh+ffyLDljLxCy1iJw80=
github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87/go.mod h1:DXUve3Dpr1UfpPtxFw+EFuQ41HhCWZfha5jSVRG7C7I=
github.com/go-openapi/swag v0.17.0/go.mod h1:AByQ+nYG6gQg71GINrmuDXCPWdL640yX49/kXLo40Tg=
github.com/go-openapi/swag v0.18.0/go.mod h1:AByQ+nYG6gQg71GINrmuDXCPWdL640yX49/kXLo40Tg=
github.com/go-openapi/swag v0.19.2/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/go-openapi/swag v0.19.5 h1:lTz6Ys4CmqqCQmZPBlbQENR1/GucA2bzYTE12Pw4tFY=
github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/go-openapi/validate v0.18.0/go.mod h1:Uh4HdOzKt19xGIGm1qHf/ofbX1YQ4Y+MYsct2VUrAJ4=
github.com/go-openapi/validate v0.19.2/go.mod h1:1tRCw7m3jtI8eNWEEliiAqUIcBztB2KDnRCRMUi7GTA=
github.com/go-openapi/validate v0.19.5/go.mod h1:8DJv2CVJQ6kGNpFW6eV9N3JviE1C85nY1c2z52x1Gk4=
github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
@ -301,14 +381,18 @@ github.com/gobuffalo/x v0.0.0-20181003152136-452098b06085/go.mod h1:WevpGD+5YOre
github.com/gobuffalo/x v0.0.0-20181007152206-913e47c59ca7/go.mod h1:9rDPXaB3kXdKWzMc4odGQQdG2e2DIEmANy5aSJ9yesY=
github.com/gofrs/uuid v3.1.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gogo/protobuf v0.0.0-20171007142547-342cbe0a0415/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/gogo/protobuf v1.2.2-0.20190723190241-65acae22fc9d/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls=
github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/golang/gddo v0.0.0-20180828051604-96d2a289f41e/go.mod h1:xEhNfoBDX1hzLm2Nf80qUvZ2sVwoMZ8d6IE2SrsQfh4=
github.com/golang/gddo v0.0.0-20190904175337-72a348e765d2/go.mod h1:xEhNfoBDX1hzLm2Nf80qUvZ2sVwoMZ8d6IE2SrsQfh4=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20180513044358-24b0969c4cb7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7 h1:5ZkaAPbicIKTF2I64qf5Fh8Aa83Q/dnOafMYV0OMwjA=
@ -333,6 +417,7 @@ github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMyw
github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/gofuzz v0.0.0-20161122191042-44d81051d367/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI=
github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI=
github.com/google/gofuzz v1.0.0 h1:A8PeW59pxE9IoFRqBp37U+mSNaQoZ46F1f0f863XSXw=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
@ -341,11 +426,14 @@ github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OI
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.0 h1:Jf4mxPC/ziBnoPIdpQdPJ9OeiomAUHLvxmPRSPH9m4s=
github.com/google/uuid v1.1.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY=
github.com/googleapis/gnostic v0.3.1 h1:WeAefnSUHlBb0iJKwxFDZdbfGwkd7xRNuV+IpXMJhYk=
github.com/googleapis/gnostic v0.3.1/go.mod h1:on+2t9HRStVgn95RSsFWFz+6Q0Snyqv1awfrALZdbtU=
github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
github.com/gopherjs/gopherjs v0.0.0-20181004151105-1babbf986f6f/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gopherjs/gopherjs v0.0.0-20190328170749-bb2674552d8f/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
@ -358,9 +446,12 @@ github.com/gorilla/rpc v1.2.0/go.mod h1:V4h9r+4sF5HnzqbwIez0fKSpANP0zlYd3qR7p36j
github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=
github.com/gorilla/sessions v1.1.2/go.mod h1:8KCfur6+4Mqcc6S0FEfKuN15Vl5MgXW92AE8ovaJD0w=
github.com/gorilla/sessions v1.1.3/go.mod h1:8KCfur6+4Mqcc6S0FEfKuN15Vl5MgXW92AE8ovaJD0w=
github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gotestyourself/gotestyourself v2.2.0+incompatible/go.mod h1:zZKM6oeNM8k+FRljX1mnzVYeS8wiGgQyvST1/GafPbY=
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 h1:pdN6V1QBWetyv/0+wjACpqVH+eVULgEjkurDLq3goeM=
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 h1:+ngKgrYPPJrOjhax5N+uePQ0Fh1Z7PheYoUI/0nzkPA=
github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
@ -368,6 +459,7 @@ github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
github.com/grpc-ecosystem/grpc-gateway v1.8.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
@ -399,6 +491,8 @@ github.com/hashicorp/vault/sdk v0.1.13/go.mod h1:B+hVj7TpuQY1Y/GPbCpffmgd+tSEwvh
github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb/go.mod h1:+NfK9FKeTrX5uv1uIXGdwYDTeHna2qgaIlx54MXqjAM=
github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d/go.mod h1:+NfK9FKeTrX5uv1uIXGdwYDTeHna2qgaIlx54MXqjAM=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.8 h1:CGgOkSJeqMRmt0D9XLWExdT4m4F1vd3FV3VPt+0VxkQ=
github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/inconshreveable/go-update v0.0.0-20160112193335-8152e7eb6ccf/go.mod h1:hyb9oH7vZsitZCiBt0ZvifOrB+qc8PS5IiilCIb87rg=
@ -415,13 +509,18 @@ github.com/joho/godotenv v1.2.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqx
github.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg=
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
github.com/json-iterator/go v0.0.0-20180612202835-f2b4162afba3/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v0.0.0-20180701071628-ab8a2e0c74be/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.9 h1:9yzud/Ht36ygwatGx56VwCZtlI/2AD15T1X2sjSuGns=
github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jtolds/gls v4.2.1+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0 h1:U0609e9tgbseu3rBINet9P48AI/D3oJs4dN7jwJOQ1U=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/karrick/godirwalk v1.7.5/go.mod h1:2c9FRhkDxdIbgkOnCEvnSWs71Bhugbl46shStcFDJ34=
github.com/karrick/godirwalk v1.7.7/go.mod h1:2c9FRhkDxdIbgkOnCEvnSWs71Bhugbl46shStcFDJ34=
github.com/karrick/godirwalk v1.7.8/go.mod h1:2c9FRhkDxdIbgkOnCEvnSWs71Bhugbl46shStcFDJ34=
@ -445,11 +544,14 @@ github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.3/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kurin/blazer v0.5.4-0.20190613185654-cf2f27cc0be3/go.mod h1:4FCXMUWo9DllR2Do4TtBd377ezyAJ51vB5uTBjt0pGU=
github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.1.1/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de h1:9TO3cAIGXtEhnIaL+V+BEER86oLrvS+kWobKpbJuye0=
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de/go.mod h1:zAbeS9B/r2mtpb6U+EI2rYA5OAXxsYw6wTamcNW+zcE=
github.com/luna-duclos/instrumentedsql v1.1.2/go.mod h1:4LGbEqDnopzNAiyxPPDXhLspyunZxgPTMJBKtC6U0BQ=
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/magiconair/properties v1.8.1 h1:ZC2Vc7/ZFkGmsVC9KvOjumD+G5lXy2RtTKyzRKO2BQ4=
@ -457,6 +559,12 @@ github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czP
github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.0.0-20180730094502-03f2033d19d5 h1:0x4qcEHDpruK6ML/m/YSlFUUu0UpRD3I2PHsNCuGnyA=
github.com/mailru/easyjson v0.0.0-20180730094502-03f2033d19d5/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.0.0-20190312143242-1de009706dbe/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM=
github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
github.com/markbates/deplist v1.0.4/go.mod h1:gRRbPbbuA8TmMiRvaOzUlRfzfjeCCBqX2A6arxN01MM=
github.com/markbates/deplist v1.0.5/go.mod h1:gRRbPbbuA8TmMiRvaOzUlRfzfjeCCBqX2A6arxN01MM=
github.com/markbates/going v1.0.2/go.mod h1:UWCk3zm0UKefHZ7l8BNqi26UyiEMniznk8naLdTcy6c=
@ -483,6 +591,7 @@ github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNx
github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.5/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.7/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-sqlite3 v1.9.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
github.com/mattn/go-sqlite3 v1.10.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
@ -525,7 +634,9 @@ github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826/go.mod h1:TaXosZuwd
github.com/monoculum/formam v0.0.0-20180901015400-4e68be1d79ba/go.mod h1:RKgILGEJq24YyJ2ban8EO0RUVSJlF1pGsEvoLEACr/Q=
github.com/moul/http2curl v0.0.0-20170919181001-9ac6cf4d929b/go.mod h1:8UbvGypXm98wA/IqH45anm5Y2Z6ep6O31QGOAZ3H0fQ=
github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw=
github.com/nats-io/gnatsd v1.4.1/go.mod h1:nqco77VO78hLCJpIcVfygDP2rPGfsEHkGTUk94uh5DQ=
github.com/nats-io/go-nats v1.7.2/go.mod h1:+t7RHT5ApZebkrQdnn6AhQJmhJJiKAvJUio1PiiCtj0=
github.com/nats-io/go-nats-streaming v0.4.4/go.mod h1:gfq4R3c9sKAINOpelo0gn/b9QDMBZnmrttcsNF+lqyo=
@ -545,15 +656,21 @@ github.com/nsqio/go-nsq v1.0.7/go.mod h1:XP5zaUs3pqf+Q71EqUJs3HYfBIqfK6G83WQMdNN
github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/oleiade/reflections v1.0.0/go.mod h1:RbATFBbKYkVdqmSFtx13Bb/tVhR0lgOBXunWTZKeL4w=
github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c h1:Hww8mOyEKTeON4bZn7FrlLismspbPc1teNRUVH7wLQ8=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c h1:eSfnfIuwhxZyULg1NNuZycJcYkjYVGYe7FczwQReM6U=
github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v0.0.0-20190113212917-5533ce8a0da3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.4.1/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v1.4.2/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.8.1/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoTdcA=
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opencontainers/runc v0.1.1/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
@ -564,6 +681,8 @@ github.com/ory/fosite v0.29.0/go.mod h1:0atSZmXO7CAcs6NPMI/Qtot8tmZYj04Nddoold4S
github.com/ory/go-acc v0.0.0-20181118080137-ddc355013f90/go.mod h1:sxnvPCxChFuSmTJGj8FdMupeq1BezCiEpDjTUXQ4hf4=
github.com/ory/go-acc v0.1.0 h1:ibxdv3n2k4cqh0VXWa9RHARtbR7tcT7jsnSnmGZkDIM=
github.com/ory/go-acc v0.1.0/go.mod h1:0omgy2aa3nDBJ45VAKeLHH8ccPBudxLeic4xiDRtug0=
github.com/ory/go-acc v0.2.1 h1:Pwcmwd/cSnwJsYN76+w3HU7oXeWFTkwj/KUj1qGDrVw=
github.com/ory/go-acc v0.2.1/go.mod h1:0omgy2aa3nDBJ45VAKeLHH8ccPBudxLeic4xiDRtug0=
github.com/ory/go-convenience v0.1.0/go.mod h1:uEY/a60PL5c12nYz4V5cHY03IBmwIAEm8TWB0yn9KNs=
github.com/ory/gojsonreference v0.0.0-20190720135523-6b606c2d8ee8/go.mod h1:wsH1C4nIeeQClDtD5AH7kF1uTS6zWyqfjVDTmB0Em7A=
github.com/ory/gojsonschema v1.1.1-0.20190919112458-f254ca73d5e9/go.mod h1:BNZpdJgB74KOLSsWFvzw6roXg1I6O51WO8roMmW+T7Y=
@ -591,10 +710,12 @@ github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.2/go.mod h1:OsXs2jCmiKlQ1lTBmv21f2mNfw4xf/QclQDMrYNZzcM=
github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
@ -602,10 +723,12 @@ github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7q
github.com/prometheus/common v0.0.0-20181126121408-4724e9255275/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/rcrowley/go-metrics v0.0.0-20190704165056-9c2d0518ed81/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
@ -614,6 +737,8 @@ github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6So
github.com/rogpeppe/go-internal v1.0.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.1.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.2.2/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/godef v1.1.2 h1:c5mCx0EcCORJOdVMREX7Lgh1raTxAHFmOfXdEB9u8Jw=
github.com/rogpeppe/godef v1.1.2/go.mod h1:WtY9A/ovuQ+UakAJ1/CEqwwulX/WJjb2kgkokCHi/GY=
github.com/rs/cors v1.6.0/go.mod h1:gFx+x8UowdsKA9AchylcLynDq+nNFfI8FkUZdN/jGCU=
github.com/rubenv/sql-migrate v0.0.0-20190212093014-1007f53448d7/go.mod h1:WS0rl9eEliYI8DPnr3TOwz4439pay+qNgzJoVya/DmY=
github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
@ -666,6 +791,7 @@ github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb6
github.com/spf13/jwalterweatherman v1.1.0 h1:ue6voC5bR5F8YxI5S67j9i582FU4Qvo2bmqnqMYADFk=
github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=
github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.2/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
@ -677,6 +803,7 @@ github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DM
github.com/streadway/amqp v0.0.0-20190404075320-75d898a42a94/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
github.com/stretchr/testify v0.0.0-20151208002404-e3a8ff8ce365/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
@ -690,6 +817,7 @@ github.com/tidwall/gjson v1.3.5/go.mod h1:P256ACg0Mn+j1RXIDXoss50DeIABTYK1PULOJH
github.com/tidwall/match v1.0.1/go.mod h1:LujAq0jyVjBy028G1WhWfIzbpQfMO8bBZ6Tyb0+pL9E=
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
github.com/tidwall/sjson v1.0.4/go.mod h1:bURseu1nuBkFpIES5cz6zBtjmYeOQmEESshn7VpF15Y=
github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM=
github.com/uber-go/atomic v1.3.2/go.mod h1:/Ct5t2lcmbJ4OSe/waGBoaVvVqtO0bmtfVNex1PFV8g=
@ -701,8 +829,10 @@ github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljT
github.com/ugorji/go/codec v1.1.5-pre/go.mod h1:tULtS6Gy1AE1yCENaw4Vb//HLH5njI2tfCQDUqRd8fI=
github.com/unrolled/secure v0.0.0-20180918153822-f340ee86eb8b/go.mod h1:mnPT77IAdsi/kV7+Es7y+pXALeV3h7G6dQF6mNYjcLA=
github.com/unrolled/secure v0.0.0-20181005190816-ff9db2ff917f/go.mod h1:mnPT77IAdsi/kV7+Es7y+pXALeV3h7G6dQF6mNYjcLA=
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/urfave/negroni v1.0.0/go.mod h1:Meg73S6kFm/4PpbYdq35yYWoCZ9mS/YSx+lKnmiohz4=
github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a/go.mod h1:v3UYOV9WzVtRmSR+PDvWpU/qWl4Wa5LApYYX4ZtKbio=
github.com/vektah/gqlparser v1.1.2/go.mod h1:1ycwN7Ij5njmMkPPAOaRFY4rET2Enx7IkVv3vaXspKw=
github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c/go.mod h1:lB8K/P019DLNhemzwFU4jHLhdvlE6uDZjXFejJXr49I=
github.com/xdg/stringprep v1.0.0/go.mod h1:Jhud4/sHMO4oL310DaZAKk9ZaJ08SJfe+sJh0HrGL1Y=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
@ -712,6 +842,10 @@ github.com/xtgo/uuid v0.0.0-20140804021211-a0b114877d4c/go.mod h1:UrdRz5enIKZ63M
github.com/ziutek/mymysql v1.5.4/go.mod h1:LMSpPZ6DbqWFxNCHW77HeMg9I646SAhApZ/wKdgO/C0=
go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738/go.mod h1:dnLIgRNXwCJa5e+c6mIZCrds/GIG4ncV9HhK5PX7jPg=
go.mongodb.org/mongo-driver v1.0.3/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM=
go.mongodb.org/mongo-driver v1.1.1/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM=
go.mongodb.org/mongo-driver v1.1.2/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.1/go.mod h1:Ap50jQcDJrx6rB6VgeeFPtuPIf3wMRvRfrfYDO6+BmA=
@ -734,15 +868,23 @@ golang.org/x/crypto v0.0.0-20181127143415-eb0de9b17e85/go.mod h1:6SG95UA2DQfeDnf
golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190102171810-8d7daa0c54b3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190103213133-ff983b9c42bc/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190320223903-b7391e95e576/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190404164418-38d8ce5564a5/go.mod h1:WFFai1msRO1wXaEeE5yQxYXgSfI8pQAWXbQop6sCtWE=
golang.org/x/crypto v0.0.0-20190426145343-a29dc8fdc734/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190513172903-22d7a77e9e5f/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190617133340-57b3e21c3d56/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191117063200-497ca9f6d64f/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200109152110-61a87790db17 h1:nVJ3guKA9qdkEQ3TUdXI9QSINo2CUPM/cySEvw2w8I0=
golang.org/x/crypto v0.0.0-20200109152110-61a87790db17/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 h1:/Tl7pH94bvbAAHBdZJT947M/+gp0+CqQXDtMRC0fseo=
golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
@ -752,6 +894,7 @@ golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTk
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180816102801-aaf60122140d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@ -771,16 +914,22 @@ golang.org/x/net v0.0.0-20181207154023-610586996380/go.mod h1:mL1N/T3taQHkDXs73r
golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190125091013-d26f9f9a57f3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190206173232-65e2d4e15006/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190320064053-1272bf9dcd53/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190923162816-aa69164e4478/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191004110552-13f9640d40b9/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 h1:efeOvDhwQ29Dj3SdAV/MJf8oukgn+8D8WgaCaRMchF8=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
@ -822,9 +971,11 @@ golang.org/x/sys v0.0.0-20190102155601-82a175fd1598/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20190116161447-11f53e031339/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190129075346-302c3dd5f1cc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190130150945-aca44879d564/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190209173611-3b5209105503/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190321052220-f7bb7a8bee54/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190403152447-81d4e9dc473e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@ -832,8 +983,10 @@ golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190523142557-0e01d883c5c5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190922100055-0a153f010e69/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191105231009-c1f44814a5cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1 h1:gZpLHxUX5BdYLA08Lj4YCJNN/jk7KtquiArPoeX0WvA=
@ -844,6 +997,7 @@ golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3
golang.org/x/text v0.3.1-0.20181227161524-e6919f6577db/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0 h1:/5xXl8Y5W96D+TtHSlonuFqGHIWVuyCkGJLwGh9JJFs=
@ -875,6 +1029,7 @@ golang.org/x/tools v0.0.0-20190104182027-498d95493402/go.mod h1:n7NCudcB/nEzxVGm
golang.org/x/tools v0.0.0-20190111214448-fc1d57b08d7b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190118193359-16909d206f00/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190125232054-d66bd3c5d5a6/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
@ -884,9 +1039,17 @@ golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBn
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190617190820-da514acc4774/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190624190245-7f2218787638/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190711191110-9a621aea19f8/go.mod h1:jcCCGcm9btYwXyDqrUWc6MKQKKGJCWEQ3AfLSRIbEuI=
golang.org/x/tools v0.0.0-20190920225731-5eefd052ad72/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200226224502-204d844ad48d h1:loGv/4fxITSrCD4t2P8ZF4oUC4RlRFDAsczcoUS2g6c=
golang.org/x/tools v0.0.0-20200226224502-204d844ad48d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gomodules.xyz/jsonpatch/v2 v2.0.1/go.mod h1:IhYNNY4jnS53ZnfE4PAmpKtDpTCj1JFXc+3mwe7XcUU=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.5.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
@ -918,6 +1081,7 @@ google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiq
google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.22.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc/go.mod h1:m7x9LTH6d71AHyAX77c9yqWCCa3UKHcVEj9y7hAtKDk=
@ -925,12 +1089,15 @@ gopkg.in/asn1-ber.v1 v1.0.0-20181015200546-f715ec2f112d/go.mod h1:cuepJuh7vyXfUy
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
gopkg.in/cheggaaa/pb.v1 v1.0.28/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2/go.mod h1:Xk6kEKp8OKb+X14hQBKWaSkCsqBpgog8nAV2xsGOxlo=
gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df/go.mod h1:LRQQ+SO6ZHR7tOkpBDuZnXENFzX8qRjMDMyPD6BRkCw=
gopkg.in/gorp.v1 v1.7.2/go.mod h1:Wo3h+DBQZIxATwftsglhdD/62zRFPhGhTiu5jUJmCaw=
gopkg.in/inf.v0 v0.9.0/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/ini.v1 v1.42.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
@ -942,6 +1109,7 @@ gopkg.in/jcmturner/gokrb5.v7 v7.2.3/go.mod h1:l8VISx+WGYp+Fp7KRbsiUuXTTOnxIc3Tuv
gopkg.in/jcmturner/rpc.v1 v1.1.0/go.mod h1:YIdkC4XfD6GXbzje11McwsDuOlZQSb9W4vfLvuNnlv8=
gopkg.in/ldap.v3 v3.0.3/go.mod h1:oxD7NyBuxchC+SgJDE1Q5Od05eGt29SDQVBmV+HYbzw=
gopkg.in/mail.v2 v2.0.0-20180731213649-a0242b2233b4/go.mod h1:htwXN1Qh09vZJ1NVKxQqHPBaCBbzKhp5GzuJEA4VJWw=
gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k=
gopkg.in/olivere/elastic.v5 v5.0.80/go.mod h1:uhHoB4o3bvX5sorxBU29rPcmBQdV2Qfg0FBrx5D6pV0=
gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
gopkg.in/square/go-jose.v2 v2.1.9/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=
@ -955,6 +1123,8 @@ gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.7 h1:VUgggvou5XRW9mHwD/yXxIYSMtY0zoKQf/v226p2nyo=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
@ -963,24 +1133,56 @@ honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWh
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
k8s.io/api v0.0.0-20190409021203-6e4e0e4f393b h1:aBGgKJUM9Hk/3AE8WaZIApnTxG35kbuQba2w+SXqezo=
k8s.io/api v0.0.0-20190409021203-6e4e0e4f393b/go.mod h1:iuAfoD4hCxJ8Onx9kaTIt30j7jUFS00AXQi6QMi99vA=
k8s.io/api v0.17.2/go.mod h1:BS9fjjLc4CMuqfSO8vgbHPKMt5+SF0ET6u/RVDihTo4=
k8s.io/api v0.17.4 h1:HbwOhDapkguO8lTAE8OX3hdF2qp8GtpC9CW/MQATXXo=
k8s.io/api v0.17.4/go.mod h1:5qxx6vjmwUVG2nHQTKGlLts8Tbok8PzHl4vHtVFuZCA=
k8s.io/apiextensions-apiserver v0.17.2/go.mod h1:4KdMpjkEjjDI2pPfBA15OscyNldHWdBCfsWMDWAmSTs=
k8s.io/apimachinery v0.0.0-20190404173353-6a84e37a896d h1:Jmdtdt1ZnoGfWWIIik61Z7nKYgO3J+swQJtPYsP9wHA=
k8s.io/apimachinery v0.0.0-20190404173353-6a84e37a896d/go.mod h1:ccL7Eh7zubPUSh9A3USN90/OzHNSVN6zxzde07TDCL0=
k8s.io/apimachinery v0.0.0-20190612125636-6a5db36e93ad/go.mod h1:I4A+glKBHiTgiEjQiCCQfCAIcIMFGt291SmsvcrFzJA=
k8s.io/apimachinery v0.17.2/go.mod h1:b9qmWdKlLuU9EBh+06BtLcSf/Mu89rWL33naRxs1uZg=
k8s.io/apimachinery v0.17.4 h1:UzM+38cPUJnzqSQ+E1PY4YxMHIzQyCg29LOoGfo79Zw=
k8s.io/apimachinery v0.17.4/go.mod h1:gxLnyZcGNdZTCLnq3fgzyg2A5BVCHTNDFrw8AmuJ+0g=
k8s.io/apimachinery v0.18.3 h1:pOGcbVAhxADgUYnjS08EFXs9QMl8qaH5U4fr5LGUrSk=
k8s.io/apiserver v0.17.2/go.mod h1:lBmw/TtQdtxvrTk0e2cgtOxHizXI+d0mmGQURIHQZlo=
k8s.io/cli-runtime v0.0.0-20191004110135-b9eb767d2e1a h1:REMzGxu+NpG9dPRsE9my/fw9iYIecz1S8UFFl6hbe18=
k8s.io/cli-runtime v0.0.0-20191004110135-b9eb767d2e1a/go.mod h1:qWnH3/b8sp/l7EvlDh7ulDU3UWA4P4N1NFbEEP791tM=
k8s.io/cli-runtime v0.17.4 h1:ZIJdxpBEszZqUhydrCoiI5rLXS2J/1AF5xFok2QJ9bc=
k8s.io/cli-runtime v0.17.4/go.mod h1:IVW4zrKKx/8gBgNNkhiUIc7nZbVVNhc1+HcQh+PiNHc=
k8s.io/client-go v0.17.2/go.mod h1:QAzRgsa0C2xl4/eVpeVAZMvikCn8Nm81yqVx3Kk9XYI=
k8s.io/client-go v0.17.4 h1:VVdVbpTY70jiNHS1eiFkUt7ZIJX3txd29nDxxXH4en8=
k8s.io/client-go v0.17.4/go.mod h1:ouF6o5pz3is8qU0/qYL2RnoxOPqgfuidYLowytyLJmc=
k8s.io/client-go v1.5.1 h1:XaX/lo2/u3/pmFau8HN+sB5C/b4dc4Dmm2eXjBH4p1E=
k8s.io/client-go v11.0.0+incompatible h1:LBbX2+lOwY9flffWlJM7f1Ct8V2SRNiMRDFeiwnJo9o=
k8s.io/client-go v11.0.1-0.20190516230509-ae8359b20417+incompatible h1:bK03DJulJi9j05gwnXUufcs2j7h4M85YFvJ0dIlQ9k4=
k8s.io/client-go v11.0.1-0.20190516230509-ae8359b20417+incompatible/go.mod h1:7vJpHMYJwNQCWgzmNV+VYUl1zCObLyodBc8nIyt8L5s=
k8s.io/code-generator v0.0.0-20200306081859-6a048a382944/go.mod h1:+UHX5rSbxmR8kzS+FAv7um6dtYrZokQvjHpDSYRVkTc=
k8s.io/component-base v0.0.0-20190612130303-4062e14deebe/go.mod h1:MmIDXnint3qMN0cqXHKrSiJ2XQKo3J1BPIz7in7NvO0=
k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20200114144118-36b2048a9120/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
k8s.io/klog v0.3.0/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
k8s.io/klog v0.3.1/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
k8s.io/klog v1.0.0 h1:Pt+yjF5aB1xDSVbau4VsWe+dQNzA0qv1LlXdC2dF6Q8=
k8s.io/klog v1.0.0/go.mod h1:4Bi6QPql/J/LkTDqv7R/cd3hPo4k2DG6Ptcz060Ez5I=
k8s.io/kube-openapi v0.0.0-20190228160746-b3a7cee44a30/go.mod h1:BXM9ceUBTj2QnfH2MK1odQs778ajze1RxcmP6S8RVVc=
k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a h1:UcxjrRMyNx/i/y8G7kPvLyy7rfbeuf1PYyBf973pgyU=
k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a/go.mod h1:1TqjTSzOxsLGIKfj0lK8EeCP7K1iUG65v09OM0/WG5E=
k8s.io/kube-openapi v0.0.0-20200121204235-bf4fb3bd569c h1:/KUFqjjqAcY4Us6luF5RDNZ16KJtb49HfR3ZHB9qYXM=
k8s.io/kube-openapi v0.0.0-20200121204235-bf4fb3bd569c/go.mod h1:GRQhZsXIAJ1xR0C9bd8UpWHZ5plfAS9fzPjJuQ6JL3E=
k8s.io/utils v0.0.0-20190221042446-c2654d5206da/go.mod h1:8k8uAuAQ0rXslZKaEWd0c3oVhZz7sSzSiPnVZayjIX0=
k8s.io/utils v0.0.0-20191114184206-e782cd3c129f/go.mod h1:sZAwmy6armz5eXlNoLmJcl4F1QuKu7sr+mFQ0byX7Ew=
k8s.io/utils v0.0.0-20200109141947-94aeca20bf09 h1:sz6xjn8QP74104YNmJpzLbJ+a3ZtHt0tkD0g8vpdWNw=
k8s.io/utils v0.0.0-20200109141947-94aeca20bf09/go.mod h1:sZAwmy6armz5eXlNoLmJcl4F1QuKu7sr+mFQ0byX7Ew=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
sigs.k8s.io/controller-runtime v0.5.0 h1:CbqIy5fbUX+4E9bpnBFd204YAzRYlM9SWW77BbrcDQo=
sigs.k8s.io/controller-runtime v0.5.0/go.mod h1:REiJzC7Y00U+2YkMbT8wxgrsX5USpXKGhb2sCtAXiT8=
sigs.k8s.io/kustomize v2.0.3+incompatible h1:JUufWFNlI44MdtnjUqVnvh29rR37PQFzPbLXqhyOyX0=
sigs.k8s.io/kustomize v2.0.3+incompatible/go.mod h1:MkjgH3RdOWrievjo6c9T245dYlB5QeXV4WCbnt/PEpU=
sigs.k8s.io/structured-merge-diff v0.0.0-20190525122527-15d366b2352e/go.mod h1:wWxsB5ozmmv/SG7nM11ayaAW51xMvak/t1r0CSlcokI=
sigs.k8s.io/structured-merge-diff v1.0.1-0.20191108220359-b1b620dd3f06/go.mod h1:/ULNhyfzRopfcjskuui0cTITekDduZ7ycKN3oUT9R18=
sigs.k8s.io/structured-merge-diff/v3 v3.0.0-20200116222232-67a7b8c61874/go.mod h1:PlARxl6Hbt/+BC80dRLi1qAmnMqwqDg62YvvVkZjemw=
sigs.k8s.io/yaml v1.1.0 h1:4A07+ZFc2wgJwo8YNlQpr1rVlgUDlxXHhPJciaPY5gs=
sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=
sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=

15
pkg/api/kyverno/v1/types.go
View file

@ -121,7 +121,7 @@ type Policy struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec Spec `json:"spec"`
Status PolicyStatus `json:"status"`
Status PolicyStatus `json:"status,omitempty"`
}
// Spec describes policy behavior by its rules
@ -155,9 +155,11 @@ type ConditionOperator string
const (
//Equal for Equal operator
Equal ConditionOperator = "Equal"
Equal ConditionOperator = "Equal"
Equals ConditionOperator = "Equals"
//NotEqual for NotEqual operator
NotEqual ConditionOperator = "NotEqual"
NotEqual ConditionOperator = "NotEqual"
NotEquals ConditionOperator = "NotEquals"
//In for In operator
In ConditionOperator = "In"
//NotIn for NotIn operator
@ -211,6 +213,11 @@ type Validation struct {
Message string `json:"message,omitempty"`
Pattern interface{} `json:"pattern,omitempty"`
AnyPattern []interface{} `json:"anyPattern,omitempty"`
Deny *Deny `json:"deny,omitempty"`
}
type Deny struct {
Conditions []Condition `json:"conditions,omitempty"`
}
// Generation describes which resources will be created when other resource is created
@ -230,7 +237,7 @@ type CloneFrom struct {
// PolicyStatus mostly contains statistics related to policy
type PolicyStatus struct {
// average time required to process the policy rules on a resource
AvgExecutionTime string `json:"averageExecutionTime"`
AvgExecutionTime string `json:"averageExecutionTime,omitempty"`
// number of violations created by this policy
ViolationCount int `json:"violationCount,omitempty"`
// Count of rules that failed

10
pkg/api/kyverno/v1/utils.go
View file

@ -3,7 +3,7 @@ package v1
import "reflect"
//HasMutateOrValidateOrGenerate checks for rule types
func (p ClusterPolicy) HasMutateOrValidateOrGenerate() bool {
func (p *ClusterPolicy) HasMutateOrValidateOrGenerate() bool {
for _, rule := range p.Spec.Rules {
if rule.HasMutate() || rule.HasValidate() || rule.HasGenerate() {
return true
@ -12,6 +12,14 @@ func (p ClusterPolicy) HasMutateOrValidateOrGenerate() bool {
return false
}
func (p *ClusterPolicy) BackgroundProcessingEnabled() bool {
if p.Spec.Background == nil {
return true
}
return *p.Spec.Background
}
//HasMutate checks for mutate rule
func (r Rule) HasMutate() bool {
return !reflect.DeepEqual(r.Mutation, Mutation{})

8
pkg/api/kyverno/v1/zz_generated.deepcopy.go
View file

@ -74,7 +74,7 @@ func (in *ClusterPolicy) DeepCopyObject() runtime.Object {
func (in *ClusterPolicyList) DeepCopyInto(out *ClusterPolicyList) {
*out = *in
out.TypeMeta = in.TypeMeta
out.ListMeta = in.ListMeta
in.ListMeta.DeepCopyInto(&out.ListMeta)
if in.Items != nil {
in, out := &in.Items, &out.Items
*out = make([]ClusterPolicy, len(*in))
@ -135,7 +135,7 @@ func (in *ClusterPolicyViolation) DeepCopyObject() runtime.Object {
func (in *ClusterPolicyViolationList) DeepCopyInto(out *ClusterPolicyViolationList) {
*out = *in
out.TypeMeta = in.TypeMeta
out.ListMeta = in.ListMeta
in.ListMeta.DeepCopyInto(&out.ListMeta)
if in.Items != nil {
in, out := &in.Items, &out.Items
*out = make([]ClusterPolicyViolation, len(*in))
@ -241,7 +241,7 @@ func (in *GenerateRequestContext) DeepCopy() *GenerateRequestContext {
func (in *GenerateRequestList) DeepCopyInto(out *GenerateRequestList) {
*out = *in
out.TypeMeta = in.TypeMeta
out.ListMeta = in.ListMeta
in.ListMeta.DeepCopyInto(&out.ListMeta)
if in.Items != nil {
in, out := &in.Items, &out.Items
*out = make([]GenerateRequest, len(*in))
@ -420,7 +420,7 @@ func (in *PolicyViolation) DeepCopyObject() runtime.Object {
func (in *PolicyViolationList) DeepCopyInto(out *PolicyViolationList) {
*out = *in
out.TypeMeta = in.TypeMeta
out.ListMeta = in.ListMeta
in.ListMeta.DeepCopyInto(&out.ListMeta)
if in.Items != nil {
in, out := &in.Items, &out.Items
*out = make([]PolicyViolation, len(*in))

108
pkg/auth/auth.go Normal file
View file

@ -0,0 +1,108 @@
package auth
import (
"fmt"
"reflect"
"github.com/go-logr/logr"
client "github.com/nirmata/kyverno/pkg/dclient"
authorizationv1 "k8s.io/api/authorization/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
)
//CanIOptions provides utility ti check if user has authorization for the given operation
type CanIOptions struct {
namespace string
verb string
kind string
client *client.Client
log logr.Logger
}
//NewCanI returns a new instance of operation access controler evaluator
func NewCanI(client *client.Client, kind, namespace, verb string, log logr.Logger) *CanIOptions {
o := CanIOptions{
client: client,
log: log,
}
o.namespace = namespace
o.kind = kind
o.verb = verb
return &o
}
//RunAccessCheck checks if the caller can perform the operation
// - operation is a combination of namespace, kind, verb
// - can only evaluate a single verb
// - group version resource is determined from the kind using the discovery client REST mapper
// - If disallowed, the reason and evaluationError is avialable in the logs
// - each can generates a SelfSubjectAccessReview resource and response is evaluated for permissions
func (o *CanIOptions) RunAccessCheck() (bool, error) {
// get GroupVersionResource from RESTMapper
// get GVR from kind
gvr := o.client.DiscoveryClient.GetGVRFromKind(o.kind)
if reflect.DeepEqual(gvr, schema.GroupVersionResource{}) {
// cannot find GVR
return false, fmt.Errorf("failed to get the Group Version Resource for kind %s", o.kind)
}
sar := &authorizationv1.SelfSubjectAccessReview{
Spec: authorizationv1.SelfSubjectAccessReviewSpec{
ResourceAttributes: &authorizationv1.ResourceAttributes{
Namespace: o.namespace,
Verb: o.verb,
Group: gvr.Group,
Resource: gvr.Resource,
},
},
}
// Set self subject access review
// - namespace
// - verb
// - resource
// - subresource
logger := o.log.WithValues("kind", sar.Kind, "namespace", sar.Namespace, "name", sar.Name)
// Create the Resource
resp, err := o.client.CreateResource("SelfSubjectAccessReview", "", sar, false)
if err != nil {
logger.Error(err, "failed to create resource")
return false, err
}
// status.allowed
allowed, ok, err := unstructured.NestedBool(resp.Object, "status", "allowed")
if !ok {
if err != nil {
logger.Error(err, "failed to get the field", "field", "status.allowed")
}
logger.Info("field not found", "field", "status.allowed")
}
if !allowed {
// status.reason
reason, ok, err := unstructured.NestedString(resp.Object, "status", "reason")
if !ok {
if err != nil {
logger.Error(err, "failed to get the field", "field", "status.reason")
}
logger.Info("field not found", "field", "status.reason")
}
// status.evaluationError
evaluationError, ok, err := unstructured.NestedString(resp.Object, "status", "evaludationError")
if !ok {
if err != nil {
logger.Error(err, "failed to get the field", "field", "status.evaluationError")
}
logger.Info("field not found", "field", "status.evaluationError")
}
// Reporting ? (just logs)
logger.Info("disallowed operation", "reason", reason, "evaluationError", evaluationError)
}
return allowed, nil
}

48
pkg/auth/auth_test.go Normal file
View file

@ -0,0 +1,48 @@
package auth
// import (
// "testing"
// "time"
// "github.com/golang/glog"
// "github.com/nirmata/kyverno/pkg/config"
// dclient "github.com/nirmata/kyverno/pkg/dclient"
// "github.com/nirmata/kyverno/pkg/signal"
// )
// func Test_Auth_pass(t *testing.T) {
// // needs running cluster
// var kubeconfig string
// stopCh := signal.SetupSignalHandler()
// kubeconfig = "/Users/shivd/.kube/config"
// clientConfig, err := config.CreateClientConfig(kubeconfig)
// if err != nil {
// glog.Fatalf("Error building kubeconfig: %v\n", err)
// }
// // DYNAMIC CLIENT
// // - client for all registered resources
// // - invalidate local cache of registered resource every 10 seconds
// client, err := dclient.NewClient(clientConfig, 10*time.Second, stopCh)
// if err != nil {
// glog.Fatalf("Error creating client: %v\n", err)
// }
// // Can i authenticate
// kind := "Deployment"
// namespace := "default"
// verb := "test"
// canI := NewCanI(client, kind, namespace, verb)
// ok, err := canI.RunAccessCheck()
// if err != nil {
// t.Error(err)
// }
// if ok {
// t.Log("allowed")
// } else {
// t.Log("notallowed")
// }
// t.FailNow()
// }

57
pkg/checker/checker.go
View file

@ -1,10 +1,11 @@
package checker
import (
"fmt"
"sync"
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyvernolister "github.com/nirmata/kyverno/pkg/client/listers/kyverno/v1"
dclient "github.com/nirmata/kyverno/pkg/dclient"
"github.com/nirmata/kyverno/pkg/event"
@ -20,8 +21,9 @@ const (
// LastReqTime stores the lastrequest times for incoming api-requests
type LastReqTime struct {
t time.Time
mu sync.RWMutex
t time.Time
mu sync.RWMutex
log logr.Logger
}
//Time returns the lastrequest time
@ -31,25 +33,28 @@ func (t *LastReqTime) Time() time.Time {
return t.t
}
//SetTime stes the lastrequest time
//SetTime updates the lastrequest time
func (t *LastReqTime) SetTime(tm time.Time) {
t.mu.Lock()
defer t.mu.Unlock()
t.t = tm
}
//NewLastReqTime returns a new instance of LastRequestTime store
func NewLastReqTime() *LastReqTime {
func NewLastReqTime(log logr.Logger) *LastReqTime {
return &LastReqTime{
t: time.Now(),
t: time.Now(),
log: log,
}
}
func checkIfPolicyWithMutateAndGenerateExists(pLister kyvernolister.ClusterPolicyLister) bool {
func checkIfPolicyWithMutateAndGenerateExists(pLister kyvernolister.ClusterPolicyLister, log logr.Logger) bool {
policies, err := pLister.ListResources(labels.NewSelector())
if err != nil {
glog.Error()
log.Error(err, "failed to list cluster policies")
}
for _, policy := range policies {
if policy.HasMutateOrValidateOrGenerate() {
// as there exists one policy with mutate or validate rule
@ -57,20 +62,22 @@ func checkIfPolicyWithMutateAndGenerateExists(pLister kyvernolister.ClusterPolic
return true
}
}
return false
}
//Run runs the checker and verify the resource update
func (t *LastReqTime) Run(pLister kyvernolister.ClusterPolicyLister, eventGen event.Interface, client *dclient.Client, defaultResync time.Duration, deadline time.Duration, stopCh <-chan struct{}) {
glog.V(2).Infof("starting default resync for webhook checker with resync time %d nanoseconds", defaultResync)
logger := t.log
logger.V(4).Info("starting default resync for webhook checker", "resyncTime", defaultResync)
maxDeadline := deadline * time.Duration(MaxRetryCount)
ticker := time.NewTicker(defaultResync)
/// interface to update and increment kyverno webhook status via annotations
statuscontrol := NewVerifyControl(client, eventGen)
statuscontrol := NewVerifyControl(client, eventGen, logger.WithName("StatusControl"))
// send the initial update status
if checkIfPolicyWithMutateAndGenerateExists(pLister) {
if checkIfPolicyWithMutateAndGenerateExists(pLister, logger) {
if err := statuscontrol.SuccessStatus(); err != nil {
glog.Error(err)
logger.Error(err, "failed to set 'success' status")
}
}
@ -82,38 +89,40 @@ func (t *LastReqTime) Run(pLister kyvernolister.ClusterPolicyLister, eventGen ev
for {
select {
case <-ticker.C:
// if there are no policies then we dont have a webhook on resource.
// we indirectly check if the resource
if !checkIfPolicyWithMutateAndGenerateExists(pLister) {
if !checkIfPolicyWithMutateAndGenerateExists(pLister, logger) {
continue
}
// get current time
timeDiff := time.Since(t.Time())
if timeDiff > maxDeadline {
glog.Infof("failed to receive any request for more than %v ", maxDeadline)
glog.Info("Admission Control failing: Webhook is not receiving requests forwarded by api-server as per webhook configurations")
// set the status unavailable
err := fmt.Errorf("Admission control configuration error")
logger.Error(err, "webhook check failed", "deadline", maxDeadline)
if err := statuscontrol.FailedStatus(); err != nil {
glog.Error(err)
logger.Error(err, "error setting webhook check status to failed")
}
continue
}
if timeDiff > deadline {
glog.Info("Admission Control failing: Webhook is not receiving requests forwarded by api-server as per webhook configurations")
logger.V(3).Info("webhook check deadline exceeded", "deadline", deadline)
// send request to update the kyverno deployment
if err := statuscontrol.IncrementAnnotation(); err != nil {
glog.Error(err)
logger.Error(err, "failed to increment annotation")
}
continue
}
// if the status was false before then we update it to true
// send request to update the kyverno deployment
if err := statuscontrol.SuccessStatus(); err != nil {
glog.Error(err)
logger.Error(err, "error setting webhook check status to success")
}
case <-stopCh:
// handler termination signal
glog.V(2).Infof("stopping default resync for webhook checker")
logger.V(2).Info("stopping default resync for webhook checker")
return
}
}

49
pkg/checker/status.go
View file

@ -4,7 +4,7 @@ import (
"fmt"
"strconv"
"github.com/golang/glog"
"github.com/go-logr/logr"
dclient "github.com/nirmata/kyverno/pkg/dclient"
"github.com/nirmata/kyverno/pkg/event"
)
@ -13,7 +13,7 @@ const deployName string = "kyverno"
const deployNamespace string = "kyverno"
const annCounter string = "kyverno.io/generationCounter"
const annWebhookStats string = "kyverno.io/webhookActive"
const annWebhookStatus string = "kyverno.io/webhookActive"
//StatusInterface provides api to update webhook active annotations on kyverno deployments
type StatusInterface interface {
@ -29,6 +29,7 @@ type StatusInterface interface {
type StatusControl struct {
client *dclient.Client
eventGen event.Interface
log logr.Logger
}
//SuccessStatus ...
@ -42,44 +43,51 @@ func (vc StatusControl) FailedStatus() error {
}
// NewVerifyControl ...
func NewVerifyControl(client *dclient.Client, eventGen event.Interface) *StatusControl {
func NewVerifyControl(client *dclient.Client, eventGen event.Interface, log logr.Logger) *StatusControl {
return &StatusControl{
client: client,
eventGen: eventGen,
log: log,
}
}
func (vc StatusControl) setStatus(status string) error {
glog.Infof("setting deployment %s in ns %s annotation %s to %s", deployName, deployNamespace, annWebhookStats, status)
logger := vc.log.WithValues("name", deployName, "namespace", deployNamespace)
var ann map[string]string
var err error
deploy, err := vc.client.GetResource("Deployment", deployNamespace, deployName)
if err != nil {
glog.V(4).Infof("failed to get deployment %s in namespace %s: %v", deployName, deployNamespace, err)
logger.Error(err, "failed to get deployment")
return err
}
ann = deploy.GetAnnotations()
if ann == nil {
ann = map[string]string{}
ann[annWebhookStats] = status
ann[annWebhookStatus] = status
}
webhookAction, ok := ann[annWebhookStats]
deployStatus, ok := ann[annWebhookStatus]
if ok {
// annotatiaion is present
if webhookAction == status {
glog.V(4).Infof("annotation %s already set to '%s'", annWebhookStats, status)
if deployStatus == status {
logger.V(4).Info(fmt.Sprintf("annotation %s already set to '%s'", annWebhookStatus, status))
return nil
}
}
// set the status
ann[annWebhookStats] = status
logger.Info("updating deployment annotation", "key", annWebhookStatus, "val", status)
ann[annWebhookStatus] = status
deploy.SetAnnotations(ann)
// update counter
_, err = vc.client.UpdateResource("Deployment", deployNamespace, deploy, false)
if err != nil {
glog.V(4).Infof("failed to update annotation %s for deployment %s in namespace %s: %v", annWebhookStats, deployName, deployNamespace, err)
logger.Error(err, "failed to update deployment annotation", "key", annWebhookStatus, "val", status)
return err
}
// create event on kyverno deployment
createStatusUpdateEvent(status, vc.eventGen)
return nil
@ -97,34 +105,43 @@ func createStatusUpdateEvent(status string, eventGen event.Interface) {
//IncrementAnnotation ...
func (vc StatusControl) IncrementAnnotation() error {
glog.Infof("setting deployment %s in ns %s annotation %s", deployName, deployNamespace, annCounter)
logger := vc.log
var ann map[string]string
var err error
deploy, err := vc.client.GetResource("Deployment", deployNamespace, deployName)
if err != nil {
glog.V(4).Infof("failed to get deployment %s in namespace %s: %v", deployName, deployNamespace, err)
logger.Error(err, "failed to find deployment %s in namespace %s", deployName, deployNamespace)
return err
}
ann = deploy.GetAnnotations()
if ann == nil {
ann = map[string]string{}
}
if ann[annCounter] == "" {
ann[annCounter] = "0"
}
counter, err := strconv.Atoi(ann[annCounter])
if err != nil {
glog.V(4).Infof("failed to parse string: %v", err)
logger.Error(err, "Failed to parse string", "name", annCounter, "value", ann[annCounter])
return err
}
// increment counter
counter++
ann[annCounter] = strconv.Itoa(counter)
glog.Infof("incrementing annotation %s counter to %d", annCounter, counter)
logger.V(3).Info("updating webhook test annotation", "key", annCounter, "value", counter, "deployment", deployName, "namespace", deployNamespace)
deploy.SetAnnotations(ann)
// update counter
_, err = vc.client.UpdateResource("Deployment", deployNamespace, deploy, false)
if err != nil {
glog.V(4).Infof("failed to update annotation %s for deployment %s in namespace %s: %v", annCounter, deployName, deployNamespace, err)
logger.Error(err, fmt.Sprintf("failed to update annotation %s for deployment %s in namespace %s", annCounter, deployName, deployNamespace))
return err
}
return nil
}

7
pkg/client/clientset/versioned/clientset.go
View file

@ -19,6 +19,8 @@ limitations under the License.
package versioned
import (
"fmt"
kyvernov1 "github.com/nirmata/kyverno/pkg/client/clientset/versioned/typed/kyverno/v1"
discovery "k8s.io/client-go/discovery"
rest "k8s.io/client-go/rest"
@ -51,9 +53,14 @@ func (c *Clientset) Discovery() discovery.DiscoveryInterface {
}
// NewForConfig creates a new Clientset for the given config.
// If config's RateLimiter is not set and QPS and Burst are acceptable,
// NewForConfig will generate a rate-limiter in configShallowCopy.
func NewForConfig(c *rest.Config) (*Clientset, error) {
configShallowCopy := *c
if configShallowCopy.RateLimiter == nil && configShallowCopy.QPS > 0 {
if configShallowCopy.Burst <= 0 {
return nil, fmt.Errorf("Burst is required to be greater than 0 when RateLimiter is not set and QPS is set to greater than 0")
}
configShallowCopy.RateLimiter = flowcontrol.NewTokenBucketRateLimiter(configShallowCopy.QPS, configShallowCopy.Burst)
}
var cs Clientset

7
pkg/client/clientset/versioned/fake/clientset_generated.go
View file

@ -41,7 +41,7 @@ func NewSimpleClientset(objects ...runtime.Object) *Clientset {
}
}
cs := &Clientset{}
cs := &Clientset{tracker: o}
cs.discovery = &fakediscovery.FakeDiscovery{Fake: &cs.Fake}
cs.AddReactor("*", "*", testing.ObjectReaction(o))
cs.AddWatchReactor("*", func(action testing.Action) (handled bool, ret watch.Interface, err error) {
@ -63,12 +63,17 @@ func NewSimpleClientset(objects ...runtime.Object) *Clientset {
type Clientset struct {
testing.Fake
discovery *fakediscovery.FakeDiscovery
tracker testing.ObjectTracker
}
func (c *Clientset) Discovery() discovery.DiscoveryInterface {
return c.discovery
}
func (c *Clientset) Tracker() testing.ObjectTracker {
return c.tracker
}
var _ clientset.Interface = &Clientset{}
// KyvernoV1 retrieves the KyvernoV1Client

3
pkg/client/clientset/versioned/fake/register.go
View file

@ -29,7 +29,8 @@ import (
var scheme = runtime.NewScheme()
var codecs = serializer.NewCodecFactory(scheme)
var parameterCodec = runtime.NewParameterCodec(scheme)
// var parameterCodec = runtime.NewParameterCodec(scheme)
var localSchemeBuilder = runtime.SchemeBuilder{
kyvernov1.AddToScheme,
}

3
pkg/client/clientset/versioned/typed/kyverno/v1/kyverno_client.go
View file

@ -21,7 +21,6 @@ package v1
import (
v1 "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/client/clientset/versioned/scheme"
serializer "k8s.io/apimachinery/pkg/runtime/serializer"
rest "k8s.io/client-go/rest"
)
@ -86,7 +85,7 @@ func setConfigDefaults(config *rest.Config) error {
gv := v1.SchemeGroupVersion
config.GroupVersion = &gv
config.APIPath = "/apis"
config.NegotiatedSerializer = serializer.DirectCodecFactory{CodecFactory: scheme.Codecs}
config.NegotiatedSerializer = scheme.Codecs.WithoutConversion()
if config.UserAgent == "" {
config.UserAgent = rest.DefaultKubernetesUserAgent()

32
pkg/config/config.go
View file

@ -1,9 +1,7 @@
package config
import (
"flag"
"github.com/golang/glog"
"github.com/go-logr/logr"
rest "k8s.io/client-go/rest"
clientcmd "k8s.io/client-go/tools/clientcmd"
)
@ -72,31 +70,19 @@ var (
PolicyMutatingWebhookServicePath = "/policymutate"
//VerifyMutatingWebhookServicePath is the path for verify webhook(used to veryfing if admission control is enabled and active)
VerifyMutatingWebhookServicePath = "/verifymutate"
// LivenessServicePath is the path for check liveness health
LivenessServicePath = "/health/liveness"
// ReadinessServicePath is the path for check readness health
ReadinessServicePath = "/health/readiness"
)
//LogDefaultFlags sets default glog flags
func LogDefaultFlags() {
var err error
err = flag.Set("logtostderr", "true")
if err != nil {
glog.Fatalf("failed to set flag 'logtostderr' to 'true':%v", err)
}
err = flag.Set("stderrthreshold", "WARNING")
if err != nil {
glog.Fatalf("failed to set flag 'stderrthreshold' to 'WARNING':%v", err)
}
flag.Set("v", "2")
if err != nil {
glog.Fatalf("failed to set flag 'v' to '2':%v", err)
}
}
//CreateClientConfig creates client config
func CreateClientConfig(kubeconfig string) (*rest.Config, error) {
func CreateClientConfig(kubeconfig string, log logr.Logger) (*rest.Config, error) {
logger := log.WithName("CreateClientConfig")
if kubeconfig == "" {
glog.Info("Using in-cluster configuration")
logger.Info("Using in-cluster configuration")
return rest.InClusterConfig()
}
glog.V(4).Infof("Using configuration from '%s'", kubeconfig)
logger.V(4).Info("Using specified kubeconfig", "kubeconfig", kubeconfig)
return clientcmd.BuildConfigFromFlags("", kubeconfig)
}

39
pkg/config/dynamicconfig.go
View file

@ -1,14 +1,13 @@
package config
import (
"fmt"
"os"
"reflect"
"regexp"
"strings"
"sync"
"github.com/golang/glog"
"github.com/go-logr/logr"
"github.com/minio/minio/pkg/wildcard"
v1 "k8s.io/api/core/v1"
informers "k8s.io/client-go/informers/core/v1"
@ -31,6 +30,7 @@ type ConfigData struct {
filters []k8Resource
// hasynced
cmSycned cache.InformerSynced
log logr.Logger
}
// ToFilter checks if the given resource is set to be filtered in the configuration
@ -51,20 +51,21 @@ type Interface interface {
}
// NewConfigData ...
func NewConfigData(rclient kubernetes.Interface, cmInformer informers.ConfigMapInformer, filterK8Resources string) *ConfigData {
func NewConfigData(rclient kubernetes.Interface, cmInformer informers.ConfigMapInformer, filterK8Resources string, log logr.Logger) *ConfigData {
// environment var is read at start only
if cmNameEnv == "" {
glog.Info("ConfigMap name not defined in env:INIT_CONFIG: loading no default configuration")
log.Info("ConfigMap name not defined in env:INIT_CONFIG: loading no default configuration")
}
cd := ConfigData{
client: rclient,
cmName: os.Getenv(cmNameEnv),
cmSycned: cmInformer.Informer().HasSynced,
log: log,
}
//TODO: this has been added to backward support command line arguments
// will be removed in future and the configuration will be set only via configmaps
if filterK8Resources != "" {
glog.Info("Init configuration from commandline arguments")
cd.log.Info("init configuration from commandline arguments")
cd.initFilters(filterK8Resources)
}
@ -78,9 +79,10 @@ func NewConfigData(rclient kubernetes.Interface, cmInformer informers.ConfigMapI
//Run checks syncing
func (cd *ConfigData) Run(stopCh <-chan struct{}) {
logger := cd.log
// wait for cache to populate first time
if !cache.WaitForCacheSync(stopCh, cd.cmSycned) {
glog.Error("configuration: failed to sync informer cache")
logger.Info("configuration: failed to sync informer cache")
}
}
@ -103,16 +105,17 @@ func (cd *ConfigData) updateCM(old, cur interface{}) {
}
func (cd *ConfigData) deleteCM(obj interface{}) {
logger := cd.log
cm, ok := obj.(*v1.ConfigMap)
if !ok {
tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
if !ok {
glog.Info(fmt.Errorf("Couldn't get object from tombstone %#v", obj))
logger.Info("failed to get object from tombstone")
return
}
_, ok = tombstone.Obj.(*v1.ConfigMap)
if !ok {
glog.Info(fmt.Errorf("Tombstone contained object that is not a ConfigMap %#v", obj))
logger.Info("Tombstone contained object that is not a ConfigMap", "object", obj)
return
}
}
@ -125,19 +128,20 @@ func (cd *ConfigData) deleteCM(obj interface{}) {
}
func (cd *ConfigData) load(cm v1.ConfigMap) {
logger := cd.log.WithValues("name", cm.Name, "namespace", cm.Namespace)
if cm.Data == nil {
glog.V(4).Infof("Configuration: No data defined in ConfigMap %s", cm.Name)
logger.V(4).Info("configuration: No data defined in ConfigMap")
return
}
// get resource filters
filters, ok := cm.Data["resourceFilters"]
if !ok {
glog.V(4).Infof("Configuration: No resourceFilters defined in ConfigMap %s", cm.Name)
logger.V(4).Info("configuration: No resourceFilters defined in ConfigMap")
return
}
// filters is a string
if filters == "" {
glog.V(4).Infof("Configuration: resourceFilters is empty in ConfigMap %s", cm.Name)
logger.V(4).Info("configuration: resourceFilters is empty in ConfigMap")
return
}
// parse and load the configuration
@ -146,11 +150,10 @@ func (cd *ConfigData) load(cm v1.ConfigMap) {
newFilters := parseKinds(filters)
if reflect.DeepEqual(newFilters, cd.filters) {
glog.V(4).Infof("Configuration: resourceFilters did not change in ConfigMap %s", cm.Name)
logger.V(4).Info("resourceFilters did not change")
return
}
glog.V(4).Infof("Configuration: Old resource filters %v", cd.filters)
glog.Infof("Configuration: New resource filters to %v", newFilters)
logger.V(4).Info(" Updated resource filters", "oldFilters", cd.filters, "newFilters", newFilters)
// update filters
cd.filters = newFilters
}
@ -158,20 +161,20 @@ func (cd *ConfigData) load(cm v1.ConfigMap) {
//TODO: this has been added to backward support command line arguments
// will be removed in future and the configuration will be set only via configmaps
func (cd *ConfigData) initFilters(filters string) {
logger := cd.log
// parse and load the configuration
cd.mux.Lock()
defer cd.mux.Unlock()
newFilters := parseKinds(filters)
glog.Infof("Configuration: Init resource filters to %v", newFilters)
logger.Info("Init resource filters", "filters", newFilters)
// update filters
cd.filters = newFilters
}
func (cd *ConfigData) unload(cm v1.ConfigMap) {
// TODO pick one msg
glog.Infof("Configuration: ConfigMap %s deleted, removing configuration filters", cm.Name)
glog.Infof("Configuration: Removing all resource filters as ConfigMap %s deleted", cm.Name)
logger := cd.log
logger.Info("ConfigMap deleted, removing configuration filters", "name", cm.Name, "namespace", cm.Namespace)
cd.mux.Lock()
defer cd.mux.Unlock()
cd.filters = []k8Resource{}

12
pkg/constant/constant.go Normal file
View file

@ -0,0 +1,12 @@
package constant
import "time"
const (
CRDControllerResync = 15 * time.Minute
PolicyViolationControllerResync = 15 * time.Minute
PolicyControllerResync = 15 * time.Minute
EventControllerResync = 15 * time.Minute
GenerateControllerResync = 15 * time.Minute
GenerateRequestControllerResync = 15 * time.Minute
)

37
pkg/dclient/certificates.go
View file

@ -6,7 +6,6 @@ import (
"net/url"
"time"
"github.com/golang/glog"
"github.com/nirmata/kyverno/pkg/config"
tls "github.com/nirmata/kyverno/pkg/tls"
certificates "k8s.io/api/certificates/v1beta1"
@ -19,13 +18,14 @@ import (
// Created pair is stored in cluster's secret.
// Returns struct with key/certificate pair.
func (c *Client) InitTLSPemPair(configuration *rest.Config, fqdncn bool) (*tls.TlsPemPair, error) {
logger := c.log
certProps, err := c.GetTLSCertProps(configuration)
if err != nil {
return nil, err
}
tlsPair := c.ReadTlsPair(certProps)
if tls.IsTLSPairShouldBeUpdated(tlsPair) {
glog.Info("Generating new key/certificate pair for TLS")
logger.Info("Generating new key/certificate pair for TLS")
tlsPair, err = c.generateTLSPemPair(certProps, fqdncn)
if err != nil {
return nil, err
@ -35,8 +35,7 @@ func (c *Client) InitTLSPemPair(configuration *rest.Config, fqdncn bool) (*tls.T
}
return tlsPair, nil
}
glog.Infoln("Using existing TLS key/certificate pair")
logger.Info("Using existing TLS key/certificate pair")
return tlsPair, nil
}
@ -71,6 +70,7 @@ func (c *Client) generateTLSPemPair(props tls.TlsCertificateProps, fqdncn bool)
// Submits and approves certificate request, returns request which need to be fetched
func (c *Client) submitAndApproveCertificateRequest(req *certificates.CertificateSigningRequest) (*certificates.CertificateSigningRequest, error) {
logger := c.log.WithName("submitAndApproveCertificateRequest")
certClient, err := c.GetCSRInterface()
if err != nil {
return nil, err
@ -86,7 +86,7 @@ func (c *Client) submitAndApproveCertificateRequest(req *certificates.Certificat
if err != nil {
return nil, fmt.Errorf("Unable to delete existing certificate request: %v", err)
}
glog.Info("Old certificate request is deleted")
logger.Info("Old certificate request is deleted")
break
}
}
@ -95,7 +95,7 @@ func (c *Client) submitAndApproveCertificateRequest(req *certificates.Certificat
if err != nil {
return nil, err
}
glog.Infof("Certificate request %s is created", unstrRes.GetName())
logger.Info("Certificate request created", "name", unstrRes.GetName())
res, err := convertToCSR(unstrRes)
if err != nil {
@ -110,7 +110,7 @@ func (c *Client) submitAndApproveCertificateRequest(req *certificates.Certificat
if err != nil {
return nil, fmt.Errorf("Unable to approve certificate request: %v", err)
}
glog.Infof("Certificate request %s is approved", res.ObjectMeta.Name)
logger.Info("Certificate request is approved", "name", res.ObjectMeta.Name)
return res, nil
}
@ -144,9 +144,10 @@ func (c *Client) fetchCertificateFromRequest(req *certificates.CertificateSignin
//ReadRootCASecret returns the RootCA from the pre-defined secret
func (c *Client) ReadRootCASecret() (result []byte) {
logger := c.log.WithName("ReadRootCASecret")
certProps, err := c.GetTLSCertProps(c.clientConfig)
if err != nil {
glog.Error(err)
logger.Error(err, "failed to get TLS Cert Properties")
return result
}
sname := generateRootCASecretName(certProps)
@ -156,16 +157,16 @@ func (c *Client) ReadRootCASecret() (result []byte) {
}
tlsca, err := convertToSecret(stlsca)
if err != nil {
glog.Error(err)
logger.Error(err, "failed to convert secret", "name", sname, "namespace", certProps.Namespace)
return result
}
result = tlsca.Data[rootCAKey]
if len(result) == 0 {
glog.Warningf("root CA certificate not found in secret %s/%s", certProps.Namespace, tlsca.Name)
logger.Info("root CA certificate not found in secret", "name", tlsca.Name, "namespace", certProps.Namespace)
return result
}
glog.V(4).Infof("using CA bundle defined in secret %s/%s to validate the webhook's server certificate", certProps.Namespace, tlsca.Name)
logger.V(4).Info("using CA bundle defined in secret to validate the webhook's server certificate", "name", tlsca.Name, "namespace", certProps.Namespace)
return result
}
@ -174,10 +175,11 @@ const rootCAKey string = "rootCA.crt"
//ReadTlsPair Reads the pair of TLS certificate and key from the specified secret.
func (c *Client) ReadTlsPair(props tls.TlsCertificateProps) *tls.TlsPemPair {
logger := c.log.WithName("ReadTlsPair")
sname := generateTLSPairSecretName(props)
unstrSecret, err := c.GetResource(Secrets, props.Namespace, sname)
if err != nil {
glog.Warningf("Unable to get secret %s/%s: %s", props.Namespace, sname, err)
logger.Error(err, "Failed to get secret", "name", sname, "namespace", props.Namespace)
return nil
}
@ -188,7 +190,7 @@ func (c *Client) ReadTlsPair(props tls.TlsCertificateProps) *tls.TlsPemPair {
sname := generateRootCASecretName(props)
_, err := c.GetResource(Secrets, props.Namespace, sname)
if err != nil {
glog.Errorf("Root CA secret %s/%s is required while using self-signed certificates TLS pair, defaulting to generating new TLS pair", props.Namespace, sname)
logger.Error(err, "Root CA secret is required while using self-signed certificates TLS pair, defaulting to generating new TLS pair", "name", sname, "namespace", props.Namespace)
return nil
}
}
@ -201,11 +203,11 @@ func (c *Client) ReadTlsPair(props tls.TlsCertificateProps) *tls.TlsPemPair {
PrivateKey: secret.Data[v1.TLSPrivateKeyKey],
}
if len(pemPair.Certificate) == 0 {
glog.Warningf("TLS Certificate not found in secret %s/%s", props.Namespace, sname)
logger.Info("TLS Certificate not found in secret", "name", sname, "namespace", props.Namespace)
return nil
}
if len(pemPair.PrivateKey) == 0 {
glog.Warningf("TLS PrivateKey not found in secret %s/%s", props.Namespace, sname)
logger.Info("TLS PrivateKey not found in secret", "name", sname, "namespace", props.Namespace)
return nil
}
return &pemPair
@ -214,6 +216,7 @@ func (c *Client) ReadTlsPair(props tls.TlsCertificateProps) *tls.TlsPemPair {
//WriteTlsPair Writes the pair of TLS certificate and key to the specified secret.
// Updates existing secret or creates new one.
func (c *Client) WriteTlsPair(props tls.TlsCertificateProps, pemPair *tls.TlsPemPair) error {
logger := c.log.WithName("WriteTlsPair")
name := generateTLSPairSecretName(props)
_, err := c.GetResource(Secrets, props.Namespace, name)
if err != nil {
@ -235,7 +238,7 @@ func (c *Client) WriteTlsPair(props tls.TlsCertificateProps, pemPair *tls.TlsPem
_, err := c.CreateResource(Secrets, props.Namespace, secret, false)
if err == nil {
glog.Infof("Secret %s is created", name)
logger.Info("secret created", "name", name, "namespace", props.Namespace)
}
return err
}
@ -251,7 +254,7 @@ func (c *Client) WriteTlsPair(props tls.TlsCertificateProps, pemPair *tls.TlsPem
if err != nil {
return err
}
glog.Infof("Secret %s is updated", name)
logger.Info("secret updated", "name", name, "namespace", props.Namespace)
return nil
}

92
pkg/dclient/client.go
View file

@ -5,9 +5,8 @@ import (
"strings"
"time"
"github.com/go-logr/logr"
openapi_v2 "github.com/googleapis/gnostic/OpenAPIv2"
"github.com/golang/glog"
"github.com/nirmata/kyverno/pkg/config"
apps "k8s.io/api/apps/v1"
certificates "k8s.io/api/certificates/v1beta1"
@ -32,13 +31,15 @@ import (
//Client enables interaction with k8 resource
type Client struct {
client dynamic.Interface
log logr.Logger
clientConfig *rest.Config
kclient kubernetes.Interface
DiscoveryClient IDiscovery
}
//NewClient creates new instance of client
func NewClient(config *rest.Config, resync time.Duration, stopCh <-chan struct{}) (*Client, error) {
func NewClient(config *rest.Config, resync time.Duration, stopCh <-chan struct{}, log logr.Logger) (*Client, error) {
dclient, err := dynamic.NewForConfig(config)
if err != nil {
return nil, err
@ -51,9 +52,10 @@ func NewClient(config *rest.Config, resync time.Duration, stopCh <-chan struct{}
client: dclient,
clientConfig: config,
kclient: kclient,
log: log.WithName("dclient"),
}
// Set discovery client
discoveryClient := ServerPreferredResources{memory.NewMemCacheClient(kclient.Discovery())}
discoveryClient := ServerPreferredResources{cachedClient: memory.NewMemCacheClient(kclient.Discovery()), log: client.log}
// client will invalidate registered resources cache every x seconds,
// As there is no way to identify if the registered resource is available or not
// we will be invalidating the local cache, so the next request get a fresh cache
@ -127,6 +129,11 @@ func (c *Client) PatchResource(kind string, namespace string, name string, patch
return c.getResourceInterface(kind, namespace).Patch(name, patchTypes.JSONPatchType, patch, meta.PatchOptions{})
}
// GetDynamicInterface fetches underlying dynamic interface
func (c *Client) GetDynamicInterface() dynamic.Interface {
return c.client
}
// ListResource returns the list of resources in unstructured/json format
// Access items using []Items
func (c *Client) ListResource(kind string, namespace string, lselector *meta.LabelSelector) (*unstructured.UnstructuredList, error) {
@ -189,7 +196,6 @@ func (c *Client) UpdateStatusResource(kind string, namespace string, obj interfa
func convertToUnstructured(obj interface{}) *unstructured.Unstructured {
unstructuredObj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&obj)
if err != nil {
glog.Errorf("Unable to convert : %v", err)
return nil
}
return &unstructured.Unstructured{Object: unstructuredObj}
@ -215,6 +221,7 @@ func convertToCSR(obj *unstructured.Unstructured) (*certificates.CertificateSign
//IDiscovery provides interface to mange Kind and GVR mapping
type IDiscovery interface {
FindResource(kind string) (*meta.APIResource, schema.GroupVersionResource, error)
GetGVRFromKind(kind string) schema.GroupVersionResource
GetServerVersion() (*version.Info, error)
OpenAPISchema() (*openapi_v2.Document, error)
@ -228,77 +235,90 @@ func (c *Client) SetDiscovery(discoveryClient IDiscovery) {
//ServerPreferredResources stores the cachedClient instance for discovery client
type ServerPreferredResources struct {
cachedClient discovery.CachedDiscoveryInterface
log logr.Logger
}
//Poll will keep invalidate the local cache
func (c ServerPreferredResources) Poll(resync time.Duration, stopCh <-chan struct{}) {
logger := c.log.WithName("Poll")
// start a ticker
ticker := time.NewTicker(resync)
defer func() { ticker.Stop() }()
glog.Infof("Starting registered resources sync: every %d seconds", resync)
logger.Info("starting registered resources sync", "period", resync)
for {
select {
case <-stopCh:
glog.Info("Stopping registered resources sync")
logger.Info("stopping registered resources sync")
return
case <-ticker.C:
// set cache as stale
glog.V(6).Info("invalidating local client cache for registered resources")
logger.V(6).Info("invalidating local client cache for registered resources")
c.cachedClient.Invalidate()
}
}
}
// OpenAPISchema returns the API server OpenAPI schema document
func (c ServerPreferredResources) OpenAPISchema() (*openapi_v2.Document, error) {
return c.cachedClient.OpenAPISchema()
}
//GetGVRFromKind get the Group Version Resource from kind
// if kind is not found in first attempt we invalidate the cache,
// the retry will then fetch the new registered resources and check again
// if not found after 2 attempts, we declare kind is not found
// kind is Case sensitive
// GetGVRFromKind get the Group Version Resource from kind
func (c ServerPreferredResources) GetGVRFromKind(kind string) schema.GroupVersionResource {
var gvr schema.GroupVersionResource
var err error
gvr, err = loadServerResources(kind, c.cachedClient)
if err != nil && !c.cachedClient.Fresh() {
// invalidate cahce & re-try once more
c.cachedClient.Invalidate()
gvr, err = loadServerResources(kind, c.cachedClient)
if err == nil {
return gvr
}
_, gvr, err := c.FindResource(kind)
if err != nil {
c.log.Info("schema not found", "kind", kind)
return schema.GroupVersionResource{}
}
return gvr
}
//GetServerVersion returns the server version of the cluster
// GetServerVersion returns the server version of the cluster
func (c ServerPreferredResources) GetServerVersion() (*version.Info, error) {
return c.cachedClient.ServerVersion()
}
func loadServerResources(k string, cdi discovery.CachedDiscoveryInterface) (schema.GroupVersionResource, error) {
serverresources, err := cdi.ServerPreferredResources()
emptyGVR := schema.GroupVersionResource{}
if err != nil {
glog.Error(err)
return emptyGVR, err
// FindResource finds an API resource that matches 'kind'. If the resource is not
// found and the Cache is not fresh, the cache is invalidated and a retry is attempted
func (c ServerPreferredResources) FindResource(kind string) (*meta.APIResource, schema.GroupVersionResource, error) {
r, gvr, err := c.findResource(kind)
if err == nil {
return r, gvr, nil
}
if !c.cachedClient.Fresh() {
c.cachedClient.Invalidate()
if r, gvr, err = c.findResource(kind); err == nil {
return r, gvr, nil
}
}
return nil, schema.GroupVersionResource{}, err
}
func (c ServerPreferredResources) findResource(k string) (*meta.APIResource, schema.GroupVersionResource, error) {
serverresources, err := c.cachedClient.ServerPreferredResources()
if err != nil {
c.log.Error(err, "failed to get registered preferred resources")
return nil, schema.GroupVersionResource{}, err
}
for _, serverresource := range serverresources {
for _, resource := range serverresource.APIResources {
// skip the resource names with "/", to avoid comparison with subresources
// skip the resource names with "/", to avoid comparison with subresources
if resource.Kind == k && !strings.Contains(resource.Name, "/") {
gv, err := schema.ParseGroupVersion(serverresource.GroupVersion)
if err != nil {
glog.Error(err)
return emptyGVR, err
c.log.Error(err, "failed to parse groupVersion", "groupVersion", serverresource.GroupVersion)
return nil, schema.GroupVersionResource{}, err
}
return gv.WithResource(resource.Name), nil
return &resource, gv.WithResource(resource.Name), nil
}
}
}
return emptyGVR, fmt.Errorf("kind '%s' not found", k)
return nil, schema.GroupVersionResource{}, fmt.Errorf("kind '%s' not found", k)
}

6
pkg/dclient/utils.go
View file

@ -1,10 +1,12 @@
package client
import (
"fmt"
"strings"
openapi_v2 "github.com/googleapis/gnostic/OpenAPIv2"
meta "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
@ -76,6 +78,10 @@ func (c *fakeDiscoveryClient) GetGVRFromKind(kind string) schema.GroupVersionRes
return c.getGVR(resource)
}
func (c *fakeDiscoveryClient) FindResource(kind string) (*meta.APIResource, schema.GroupVersionResource, error) {
return nil, schema.GroupVersionResource{}, fmt.Errorf("Not implemented")
}
func (c *fakeDiscoveryClient) OpenAPISchema() (*openapi_v2.Document, error) {
return nil, nil
}

15
pkg/engine/anchor/anchor.go
View file

@ -4,7 +4,8 @@ import (
"fmt"
"strconv"
"github.com/golang/glog"
"github.com/go-logr/logr"
"sigs.k8s.io/controller-runtime/pkg/log"
)
//ValidationHandler for element processes
@ -12,7 +13,7 @@ type ValidationHandler interface {
Handle(handler resourceElementHandler, resourceMap map[string]interface{}, originPattern interface{}) (string, error)
}
type resourceElementHandler = func(resourceElement, patternElement, originPattern interface{}, path string) (string, error)
type resourceElementHandler = func(log logr.Logger, resourceElement, patternElement, originPattern interface{}, path string) (string, error)
//CreateElementHandler factory to process elements
func CreateElementHandler(element string, pattern interface{}, path string) ValidationHandler {
@ -82,7 +83,7 @@ func (eh EqualityHandler) Handle(handler resourceElementHandler, resourceMap map
// check if anchor is present in resource
if value, ok := resourceMap[anchorKey]; ok {
// validate the values of the pattern
returnPath, err := handler(value, eh.pattern, originPattern, currentPath)
returnPath, err := handler(log.Log, value, eh.pattern, originPattern, currentPath)
if err != nil {
return returnPath, err
}
@ -115,7 +116,7 @@ func (dh DefaultHandler) Handle(handler resourceElementHandler, resourceMap map[
} else if dh.pattern == "*" && resourceMap[dh.element] == nil {
return dh.path, fmt.Errorf("Validation rule failed at %s, Field %s is not present", dh.path, dh.element)
} else {
path, err := handler(resourceMap[dh.element], dh.pattern, originPattern, currentPath)
path, err := handler(log.Log, resourceMap[dh.element], dh.pattern, originPattern, currentPath)
if err != nil {
return path, err
}
@ -146,7 +147,7 @@ func (ch ConditionAnchorHandler) Handle(handler resourceElementHandler, resource
// check if anchor is present in resource
if value, ok := resourceMap[anchorKey]; ok {
// validate the values of the pattern
returnPath, err := handler(value, ch.pattern, originPattern, currentPath)
returnPath, err := handler(log.Log, value, ch.pattern, originPattern, currentPath)
if err != nil {
return returnPath, err
}
@ -194,7 +195,6 @@ func (eh ExistenceHandler) Handle(handler resourceElementHandler, resourceMap ma
}
return validateExistenceListResource(handler, typedResource, typedPatternMap, originPattern, currentPath)
default:
glog.Error("Invalid type: Existence ^ () anchor can be used only on list/array type resource")
return currentPath, fmt.Errorf("Invalid resource type %T: Existence ^ () anchor can be used only on list/array type resource", value)
}
}
@ -206,10 +206,9 @@ func validateExistenceListResource(handler resourceElementHandler, resourceList
// if non satisfy then throw an error
for i, resourceElement := range resourceList {
currentPath := path + strconv.Itoa(i) + "/"
_, err := handler(resourceElement, patternMap, originPattern, currentPath)
_, err := handler(log.Log, resourceElement, patternMap, originPattern, currentPath)
if err == nil {
// condition is satisfied, dont check further
glog.V(4).Infof("Existence check satisfied at path %s, for pattern %v", currentPath, patternMap)
return "", nil
}
}

36
pkg/engine/context/context.go
View file

@ -5,9 +5,12 @@ import (
"strings"
"sync"
"k8s.io/api/admission/v1beta1"
jsonpatch "github.com/evanphx/json-patch"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"sigs.k8s.io/controller-runtime/pkg/log"
)
//Interface to manage context operations
@ -33,6 +36,7 @@ type Context struct {
mu sync.RWMutex
jsonRaw []byte
whiteListVars []string
log logr.Logger
}
//NewContext returns a new context
@ -42,6 +46,7 @@ func NewContext(whiteListVars ...string) *Context {
// data: map[string]interface{}{},
jsonRaw: []byte(`{}`), // empty json struct
whiteListVars: whiteListVars,
log: log.Log.WithName("context"),
}
return &ctx
}
@ -54,19 +59,34 @@ func (ctx *Context) AddJSON(dataRaw []byte) error {
// merge json
ctx.jsonRaw, err = jsonpatch.MergePatch(ctx.jsonRaw, dataRaw)
if err != nil {
glog.V(4).Infof("failed to merge JSON data: %v", err)
ctx.log.Error(err, "failed to merge JSON data")
return err
}
return nil
}
func (ctx *Context) AddRequest(request *v1beta1.AdmissionRequest) error {
modifiedResource := struct {
Request interface{} `json:"request"`
}{
Request: request,
}
objRaw, err := json.Marshal(modifiedResource)
if err != nil {
ctx.log.Error(err, "failed to marshal the request")
return err
}
return ctx.AddJSON(objRaw)
}
//AddResource data at path: request.object
func (ctx *Context) AddResource(dataRaw []byte) error {
// unmarshall the resource struct
var data interface{}
if err := json.Unmarshal(dataRaw, &data); err != nil {
glog.V(4).Infof("failed to unmarshall the context data: %v", err)
ctx.log.Error(err, "failed to unmarshall the resource")
return err
}
@ -82,7 +102,7 @@ func (ctx *Context) AddResource(dataRaw []byte) error {
objRaw, err := json.Marshal(modifiedResource)
if err != nil {
glog.V(4).Infof("failed to marshall the updated context data")
ctx.log.Error(err, "failed to marshal the resource")
return err
}
return ctx.AddJSON(objRaw)
@ -98,7 +118,7 @@ func (ctx *Context) AddUserInfo(userRequestInfo kyverno.RequestInfo) error {
objRaw, err := json.Marshal(modifiedResource)
if err != nil {
glog.V(4).Infof("failed to marshall the updated context data")
ctx.log.Error(err, "failed to marshal the UserInfo")
return err
}
return ctx.AddJSON(objRaw)
@ -118,8 +138,6 @@ func (ctx *Context) AddSA(userName string) error {
// filter namespace
groups := strings.Split(sa, ":")
if len(groups) >= 2 {
glog.V(4).Infof("serviceAccount namespace: %s", groups[0])
glog.V(4).Infof("serviceAccount name: %s", groups[1])
saName = groups[1]
saNamespace = groups[0]
}
@ -131,7 +149,7 @@ func (ctx *Context) AddSA(userName string) error {
}
saNameRaw, err := json.Marshal(saNameObj)
if err != nil {
glog.V(4).Infof("failed to marshall the updated context data")
ctx.log.Error(err, "failed to marshal the SA")
return err
}
if err := ctx.AddJSON(saNameRaw); err != nil {
@ -145,7 +163,7 @@ func (ctx *Context) AddSA(userName string) error {
}
saNsRaw, err := json.Marshal(saNsObj)
if err != nil {
glog.V(4).Infof("failed to marshall the updated context data")
ctx.log.Error(err, "failed to marshal the SA namespace")
return err
}
if err := ctx.AddJSON(saNsRaw); err != nil {

15
pkg/engine/context/evaluate.go
View file

@ -3,8 +3,8 @@ package context
import (
"encoding/json"
"fmt"
"strings"
"github.com/golang/glog"
jmespath "github.com/jmespath/go-jmespath"
)
@ -12,14 +12,14 @@ import (
func (ctx *Context) Query(query string) (interface{}, error) {
var emptyResult interface{}
// check for white-listed variables
if ctx.isWhiteListed(query) {
if !ctx.isWhiteListed(query) {
return emptyResult, fmt.Errorf("variable %s cannot be used", query)
}
// compile the query
queryPath, err := jmespath.Compile(query)
if err != nil {
glog.V(4).Infof("incorrect query %s: %v", query, err)
ctx.log.Error(err, "incorrect query", "query", query)
return emptyResult, fmt.Errorf("incorrect query %s: %v", query, err)
}
// search
@ -28,21 +28,24 @@ func (ctx *Context) Query(query string) (interface{}, error) {
var data interface{}
if err := json.Unmarshal(ctx.jsonRaw, &data); err != nil {
glog.V(4).Infof("failed to unmarshall context: %v", err)
ctx.log.Error(err, "failed to unmarshal context")
return emptyResult, fmt.Errorf("failed to unmarshall context: %v", err)
}
result, err := queryPath.Search(data)
if err != nil {
glog.V(4).Infof("failed to search query %s: %v", query, err)
ctx.log.Error(err, "failed to search query", "query", query)
return emptyResult, fmt.Errorf("failed to search query %s: %v", query, err)
}
return result, nil
}
func (ctx *Context) isWhiteListed(variable string) bool {
if len(ctx.whiteListVars) == 0 {
return true
}
for _, wVar := range ctx.whiteListVars {
if wVar == variable {
if strings.HasPrefix(variable, wVar) {
return true
}
}

5
pkg/engine/forceMutate.go
View file

@ -12,6 +12,7 @@ import (
"github.com/nirmata/kyverno/pkg/engine/utils"
"github.com/nirmata/kyverno/pkg/engine/variables"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/log"
)
func mutateResourceWithOverlay(resource unstructured.Unstructured, overlay interface{}) (unstructured.Unstructured, error) {
@ -57,7 +58,7 @@ func ForceMutate(ctx context.EvalInterface, policy kyverno.ClusterPolicy, resour
if mutation.Overlay != nil {
overlay := mutation.Overlay
if ctx != nil {
if overlay, err = variables.SubstituteVars(ctx, overlay); err != nil {
if overlay, err = variables.SubstituteVars(log.Log, ctx, overlay); err != nil {
return unstructured.Unstructured{}, err
}
} else {
@ -72,7 +73,7 @@ func ForceMutate(ctx context.EvalInterface, policy kyverno.ClusterPolicy, resour
if rule.Mutation.Patches != nil {
var resp response.RuleResponse
resp, resource = mutate.ProcessPatches(rule, resource)
resp, resource = mutate.ProcessPatches(log.Log, rule, resource)
if !resp.Success {
return unstructured.Unstructured{}, fmt.Errorf(resp.Message)
}

18
pkg/engine/generation.go
View file

@ -3,12 +3,14 @@ package engine
import (
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/engine/context"
"github.com/nirmata/kyverno/pkg/engine/response"
"github.com/nirmata/kyverno/pkg/engine/variables"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/log"
)
// Generate checks for validity of generate rule on the resource
@ -20,10 +22,11 @@ func Generate(policyContext PolicyContext) (resp response.EngineResponse) {
resource := policyContext.NewResource
admissionInfo := policyContext.AdmissionInfo
ctx := policyContext.Context
return filterRules(policy, resource, admissionInfo, ctx)
logger := log.Log.WithName("Generate").WithValues("policy", policy.Name, "kind", resource.GetKind(), "namespace", resource.GetNamespace(), "name", resource.GetName())
return filterRules(policy, resource, admissionInfo, ctx, logger)
}
func filterRule(rule kyverno.Rule, resource unstructured.Unstructured, admissionInfo kyverno.RequestInfo, ctx context.EvalInterface) *response.RuleResponse {
func filterRule(rule kyverno.Rule, resource unstructured.Unstructured, admissionInfo kyverno.RequestInfo, ctx context.EvalInterface, log logr.Logger) *response.RuleResponse {
if !rule.HasGenerate() {
return nil
}
@ -31,15 +34,14 @@ func filterRule(rule kyverno.Rule, resource unstructured.Unstructured, admission
startTime := time.Now()
if err := MatchesResourceDescription(resource, rule, admissionInfo); err != nil {
glog.V(4).Infof(err.Error())
return nil
}
// operate on the copy of the conditions, as we perform variable substitution
copyConditions := copyConditions(rule.Conditions)
// evaluate pre-conditions
if !variables.EvaluateConditions(ctx, copyConditions) {
glog.V(4).Infof("resource %s/%s does not satisfy the conditions for the rule ", resource.GetNamespace(), resource.GetName())
if !variables.EvaluateConditions(log, ctx, copyConditions) {
log.V(4).Info("preconditions not satisfied, skipping rule", "rule", rule.Name)
return nil
}
// build rule Response
@ -53,7 +55,7 @@ func filterRule(rule kyverno.Rule, resource unstructured.Unstructured, admission
}
}
func filterRules(policy kyverno.ClusterPolicy, resource unstructured.Unstructured, admissionInfo kyverno.RequestInfo, ctx context.EvalInterface) response.EngineResponse {
func filterRules(policy kyverno.ClusterPolicy, resource unstructured.Unstructured, admissionInfo kyverno.RequestInfo, ctx context.EvalInterface, log logr.Logger) response.EngineResponse {
resp := response.EngineResponse{
PolicyResponse: response.PolicyResponse{
Policy: policy.Name,
@ -66,7 +68,7 @@ func filterRules(policy kyverno.ClusterPolicy, resource unstructured.Unstructure
}
for _, rule := range policy.Spec.Rules {
if ruleResp := filterRule(rule, resource, admissionInfo, ctx); ruleResp != nil {
if ruleResp := filterRule(rule, resource, admissionInfo, ctx, log); ruleResp != nil {
resp.PolicyResponse.Rules = append(resp.PolicyResponse.Rules, *ruleResp)
}
}

49
pkg/engine/mutate/overlay.go
View file

@ -10,57 +10,59 @@ import (
"strings"
"time"
"github.com/golang/glog"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/log"
jsonpatch "github.com/evanphx/json-patch"
"github.com/go-logr/logr"
"github.com/nirmata/kyverno/pkg/engine/anchor"
"github.com/nirmata/kyverno/pkg/engine/response"
"github.com/nirmata/kyverno/pkg/engine/utils"
)
// ProcessOverlay processes mutation overlay on the resource
func ProcessOverlay(ruleName string, overlay interface{}, resource unstructured.Unstructured) (resp response.RuleResponse, patchedResource unstructured.Unstructured) {
func ProcessOverlay(log logr.Logger, ruleName string, overlay interface{}, resource unstructured.Unstructured) (resp response.RuleResponse, patchedResource unstructured.Unstructured) {
startTime := time.Now()
glog.V(4).Infof("started applying overlay rule %q (%v)", ruleName, startTime)
logger := log.WithValues("rule", ruleName)
logger.V(4).Info("started applying overlay rule", "startTime", startTime)
resp.Name = ruleName
resp.Type = utils.Mutation.String()
defer func() {
resp.RuleStats.ProcessingTime = time.Since(startTime)
glog.V(4).Infof("finished applying overlay rule %q (%v)", resp.Name, resp.RuleStats.ProcessingTime)
logger.V(4).Info("finished applying overlay rule", "processingTime", resp.RuleStats.ProcessingTime)
}()
patches, overlayerr := processOverlayPatches(resource.UnstructuredContent(), overlay)
// resource does not satisfy the overlay pattern, we don't apply this rule
patches, overlayerr := processOverlayPatches(logger, resource.UnstructuredContent(), overlay)
if !reflect.DeepEqual(overlayerr, overlayError{}) {
switch overlayerr.statusCode {
// condition key is not present in the resource, don't apply this rule
// consider as success
case conditionNotPresent:
glog.V(3).Infof("Skip applying rule '%s' on resource '%s/%s/%s': %s", ruleName, resource.GetKind(), resource.GetNamespace(), resource.GetName(), overlayerr.ErrorMsg())
logger.V(3).Info("skip applying rule", "reason", "conditionNotPresent")
resp.Success = true
return resp, resource
// conditions are not met, don't apply this rule
case conditionFailure:
glog.V(3).Infof("Skip applying rule '%s' on resource '%s/%s/%s': %s", ruleName, resource.GetKind(), resource.GetNamespace(), resource.GetName(), overlayerr.ErrorMsg())
logger.V(3).Info("skip applying rule", "reason", "conditionFailure")
//TODO: send zero response and not consider this as applied?
resp.Success = true
resp.Message = overlayerr.ErrorMsg()
return resp, resource
// rule application failed
case overlayFailure:
glog.Errorf("Resource %s/%s/%s: failed to process overlay: %v in the rule %s", resource.GetKind(), resource.GetNamespace(), resource.GetName(), overlayerr.ErrorMsg(), ruleName)
logger.Info("failed to process overlay")
resp.Success = false
resp.Message = fmt.Sprintf("failed to process overlay: %v", overlayerr.ErrorMsg())
return resp, resource
default:
glog.Errorf("Resource %s/%s/%s: Unknown type of error: %v", resource.GetKind(), resource.GetNamespace(), resource.GetName(), overlayerr.Error())
logger.Info("failed to process overlay")
resp.Success = false
resp.Message = fmt.Sprintf("Unknown type of error: %v", overlayerr.Error())
return resp, resource
}
}
logger.V(4).Info("processing overlay rule", "patches", len(patches))
if len(patches) == 0 {
resp.Success = true
return resp, resource
@ -70,24 +72,25 @@ func ProcessOverlay(ruleName string, overlay interface{}, resource unstructured.
resourceRaw, err := resource.MarshalJSON()
if err != nil {
resp.Success = false
glog.Infof("unable to marshall resource: %v", err)
logger.Error(err, "failed to marshal resource")
resp.Message = fmt.Sprintf("failed to process JSON patches: %v", err)
return resp, resource
}
var patchResource []byte
logger.V(5).Info("applying overlay patches", "patches", string(utils.JoinPatches(patches)))
patchResource, err = utils.ApplyPatches(resourceRaw, patches)
if err != nil {
msg := fmt.Sprintf("failed to apply JSON patches: %v", err)
glog.V(2).Infof("%s, patches=%s", msg, string(utils.JoinPatches(patches)))
resp.Success = false
resp.Message = msg
return resp, resource
}
logger.V(5).Info("patched resource", "patches", string(patchResource))
err = patchedResource.UnmarshalJSON(patchResource)
if err != nil {
glog.Infof("failed to unmarshall resource to undstructured: %v", err)
logger.Error(err, "failed to unmarshal resource")
resp.Success = false
resp.Message = fmt.Sprintf("failed to process JSON patches: %v", err)
return resp, resource
@ -101,17 +104,17 @@ func ProcessOverlay(ruleName string, overlay interface{}, resource unstructured.
return resp, patchedResource
}
func processOverlayPatches(resource, overlay interface{}) ([][]byte, overlayError) {
if path, overlayerr := meetConditions(resource, overlay); !reflect.DeepEqual(overlayerr, overlayError{}) {
func processOverlayPatches(log logr.Logger, resource, overlay interface{}) ([][]byte, overlayError) {
if path, overlayerr := meetConditions(log, resource, overlay); !reflect.DeepEqual(overlayerr, overlayError{}) {
switch overlayerr.statusCode {
// anchor key does not exist in the resource, skip applying policy
case conditionNotPresent:
glog.V(4).Infof("Mutate rule: skip applying policy: %v at %s", overlayerr, path)
log.V(4).Info("skip applying policy", "path", path, "error", overlayerr)
return nil, newOverlayError(overlayerr.statusCode, fmt.Sprintf("Policy not applied, condition tag not present: %v at %s", overlayerr.ErrorMsg(), path))
// anchor key is not satisfied in the resource, skip applying policy
case conditionFailure:
// anchor key is not satisfied in the resource, skip applying policy
glog.V(4).Infof("Mutate rule: failed to validate condition at %s, err: %v", path, overlayerr)
log.V(4).Info("failed to validate condition", "path", path, "error", overlayerr)
return nil, newOverlayError(overlayerr.statusCode, fmt.Sprintf("Policy not applied, conditions are not met at %s, %v", path, overlayerr))
}
}
@ -383,7 +386,7 @@ func prepareJSONValue(overlay interface{}) string {
overlayWithoutAnchors := removeAnchorFromSubTree(overlay)
jsonOverlay, err := json.Marshal(overlayWithoutAnchors)
if err != nil || hasOnlyAnchors(overlay) {
glog.V(3).Info(err)
log.Log.Error(err, "failed to marshall withoutanchors or has only anchors")
return ""
}
@ -409,7 +412,7 @@ func removeAnchorFromSubTree(overlay interface{}) interface{} {
}
func removeAnchroFromMap(overlay map[string]interface{}) map[string]interface{} {
result := make(map[string]interface{}, 0)
result := make(map[string]interface{})
for k, v := range overlay {
result[getRawKeyIfWrappedWithAttributes(k)] = removeAnchorFromSubTree(v)
}

27
pkg/engine/mutate/overlayCondition.go
View file

@ -5,17 +5,18 @@ import (
"reflect"
"strconv"
"github.com/golang/glog"
"github.com/go-logr/logr"
"github.com/nirmata/kyverno/pkg/engine/anchor"
"github.com/nirmata/kyverno/pkg/engine/validate"
"sigs.k8s.io/controller-runtime/pkg/log"
)
func meetConditions(resource, overlay interface{}) (string, overlayError) {
return checkConditions(resource, overlay, "/")
func meetConditions(log logr.Logger, resource, overlay interface{}) (string, overlayError) {
return checkConditions(log, resource, overlay, "/")
}
// resource and overlay should be the same type
func checkConditions(resource, overlay interface{}, path string) (string, overlayError) {
func checkConditions(log logr.Logger, resource, overlay interface{}, path string) (string, overlayError) {
// overlay has no anchor, return true
if !hasNestedAnchors(overlay) {
return "", overlayError{}
@ -26,7 +27,7 @@ func checkConditions(resource, overlay interface{}, path string) (string, overla
// condition never be true in this case
if reflect.TypeOf(resource) != reflect.TypeOf(overlay) {
if hasNestedAnchors(overlay) {
glog.V(4).Infof("Found anchor on different types of element at path %s: overlay %T, resource %T", path, overlay, resource)
log.V(4).Info(fmt.Sprintf("Found anchor on different types of element at path %s: overlay %T, resource %T", path, overlay, resource))
return path, newOverlayError(conditionFailure,
fmt.Sprintf("Found anchor on different types of element at path %s: overlay %T %v, resource %T %v", path, overlay, overlay, resource, resource))
@ -44,7 +45,7 @@ func checkConditions(resource, overlay interface{}, path string) (string, overla
default:
// anchor on non map/array is invalid:
// - anchor defined on values
glog.Warningln("Found invalid conditional anchor: anchor defined on values")
log.Info("Found invalid conditional anchor: anchor defined on values")
return "", overlayError{}
}
}
@ -68,12 +69,12 @@ func checkConditionOnMap(resourceMap, overlayMap map[string]interface{}, path st
func checkConditionOnArray(resource, overlay []interface{}, path string) (string, overlayError) {
if 0 == len(overlay) {
glog.Infof("Mutate overlay pattern is empty, path %s", path)
log.Log.V(4).Info("Mutate overlay pattern is empty", "path", path)
return "", overlayError{}
}
if reflect.TypeOf(resource[0]) != reflect.TypeOf(overlay[0]) {
glog.V(4).Infof("Overlay array and resource array have elements of different types: %T and %T", overlay[0], resource[0])
log.Log.V(4).Info(fmt.Sprintf("Overlay array and resource array have elements of different types: %T and %T", overlay[0], resource[0]))
return path, newOverlayError(conditionFailure,
fmt.Sprintf("Overlay array and resource array have elements of different types: %T and %T", overlay[0], resource[0]))
}
@ -111,7 +112,7 @@ func validateConditionAnchorMap(resourceMap, anchors map[string]interface{}, pat
// resource - A: B2
func compareOverlay(resource, overlay interface{}, path string) (string, overlayError) {
if reflect.TypeOf(resource) != reflect.TypeOf(overlay) {
glog.V(4).Infof("Found anchor on different types of element: overlay %T, resource %T", overlay, resource)
log.Log.V(4).Info("Found anchor on different types of element: overlay %T, resource %T", overlay, resource)
return path, newOverlayError(conditionFailure, fmt.Sprintf("Found anchor on different types of element: overlay %T, resource %T", overlay, resource))
}
@ -139,8 +140,8 @@ func compareOverlay(resource, overlay interface{}, path string) (string, overlay
}
}
case string, float64, int, int64, bool, nil:
if !validate.ValidateValueWithPattern(resource, overlay) {
glog.V(4).Infof("Mutate rule: failed validating value %v with overlay %v", resource, overlay)
if !validate.ValidateValueWithPattern(log.Log, resource, overlay) {
log.Log.V(4).Info(fmt.Sprintf("Mutate rule: failed validating value %v with overlay %v", resource, overlay))
return path, newOverlayError(conditionFailure, fmt.Sprintf("Failed validating value %v with overlay %v", resource, overlay))
}
default:
@ -165,7 +166,7 @@ func validateNonAnchorOverlayMap(resourceMap, overlayWithoutAnchor map[string]in
continue
}
}
if newPath, err := checkConditions(resourceValue, overlayValue, curPath); !reflect.DeepEqual(err, overlayError{}) {
if newPath, err := checkConditions(log.Log, resourceValue, overlayValue, curPath); !reflect.DeepEqual(err, overlayError{}) {
return newPath, err
}
}
@ -179,7 +180,7 @@ func checkConditionsOnArrayOfSameTypes(resource, overlay []interface{}, path str
default:
for i, overlayElement := range overlay {
curPath := path + strconv.Itoa(i) + "/"
path, err := checkConditions(resource[i], overlayElement, curPath)
path, err := checkConditions(log.Log, resource[i], overlayElement, curPath)
if !reflect.DeepEqual(err, overlayError{}) {
return path, err
}

111
pkg/engine/mutate/overlayCondition_test.go
View file

@ -7,6 +7,7 @@ import (
"testing"
"gotest.tools/assert"
"sigs.k8s.io/controller-runtime/pkg/log"
)
func TestMeetConditions_NoAnchor(t *testing.T) {
@ -26,9 +27,11 @@ func TestMeetConditions_NoAnchor(t *testing.T) {
}`)
var overlay interface{}
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(overlayRaw, &overlay)
assert.Assert(t, reflect.DeepEqual(err, nil))
_, err = meetConditions(log.Log, nil, overlay)
_, err := meetConditions(nil, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
}
@ -78,10 +81,12 @@ func TestMeetConditions_conditionalAnchorOnMap(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err := meetConditions(resource, overlay)
_, err = meetConditions(log.Log, resource, overlay)
assert.Assert(t, !reflect.DeepEqual(err, overlayError{}))
overlayRaw = []byte(`
@ -99,9 +104,10 @@ func TestMeetConditions_conditionalAnchorOnMap(t *testing.T) {
]
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, overlayerr := meetConditions(resource, overlay)
_, overlayerr := meetConditions(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
}
@ -136,11 +142,14 @@ func TestMeetConditions_DifferentTypes(t *testing.T) {
}`)
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
// anchor exist
_, err := meetConditions(resource, overlay)
_, err = meetConditions(log.Log, resource, overlay)
assert.Assert(t, strings.Contains(err.Error(), "Found anchor on different types of element at path /subsets/"))
}
@ -190,10 +199,12 @@ func TestMeetConditions_anchosInSameObject(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err := meetConditions(resource, overlay)
_, err = meetConditions(log.Log, resource, overlay)
assert.Error(t, err, "[overlayError:0] Failed validating value 443 with overlay 444")
}
@ -248,10 +259,13 @@ func TestMeetConditions_anchorOnPeer(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(log.Log, resource, overlay)
_, err := meetConditions(resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
}
@ -325,10 +339,13 @@ func TestMeetConditions_anchorsOnMetaAndSpec(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(log.Log, resource, overlay)
_, err := meetConditions(resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
}
@ -406,10 +423,13 @@ func TestMeetConditions_anchorsOnPeer_single(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRawAnchorOnPeers, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRawAnchorOnPeers, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(log.Log, resource, overlay)
_, err := meetConditions(resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
}
@ -440,10 +460,13 @@ func TestMeetConditions_anchorsOnPeer_two(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRawAnchorOnPeers, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRawAnchorOnPeers, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(log.Log, resource, overlay)
_, err := meetConditions(resource, overlay)
assert.Error(t, err, "[overlayError:0] Failed validating value true with overlay false")
overlayRaw = []byte(`{
@ -470,9 +493,10 @@ func TestMeetConditions_anchorsOnPeer_two(t *testing.T) {
}
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(resource, overlay)
_, err = meetConditions(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
overlayRaw = []byte(`{
@ -499,9 +523,10 @@ func TestMeetConditions_anchorsOnPeer_two(t *testing.T) {
}
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(resource, overlay)
_, err = meetConditions(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
}
@ -532,10 +557,13 @@ func TestMeetConditions_anchorsOnPeer_multiple(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRawAnchorOnPeers, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRawAnchorOnPeers, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(log.Log, resource, overlay)
_, err := meetConditions(resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
overlayRaw = []byte(`{
@ -562,9 +590,10 @@ func TestMeetConditions_anchorsOnPeer_multiple(t *testing.T) {
}
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(resource, overlay)
_, err = meetConditions(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
overlayRaw = []byte(`{
@ -591,9 +620,10 @@ func TestMeetConditions_anchorsOnPeer_multiple(t *testing.T) {
}
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
_, err = meetConditions(resource, overlay)
_, err = meetConditions(log.Log, resource, overlay)
assert.Error(t, err, "[overlayError:0] Failed validating value ENV_VALUE with overlay ENV_VALUE1")
}
@ -649,10 +679,13 @@ func TestMeetConditions_AtleastOneExist(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
path, err := meetConditions(log.Log, resource, overlay)
path, err := meetConditions(resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
assert.Assert(t, len(path) == 0)
}

93
pkg/engine/mutate/overlay_test.go
View file

@ -8,6 +8,7 @@ import (
jsonpatch "github.com/evanphx/json-patch"
"github.com/nirmata/kyverno/pkg/engine/utils"
"gotest.tools/assert"
"sigs.k8s.io/controller-runtime/pkg/log"
)
func compareJSONAsMap(t *testing.T, expected, actual []byte) {
@ -63,10 +64,12 @@ func TestProcessOverlayPatches_NestedListWithAnchor(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, patches != nil)
@ -163,10 +166,12 @@ func TestProcessOverlayPatches_InsertIntoArray(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, patches != nil)
@ -284,10 +289,12 @@ func TestProcessOverlayPatches_TestInsertToArray(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, patches != nil)
@ -367,10 +374,12 @@ func TestProcessOverlayPatches_ImagePullPolicy(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, len(patches) != 0)
@ -456,9 +465,10 @@ func TestProcessOverlayPatches_ImagePullPolicy(t *testing.T) {
}
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, err = processOverlayPatches(resource, overlay)
patches, err = processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
assert.Assert(t, len(patches) != 0)
@ -492,9 +502,10 @@ func TestProcessOverlayPatches_ImagePullPolicy(t *testing.T) {
}
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, err = processOverlayPatches(resource, overlay)
patches, err = processOverlayPatches(log.Log, resource, overlay)
assert.Error(t, err, "[overlayError:0] Policy not applied, conditions are not met at /spec/template/metadata/labels/app/, [overlayError:0] Failed validating value nginx with overlay nginx1")
assert.Assert(t, len(patches) == 0)
}
@ -520,10 +531,12 @@ func TestProcessOverlayPatches_AddingAnchor(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, len(patches) != 0)
@ -605,10 +618,12 @@ func TestProcessOverlayPatches_AddingAnchorInsideListElement(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, len(patches) != 0)
@ -684,9 +699,10 @@ func TestProcessOverlayPatches_AddingAnchorInsideListElement(t *testing.T) {
}
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, err = processOverlayPatches(resource, overlay)
patches, err = processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(err, overlayError{}))
assert.Assert(t, len(patches) != 0)
@ -747,10 +763,12 @@ func TestProcessOverlayPatches_anchorOnPeer(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, len(patches) != 0)
@ -805,9 +823,10 @@ func TestProcessOverlayPatches_anchorOnPeer(t *testing.T) {
]
}`)
json.Unmarshal(overlayRaw, &overlay)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, err = processOverlayPatches(resource, overlay)
patches, err = processOverlayPatches(log.Log, resource, overlay)
assert.Error(t, err, "[overlayError:0] Policy not applied, conditions are not met at /subsets/0/ports/0/port/, [overlayError:0] Failed validating value 443 with overlay 444")
assert.Assert(t, len(patches) == 0)
}
@ -886,10 +905,12 @@ func TestProcessOverlayPatches_insertWithCondition(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, len(patches) != 0)
@ -997,10 +1018,12 @@ func TestProcessOverlayPatches_InsertIfNotPresentWithConditions(t *testing.T) {
var resource, overlay interface{}
json.Unmarshal(resourceRaw, &resource)
json.Unmarshal(overlayRaw, &overlay)
err := json.Unmarshal(resourceRaw, &resource)
assert.NilError(t, err)
err = json.Unmarshal(overlayRaw, &overlay)
assert.NilError(t, err)
patches, overlayerr := processOverlayPatches(resource, overlay)
patches, overlayerr := processOverlayPatches(log.Log, resource, overlay)
assert.Assert(t, reflect.DeepEqual(overlayerr, overlayError{}))
assert.Assert(t, len(patches) != 0)

17
pkg/engine/mutate/patches.go
View file

@ -6,7 +6,7 @@ import (
"strings"
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/engine/response"
"github.com/nirmata/kyverno/pkg/engine/utils"
@ -20,21 +20,22 @@ func applyPatch(resource []byte, patchRaw []byte) ([]byte, error) {
}
//ProcessPatches applies the patches on the resource and returns the patched resource
func ProcessPatches(rule kyverno.Rule, resource unstructured.Unstructured) (resp response.RuleResponse, patchedResource unstructured.Unstructured) {
func ProcessPatches(log logr.Logger, rule kyverno.Rule, resource unstructured.Unstructured) (resp response.RuleResponse, patchedResource unstructured.Unstructured) {
logger := log.WithValues("rule", rule.Name)
startTime := time.Now()
glog.V(4).Infof("started JSON patch rule %q (%v)", rule.Name, startTime)
logger.V(4).Info("started JSON patch", "startTime", startTime)
resp.Name = rule.Name
resp.Type = utils.Mutation.String()
defer func() {
resp.RuleStats.ProcessingTime = time.Since(startTime)
glog.V(4).Infof("finished JSON patch rule %q (%v)", resp.Name, resp.RuleStats.ProcessingTime)
logger.V(4).Info("finished JSON patch", "processingTime", resp.RuleStats.ProcessingTime)
}()
// convert to RAW
resourceRaw, err := resource.MarshalJSON()
if err != nil {
resp.Success = false
glog.Infof("unable to marshall resource: %v", err)
logger.Error(err, "failed to marshal resource")
resp.Message = fmt.Sprintf("failed to process JSON patches: %v", err)
return resp, resource
}
@ -45,14 +46,14 @@ func ProcessPatches(rule kyverno.Rule, resource unstructured.Unstructured) (resp
// JSON patch
patchRaw, err := json.Marshal(patch)
if err != nil {
glog.V(4).Infof("failed to marshall JSON patch %v: %v", patch, err)
logger.Error(err, "failed to marshal JSON patch")
errs = append(errs, err)
continue
}
patchResource, err := applyPatch(resourceRaw, patchRaw)
// TODO: continue on error if one of the patches fails, will add the failure event in such case
if err != nil && patch.Operation == "remove" {
glog.Info(err)
log.Error(err, "failed to process JSON path or patch is a 'remove' operation")
continue
}
if err != nil {
@ -77,7 +78,7 @@ func ProcessPatches(rule kyverno.Rule, resource unstructured.Unstructured) (resp
}
err = patchedResource.UnmarshalJSON(resourceRaw)
if err != nil {
glog.Infof("failed to unmarshall resource to undstructured: %v", err)
logger.Error(err, "failed to unmmarshal resource")
resp.Success = false
resp.Message = fmt.Sprintf("failed to process JSON patches: %v", err)
return resp, resource

19
pkg/engine/mutate/patches_test.go
View file

@ -5,6 +5,7 @@ import (
"gotest.tools/assert"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/log"
types "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/engine/utils"
@ -41,7 +42,7 @@ func TestProcessPatches_EmptyPatches(t *testing.T) {
if err != nil {
t.Error(err)
}
rr, _ := ProcessPatches(emptyRule, *resourceUnstructured)
rr, _ := ProcessPatches(log.Log, emptyRule, *resourceUnstructured)
assert.Check(t, rr.Success)
assert.Assert(t, len(rr.Patches) == 0)
}
@ -70,14 +71,14 @@ func makeRuleWithPatches(patches []types.Patch) types.Rule {
func TestProcessPatches_EmptyDocument(t *testing.T) {
rule := makeRuleWithPatch(makeAddIsMutatedLabelPatch())
rr, _ := ProcessPatches(rule, unstructured.Unstructured{})
rr, _ := ProcessPatches(log.Log, rule, unstructured.Unstructured{})
assert.Assert(t, !rr.Success)
assert.Assert(t, len(rr.Patches) == 0)
}
func TestProcessPatches_AllEmpty(t *testing.T) {
emptyRule := types.Rule{}
rr, _ := ProcessPatches(emptyRule, unstructured.Unstructured{})
rr, _ := ProcessPatches(log.Log, emptyRule, unstructured.Unstructured{})
assert.Check(t, !rr.Success)
assert.Assert(t, len(rr.Patches) == 0)
}
@ -90,7 +91,7 @@ func TestProcessPatches_AddPathDoesntExist(t *testing.T) {
if err != nil {
t.Error(err)
}
rr, _ := ProcessPatches(rule, *resourceUnstructured)
rr, _ := ProcessPatches(log.Log, rule, *resourceUnstructured)
assert.Check(t, !rr.Success)
assert.Assert(t, len(rr.Patches) == 0)
}
@ -102,7 +103,7 @@ func TestProcessPatches_RemovePathDoesntExist(t *testing.T) {
if err != nil {
t.Error(err)
}
rr, _ := ProcessPatches(rule, *resourceUnstructured)
rr, _ := ProcessPatches(log.Log, rule, *resourceUnstructured)
assert.Check(t, rr.Success)
assert.Assert(t, len(rr.Patches) == 0)
}
@ -115,7 +116,7 @@ func TestProcessPatches_AddAndRemovePathsDontExist_EmptyResult(t *testing.T) {
if err != nil {
t.Error(err)
}
rr, _ := ProcessPatches(rule, *resourceUnstructured)
rr, _ := ProcessPatches(log.Log, rule, *resourceUnstructured)
assert.Check(t, !rr.Success)
assert.Assert(t, len(rr.Patches) == 0)
}
@ -129,7 +130,7 @@ func TestProcessPatches_AddAndRemovePathsDontExist_ContinueOnError_NotEmptyResul
if err != nil {
t.Error(err)
}
rr, _ := ProcessPatches(rule, *resourceUnstructured)
rr, _ := ProcessPatches(log.Log, rule, *resourceUnstructured)
assert.Check(t, rr.Success)
assert.Assert(t, len(rr.Patches) != 0)
assertEqStringAndData(t, `{"path":"/metadata/labels/label3","op":"add","value":"label3Value"}`, rr.Patches[0])
@ -142,7 +143,7 @@ func TestProcessPatches_RemovePathDoesntExist_EmptyResult(t *testing.T) {
if err != nil {
t.Error(err)
}
rr, _ := ProcessPatches(rule, *resourceUnstructured)
rr, _ := ProcessPatches(log.Log, rule, *resourceUnstructured)
assert.Check(t, rr.Success)
assert.Assert(t, len(rr.Patches) == 0)
}
@ -155,7 +156,7 @@ func TestProcessPatches_RemovePathDoesntExist_NotEmptyResult(t *testing.T) {
if err != nil {
t.Error(err)
}
rr, _ := ProcessPatches(rule, *resourceUnstructured)
rr, _ := ProcessPatches(log.Log, rule, *resourceUnstructured)
assert.Check(t, rr.Success)
assert.Assert(t, len(rr.Patches) == 1)
assertEqStringAndData(t, `{"path":"/metadata/labels/label2","op":"add","value":"label2Value"}`, rr.Patches[0])

96
pkg/engine/mutation.go
View file

@ -1,16 +1,17 @@
package engine
import (
"reflect"
"encoding/json"
"strings"
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/engine/mutate"
"github.com/nirmata/kyverno/pkg/engine/response"
"github.com/nirmata/kyverno/pkg/engine/variables"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/log"
)
const (
@ -28,26 +29,25 @@ func Mutate(policyContext PolicyContext) (resp response.EngineResponse) {
policy := policyContext.Policy
resource := policyContext.NewResource
ctx := policyContext.Context
logger := log.Log.WithName("Mutate").WithValues("policy", policy.Name, "kind", resource.GetKind(), "namespace", resource.GetNamespace(), "name", resource.GetName())
logger.V(4).Info("start policy processing", "startTime", startTime)
startMutateResultResponse(&resp, policy, resource)
glog.V(4).Infof("started applying mutation rules of policy %q (%v)", policy.Name, startTime)
defer endMutateResultResponse(&resp, startTime)
defer endMutateResultResponse(logger, &resp, startTime)
patchedResource := policyContext.NewResource
for _, rule := range policy.Spec.Rules {
var ruleResponse response.RuleResponse
logger := logger.WithValues("rule", rule.Name)
//TODO: to be checked before calling the resources as well
if !rule.HasMutate() && !strings.Contains(PodControllers, resource.GetKind()) {
continue
}
startTime := time.Now()
glog.V(4).Infof("Time: Mutate matchAdmissionInfo %v", time.Since(startTime))
// check if the resource satisfies the filter conditions defined in the rule
//TODO: this needs to be extracted, to filter the resource so that we can avoid passing resources that
// dont statisfy a policy rule resource description
// dont satisfy a policy rule resource description
if err := MatchesResourceDescription(resource, rule, policyContext.AdmissionInfo); err != nil {
glog.V(4).Infof("resource %s/%s does not satisfy the resource description for the rule:\n%s", resource.GetNamespace(), resource.GetName(), err.Error())
logger.V(3).Info("resource not matched", "reason", err.Error())
continue
}
@ -55,8 +55,8 @@ func Mutate(policyContext PolicyContext) (resp response.EngineResponse) {
copyConditions := copyConditions(rule.Conditions)
// evaluate pre-conditions
// - handle variable subsitutions
if !variables.EvaluateConditions(ctx, copyConditions) {
glog.V(4).Infof("resource %s/%s does not satisfy the conditions for the rule ", resource.GetNamespace(), resource.GetName())
if !variables.EvaluateConditions(logger, ctx, copyConditions) {
logger.V(3).Info("resource fails the preconditions")
continue
}
@ -66,7 +66,7 @@ func Mutate(policyContext PolicyContext) (resp response.EngineResponse) {
overlay := mutation.Overlay
// subsiitue the variables
var err error
if overlay, err = variables.SubstituteVars(ctx, overlay); err != nil {
if overlay, err = variables.SubstituteVars(logger, ctx, overlay); err != nil {
// variable subsitution failed
ruleResponse.Success = false
ruleResponse.Message = err.Error()
@ -74,15 +74,13 @@ func Mutate(policyContext PolicyContext) (resp response.EngineResponse) {
continue
}
ruleResponse, patchedResource = mutate.ProcessOverlay(rule.Name, overlay, patchedResource)
ruleResponse, patchedResource = mutate.ProcessOverlay(logger, rule.Name, overlay, patchedResource)
if ruleResponse.Success {
// - overlay pattern does not match the resource conditions
if ruleResponse.Patches == nil {
glog.V(4).Infof(ruleResponse.Message)
continue
}
glog.V(4).Infof("Mutate overlay in rule '%s' successfully applied on %s/%s/%s", rule.Name, resource.GetKind(), resource.GetNamespace(), resource.GetName())
logger.V(4).Info("overlay applied succesfully")
}
resp.PolicyResponse.Rules = append(resp.PolicyResponse.Rules, ruleResponse)
@ -92,36 +90,61 @@ func Mutate(policyContext PolicyContext) (resp response.EngineResponse) {
// Process Patches
if rule.Mutation.Patches != nil {
var ruleResponse response.RuleResponse
ruleResponse, patchedResource = mutate.ProcessPatches(rule, patchedResource)
glog.Infof("Mutate patches in rule '%s' successfully applied on %s/%s/%s", rule.Name, resource.GetKind(), resource.GetNamespace(), resource.GetName())
ruleResponse, patchedResource = mutate.ProcessPatches(logger, rule, patchedResource)
logger.V(4).Info("patches applied successfully")
resp.PolicyResponse.Rules = append(resp.PolicyResponse.Rules, ruleResponse)
incrementAppliedRuleCount(&resp)
}
}
// insert annotation to podtemplate if resource is pod controller
// skip inserting on existing resource
if reflect.DeepEqual(policyContext.AdmissionInfo, kyverno.RequestInfo{}) {
continue
}
if strings.Contains(PodControllers, resource.GetKind()) {
// insert annotation to podtemplate if resource is pod controller
// skip inserting on existing resource
if autoGenPolicy(&policy) && strings.Contains(PodControllers, resource.GetKind()) {
if !patchedResourceHasPodControllerAnnotation(patchedResource) {
var ruleResponse response.RuleResponse
ruleResponse, patchedResource = mutate.ProcessOverlay(rule.Name, podTemplateRule, patchedResource)
ruleResponse, patchedResource = mutate.ProcessOverlay(logger, "podControllerAnnotation", podTemplateRule.Mutation.Overlay, patchedResource)
if !ruleResponse.Success {
glog.Errorf("Failed to insert annotation to podTemplate of %s/%s/%s: %s", resource.GetKind(), resource.GetNamespace(), resource.GetName(), ruleResponse.Message)
continue
}
if ruleResponse.Success && ruleResponse.Patches != nil {
glog.V(2).Infof("Inserted annotation to podTemplate of %s/%s/%s: %s", resource.GetKind(), resource.GetNamespace(), resource.GetName(), ruleResponse.Message)
resp.PolicyResponse.Rules = append(resp.PolicyResponse.Rules, ruleResponse)
logger.Info("failed to insert annotation for podTemplate", "error", ruleResponse.Message)
} else {
if ruleResponse.Success && ruleResponse.Patches != nil {
logger.V(2).Info("inserted annotation for podTemplate")
resp.PolicyResponse.Rules = append(resp.PolicyResponse.Rules, ruleResponse)
}
}
}
}
// send the patched resource
resp.PatchedResource = patchedResource
return resp
}
func autoGenPolicy(policy *kyverno.ClusterPolicy) bool {
annotations := policy.GetObjectMeta().GetAnnotations()
_, ok := annotations[PodControllersAnnotation]
return ok
}
func patchedResourceHasPodControllerAnnotation(resource unstructured.Unstructured) bool {
var podController struct {
Spec struct {
Template struct {
Metadata struct {
Annotations map[string]interface{} `json:"annotations"`
} `json:"metadata"`
} `json:"template"`
} `json:"spec"`
}
resourceRaw, _ := json.Marshal(resource.Object)
_ = json.Unmarshal(resourceRaw, &podController)
val, ok := podController.Spec.Template.Metadata.Annotations[PodTemplateAnnotation]
log.Log.Info("patchedResourceHasPodControllerAnnotation", "resourceRaw", string(resourceRaw), "val", val, "ok", ok)
return ok
}
func incrementAppliedRuleCount(resp *response.EngineResponse) {
resp.PolicyResponse.RulesAppliedCount++
}
@ -137,10 +160,9 @@ func startMutateResultResponse(resp *response.EngineResponse, policy kyverno.Clu
// TODO(shuting): set response with mutationFailureAction
}
func endMutateResultResponse(resp *response.EngineResponse, startTime time.Time) {
func endMutateResultResponse(logger logr.Logger, resp *response.EngineResponse, startTime time.Time) {
resp.PolicyResponse.ProcessingTime = time.Since(startTime)
glog.V(4).Infof("finished applying mutation rules policy %v (%v)", resp.PolicyResponse.Policy, resp.PolicyResponse.ProcessingTime)
glog.V(4).Infof("Mutation Rules appplied count %v for policy %q", resp.PolicyResponse.RulesAppliedCount, resp.PolicyResponse.Policy)
logger.V(4).Info("finished processing policy", "processingTime", resp.PolicyResponse.ProcessingTime, "mutationRulesApplied", resp.PolicyResponse.RulesAppliedCount)
}
// podTemplateRule mutate pod template with annotation
@ -153,7 +175,7 @@ var podTemplateRule = kyverno.Rule{
"template": map[string]interface{}{
"metadata": map[string]interface{}{
"annotations": map[string]interface{}{
"+(pod-policies.kyverno.io/autogen-applied)": "true",
"+(" + PodTemplateAnnotation + ")": "true",
},
},
},

18
pkg/engine/mutation_test.go
View file

@ -67,11 +67,17 @@ func Test_VariableSubstitutionOverlay(t *testing.T) {
expectedPatch := []byte(`{ "op": "add", "path": "/metadata/labels", "value":{"appname":"check-root-user"} }`)
var policy kyverno.ClusterPolicy
json.Unmarshal(rawPolicy, &policy)
err := json.Unmarshal(rawPolicy, &policy)
if err != nil {
t.Error(err)
}
resourceUnstructured, err := utils.ConvertToUnstructured(rawResource)
assert.NilError(t, err)
ctx := context.NewContext()
ctx.AddResource(rawResource)
err = ctx.AddResource(rawResource)
if err != nil {
t.Error(err)
}
value, err := ctx.Query("request.object.metadata.name")
t.Log(value)
if err != nil {
@ -139,19 +145,21 @@ func Test_variableSubstitutionPathNotExist(t *testing.T) {
}`)
var policy kyverno.ClusterPolicy
json.Unmarshal(policyraw, &policy)
err := json.Unmarshal(policyraw, &policy)
assert.NilError(t, err)
resourceUnstructured, err := utils.ConvertToUnstructured(resourceRaw)
assert.NilError(t, err)
ctx := context.NewContext()
ctx.AddResource(resourceRaw)
err = ctx.AddResource(resourceRaw)
assert.NilError(t, err)
policyContext := PolicyContext{
Policy: policy,
Context: ctx,
NewResource: *resourceUnstructured}
er := Mutate(policyContext)
expectedErrorStr := "[failed to resolve request.object.metadata.name1 at path /spec/name]"
expectedErrorStr := "could not find variable request.object.metadata.name1 at path /spec/name"
t.Log(er.PolicyResponse.Rules[0].Message)
assert.Equal(t, er.PolicyResponse.Rules[0].Message, expectedErrorStr)
}

70
pkg/engine/utils.go
View file

@ -9,8 +9,7 @@ import (
"github.com/nirmata/kyverno/pkg/utils"
authenticationv1 "k8s.io/api/authentication/v1"
rbacv1 "k8s.io/api/rbac/v1"
"github.com/golang/glog"
"sigs.k8s.io/controller-runtime/pkg/log"
"github.com/minio/minio/pkg/wildcard"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
@ -19,6 +18,8 @@ import (
"k8s.io/apimachinery/pkg/labels"
)
var ExcludeUserInfo = []string{"system:nodes", "system:serviceaccounts:kube-system", "system:kube-scheduler"}
//EngineStats stores in the statistics for a single application of resource
type EngineStats struct {
// average time required to process the policy rules on a resource
@ -43,7 +44,7 @@ func checkName(name, resourceName string) bool {
func checkNameSpace(namespaces []string, resourceNameSpace string) bool {
for _, namespace := range namespaces {
if resourceNameSpace == namespace {
if wildcard.Match(namespace, resourceNameSpace) {
return true
}
}
@ -53,7 +54,7 @@ func checkNameSpace(namespaces []string, resourceNameSpace string) bool {
func checkSelector(labelSelector *metav1.LabelSelector, resourceLabels map[string]string) (bool, error) {
selector, err := metav1.LabelSelectorAsSelector(labelSelector)
if err != nil {
glog.Error(err)
log.Log.Error(err, "failed to build label selector")
return false, err
}
@ -68,36 +69,41 @@ func doesResourceMatchConditionBlock(conditionBlock kyverno.ResourceDescription,
var errs []error
if len(conditionBlock.Kinds) > 0 {
if !checkKind(conditionBlock.Kinds, resource.GetKind()) {
errs = append(errs, fmt.Errorf("resource kind does not match conditionBlock"))
errs = append(errs, fmt.Errorf("kind does not match"))
}
}
if conditionBlock.Name != "" {
if !checkName(conditionBlock.Name, resource.GetName()) {
errs = append(errs, fmt.Errorf("resource name does not match conditionBlock"))
errs = append(errs, fmt.Errorf("name does not match"))
}
}
if len(conditionBlock.Namespaces) > 0 {
if !checkNameSpace(conditionBlock.Namespaces, resource.GetNamespace()) {
errs = append(errs, fmt.Errorf("resource namespace does not match conditionBlock"))
errs = append(errs, fmt.Errorf("namespace does not match"))
}
}
if conditionBlock.Selector != nil {
hasPassed, err := checkSelector(conditionBlock.Selector, resource.GetLabels())
if err != nil {
errs = append(errs, fmt.Errorf("could not parse selector block of the policy in conditionBlock: %v", err))
errs = append(errs, fmt.Errorf("failed to parse selector: %v", err))
} else {
if !hasPassed {
errs = append(errs, fmt.Errorf("resource does not match selector of given conditionBlock"))
errs = append(errs, fmt.Errorf("selector does not match"))
}
}
}
if len(userInfo.Roles) > 0 {
if !doesSliceContainsAnyOfTheseValues(userInfo.Roles, admissionInfo.Roles...) {
keys := append(admissionInfo.AdmissionUserInfo.Groups, admissionInfo.AdmissionUserInfo.Username)
if len(userInfo.Roles) > 0 &&
!utils.SliceContains(keys, ExcludeUserInfo...) {
if !utils.SliceContains(userInfo.Roles, admissionInfo.Roles...) {
errs = append(errs, fmt.Errorf("user info does not match roles for the given conditionBlock"))
}
}
if len(userInfo.ClusterRoles) > 0 {
if !doesSliceContainsAnyOfTheseValues(userInfo.ClusterRoles, admissionInfo.ClusterRoles...) {
if len(userInfo.ClusterRoles) > 0 &&
!utils.SliceContains(keys, ExcludeUserInfo...) {
if !utils.SliceContains(userInfo.ClusterRoles, admissionInfo.ClusterRoles...) {
errs = append(errs, fmt.Errorf("user info does not match clustersRoles for the given conditionBlock"))
}
}
@ -115,6 +121,14 @@ func matchSubjects(ruleSubjects []rbacv1.Subject, userInfo authenticationv1.User
const SaPrefix = "system:serviceaccount:"
userGroups := append(userInfo.Groups, userInfo.Username)
// TODO: see issue https://github.com/nirmata/kyverno/issues/861
ruleSubjects = append(ruleSubjects,
rbacv1.Subject{Kind: "Group", Name: "system:serviceaccounts:kube-system"},
rbacv1.Subject{Kind: "Group", Name: "system:nodes"},
rbacv1.Subject{Kind: "Group", Name: "system:kube-scheduler"},
)
for _, subject := range ruleSubjects {
switch subject.Kind {
case "ServiceAccount":
@ -135,22 +149,6 @@ func matchSubjects(ruleSubjects []rbacv1.Subject, userInfo authenticationv1.User
return false
}
func doesSliceContainsAnyOfTheseValues(slice []string, values ...string) bool {
var sliceElementsMap = make(map[string]bool, len(slice))
for _, sliceElement := range slice {
sliceElementsMap[sliceElement] = true
}
for _, value := range values {
if sliceElementsMap[value] {
return true
}
}
return false
}
//MatchesResourceDescription checks if the resource matches resource description of the rule or not
func MatchesResourceDescription(resourceRef unstructured.Unstructured, ruleRef kyverno.Rule, admissionInfoRef kyverno.RequestInfo) error {
rule := *ruleRef.DeepCopy()
@ -164,26 +162,28 @@ func MatchesResourceDescription(resourceRef unstructured.Unstructured, ruleRef k
}
// checking if resource matches the rule
if !reflect.DeepEqual(rule.MatchResources.ResourceDescription, kyverno.ResourceDescription{}) {
if !reflect.DeepEqual(rule.MatchResources.ResourceDescription, kyverno.ResourceDescription{}) ||
!reflect.DeepEqual(rule.MatchResources.UserInfo, kyverno.UserInfo{}) {
matchErrs := doesResourceMatchConditionBlock(rule.MatchResources.ResourceDescription, rule.MatchResources.UserInfo, admissionInfo, resource)
reasonsForFailure = append(reasonsForFailure, matchErrs...)
} else {
reasonsForFailure = append(reasonsForFailure, fmt.Errorf("match block in rule cannot be empty"))
reasonsForFailure = append(reasonsForFailure, fmt.Errorf("match cannot be empty"))
}
// checking if resource has been excluded
if !reflect.DeepEqual(rule.ExcludeResources.ResourceDescription, kyverno.ResourceDescription{}) {
if !reflect.DeepEqual(rule.ExcludeResources.ResourceDescription, kyverno.ResourceDescription{}) ||
!reflect.DeepEqual(rule.ExcludeResources.UserInfo, kyverno.UserInfo{}) {
excludeErrs := doesResourceMatchConditionBlock(rule.ExcludeResources.ResourceDescription, rule.ExcludeResources.UserInfo, admissionInfo, resource)
if excludeErrs == nil {
reasonsForFailure = append(reasonsForFailure, fmt.Errorf("resource has been excluded since it matches the exclude block"))
reasonsForFailure = append(reasonsForFailure, fmt.Errorf("resource excluded"))
}
}
// creating final error
var errorMessage = "rule has failed to match resource for the following reasons:"
var errorMessage = "rule not matched:"
for i, reasonForFailure := range reasonsForFailure {
if reasonForFailure != nil {
errorMessage += "\n" + fmt.Sprint(i+1) + ". " + reasonForFailure.Error()
errorMessage += "\n " + fmt.Sprint(i+1) + ". " + reasonForFailure.Error()
}
}

4
pkg/engine/utils/utils.go
View file

@ -30,7 +30,7 @@ func (ri RuleType) String() string {
}
// ApplyPatches patches given resource with given patches and returns patched document
// return origin resource if any error occurs
// return original resource if any error occurs
func ApplyPatches(resource []byte, patches [][]byte) ([]byte, error) {
joinedPatches := JoinPatches(patches)
patch, err := jsonpatch.DecodePatch(joinedPatches)
@ -42,6 +42,7 @@ func ApplyPatches(resource []byte, patches [][]byte) ([]byte, error) {
if err != nil {
return resource, err
}
return patchedDocument, err
}
@ -56,6 +57,7 @@ func ApplyPatchNew(resource, patch []byte) ([]byte, error) {
if err != nil {
return nil, err
}
return patchedResource, err
}

76
pkg/engine/validate/pattern.go
View file

@ -1,12 +1,13 @@
package validate
import (
"fmt"
"math"
"regexp"
"strconv"
"strings"
"github.com/golang/glog"
"github.com/go-logr/logr"
"github.com/minio/minio/pkg/wildcard"
"github.com/nirmata/kyverno/pkg/engine/operator"
apiresource "k8s.io/apimachinery/pkg/api/resource"
@ -21,52 +22,52 @@ const (
)
// ValidateValueWithPattern validates value with operators and wildcards
func ValidateValueWithPattern(value, pattern interface{}) bool {
func ValidateValueWithPattern(log logr.Logger, value, pattern interface{}) bool {
switch typedPattern := pattern.(type) {
case bool:
typedValue, ok := value.(bool)
if !ok {
glog.V(4).Infof("Expected bool, found %T", value)
log.V(4).Info("Expected type bool", "type", fmt.Sprintf("%T", value), "value", value)
return false
}
return typedPattern == typedValue
case int:
return validateValueWithIntPattern(value, int64(typedPattern))
return validateValueWithIntPattern(log, value, int64(typedPattern))
case int64:
return validateValueWithIntPattern(value, typedPattern)
return validateValueWithIntPattern(log, value, typedPattern)
case float64:
return validateValueWithFloatPattern(value, typedPattern)
return validateValueWithFloatPattern(log, value, typedPattern)
case string:
return validateValueWithStringPatterns(value, typedPattern)
return validateValueWithStringPatterns(log, value, typedPattern)
case nil:
return validateValueWithNilPattern(value)
return validateValueWithNilPattern(log, value)
case map[string]interface{}:
// TODO: check if this is ever called?
return validateValueWithMapPattern(value, typedPattern)
return validateValueWithMapPattern(log, value, typedPattern)
case []interface{}:
// TODO: check if this is ever called?
glog.Warning("Arrays as patterns are not supported")
log.Info("arrays as patterns is not supported")
return false
default:
glog.Warningf("Unknown type as pattern: %v", typedPattern)
log.Info("Unkown type", "type", fmt.Sprintf("%T", typedPattern), "value", typedPattern)
return false
}
}
func validateValueWithMapPattern(value interface{}, typedPattern map[string]interface{}) bool {
func validateValueWithMapPattern(log logr.Logger, value interface{}, typedPattern map[string]interface{}) bool {
// verify the type of the resource value is map[string]interface,
// we only check for existence of object, not the equality of content and value
//TODO: check if adding
_, ok := value.(map[string]interface{})
if !ok {
glog.Warningf("Expected map[string]interface{}, found %T\n", value)
log.Info("Expected type map[string]interface{}", "type", fmt.Sprintf("%T", value), "value", value)
return false
}
return true
}
// Handler for int values during validation process
func validateValueWithIntPattern(value interface{}, pattern int64) bool {
func validateValueWithIntPattern(log logr.Logger, value interface{}, pattern int64) bool {
switch typedValue := value.(type) {
case int:
return int64(typedValue) == pattern
@ -78,38 +79,38 @@ func validateValueWithIntPattern(value interface{}, pattern int64) bool {
return int64(typedValue) == pattern
}
glog.Warningf("Expected int, found float: %f\n", typedValue)
log.Info("Expected type int", "type", fmt.Sprintf("%T", typedValue), "value", typedValue)
return false
case string:
// extract int64 from string
int64Num, err := strconv.ParseInt(typedValue, 10, 64)
if err != nil {
glog.Warningf("Failed to parse int64 from string: %v", err)
log.Error(err, "Failed to parse int64 from string")
return false
}
return int64Num == pattern
default:
glog.Warningf("Expected int, found: %T\n", value)
log.Info("Expected type int", "type", fmt.Sprintf("%T", value), "value", value)
return false
}
}
// Handler for float values during validation process
func validateValueWithFloatPattern(value interface{}, pattern float64) bool {
func validateValueWithFloatPattern(log logr.Logger, value interface{}, pattern float64) bool {
switch typedValue := value.(type) {
case int:
// check that float has no fraction
if pattern == math.Trunc(pattern) {
return int(pattern) == value
}
glog.Warningf("Expected float, found int: %d\n", typedValue)
log.Info("Expected type float", "type", fmt.Sprintf("%T", typedValue), "value", typedValue)
return false
case int64:
// check that float has no fraction
if pattern == math.Trunc(pattern) {
return int64(pattern) == value
}
glog.Warningf("Expected float, found int: %d\n", typedValue)
log.Info("Expected type float", "type", fmt.Sprintf("%T", typedValue), "value", typedValue)
return false
case float64:
return typedValue == pattern
@ -117,18 +118,18 @@ func validateValueWithFloatPattern(value interface{}, pattern float64) bool {
// extract float64 from string
float64Num, err := strconv.ParseFloat(typedValue, 64)
if err != nil {
glog.Warningf("Failed to parse float64 from string: %v", err)
log.Error(err, "Failed to parse float64 from string")
return false
}
return float64Num == pattern
default:
glog.Warningf("Expected float, found: %T\n", value)
log.Info("Expected type float", "type", fmt.Sprintf("%T", value), "value", value)
return false
}
}
// Handler for nil values during validation process
func validateValueWithNilPattern(value interface{}) bool {
func validateValueWithNilPattern(log logr.Logger, value interface{}) bool {
switch typed := value.(type) {
case float64:
return typed == 0.0
@ -143,20 +144,20 @@ func validateValueWithNilPattern(value interface{}) bool {
case nil:
return true
case map[string]interface{}, []interface{}:
glog.Warningf("Maps and arrays could not be checked with nil pattern")
log.Info("Maps and arrays could not be checked with nil pattern")
return false
default:
glog.Warningf("Unknown type as value when checking for nil pattern: %T\n", value)
log.Info("Unknown type as value when checking for nil pattern", "type", fmt.Sprintf("%T", value), "value", value)
return false
}
}
// Handler for pattern values during validation process
func validateValueWithStringPatterns(value interface{}, pattern string) bool {
func validateValueWithStringPatterns(log logr.Logger, value interface{}, pattern string) bool {
statements := strings.Split(pattern, "|")
for _, statement := range statements {
statement = strings.Trim(statement, " ")
if validateValueWithStringPattern(value, statement) {
if validateValueWithStringPattern(log, value, statement) {
return true
}
}
@ -166,24 +167,24 @@ func validateValueWithStringPatterns(value interface{}, pattern string) bool {
// Handler for single pattern value during validation process
// Detects if pattern has a number
func validateValueWithStringPattern(value interface{}, pattern string) bool {
func validateValueWithStringPattern(log logr.Logger, value interface{}, pattern string) bool {
operator := operator.GetOperatorFromStringPattern(pattern)
pattern = pattern[len(operator):]
number, str := getNumberAndStringPartsFromPattern(pattern)
if "" == number {
return validateString(value, str, operator)
return validateString(log, value, str, operator)
}
return validateNumberWithStr(value, pattern, operator)
return validateNumberWithStr(log, value, pattern, operator)
}
// Handler for string values
func validateString(value interface{}, pattern string, operatorVariable operator.Operator) bool {
func validateString(log logr.Logger, value interface{}, pattern string, operatorVariable operator.Operator) bool {
if operator.NotEqual == operatorVariable || operator.Equal == operatorVariable {
strValue, ok := value.(string)
if !ok {
glog.Warningf("Expected string, found %T\n", value)
log.Info("Expected type string", "type", fmt.Sprintf("%T", value), "value", value)
return false
}
@ -195,17 +196,16 @@ func validateString(value interface{}, pattern string, operatorVariable operator
return wildcardResult
}
glog.Warningf("Operators >, >=, <, <= are not applicable to strings")
log.Info("Operators >, >=, <, <= are not applicable to strings")
return false
}
// validateNumberWithStr compares quantity if pattern type is quantity
// or a wildcard match to pattern string
func validateNumberWithStr(value interface{}, pattern string, operator operator.Operator) bool {
func validateNumberWithStr(log logr.Logger, value interface{}, pattern string, operator operator.Operator) bool {
typedValue, err := convertToString(value)
if err != nil {
glog.Warning(err)
log.Error(err, "failed to convert to string")
return false
}
@ -214,7 +214,7 @@ func validateNumberWithStr(value interface{}, pattern string, operator operator.
if err == nil {
valueQuan, err := apiresource.ParseQuantity(typedValue)
if err != nil {
glog.Warningf("Invalid quantity in resource %s, err: %v\n", typedValue, err)
log.Error(err, "invalid quantity in resource", "type", fmt.Sprintf("%T", typedValue), "value", typedValue)
return false
}
@ -223,7 +223,7 @@ func validateNumberWithStr(value interface{}, pattern string, operator operator.
// 2. wildcard match
if !wildcard.Match(pattern, typedValue) {
glog.Warningf("Value '%s' has not passed wildcard check: %s", typedValue, pattern)
log.Info("value failed wildcard check", "type", fmt.Sprintf("%T", typedValue), "value", typedValue, "check", pattern)
return false
}
return true

113
pkg/engine/validate/pattern_test.go
View file

@ -6,13 +6,14 @@ import (
"github.com/nirmata/kyverno/pkg/engine/operator"
"gotest.tools/assert"
"sigs.k8s.io/controller-runtime/pkg/log"
)
func TestValidateValueWithPattern_Bool(t *testing.T) {
assert.Assert(t, ValidateValueWithPattern(true, true))
assert.Assert(t, !ValidateValueWithPattern(true, false))
assert.Assert(t, !ValidateValueWithPattern(false, true))
assert.Assert(t, ValidateValueWithPattern(false, false))
assert.Assert(t, ValidateValueWithPattern(log.Log, true, true))
assert.Assert(t, !ValidateValueWithPattern(log.Log, true, false))
assert.Assert(t, !ValidateValueWithPattern(log.Log, false, true))
assert.Assert(t, ValidateValueWithPattern(log.Log, false, false))
}
func TestValidateString_AsteriskTest(t *testing.T) {
@ -20,8 +21,8 @@ func TestValidateString_AsteriskTest(t *testing.T) {
value := "anything"
empty := ""
assert.Assert(t, validateString(value, pattern, operator.Equal))
assert.Assert(t, validateString(empty, pattern, operator.Equal))
assert.Assert(t, validateString(log.Log, value, pattern, operator.Equal))
assert.Assert(t, validateString(log.Log, empty, pattern, operator.Equal))
}
func TestValidateString_LeftAsteriskTest(t *testing.T) {
@ -29,32 +30,32 @@ func TestValidateString_LeftAsteriskTest(t *testing.T) {
value := "leftright"
right := "right"
assert.Assert(t, validateString(value, pattern, operator.Equal))
assert.Assert(t, validateString(right, pattern, operator.Equal))
assert.Assert(t, validateString(log.Log, value, pattern, operator.Equal))
assert.Assert(t, validateString(log.Log, right, pattern, operator.Equal))
value = "leftmiddle"
middle := "middle"
assert.Assert(t, !validateString(value, pattern, operator.Equal))
assert.Assert(t, !validateString(middle, pattern, operator.Equal))
assert.Assert(t, !validateString(log.Log, value, pattern, operator.Equal))
assert.Assert(t, !validateString(log.Log, middle, pattern, operator.Equal))
}
func TestValidateString_MiddleAsteriskTest(t *testing.T) {
pattern := "ab*ba"
value := "abbeba"
assert.Assert(t, validateString(value, pattern, operator.Equal))
assert.Assert(t, validateString(log.Log, value, pattern, operator.Equal))
value = "abbca"
assert.Assert(t, !validateString(value, pattern, operator.Equal))
assert.Assert(t, !validateString(log.Log, value, pattern, operator.Equal))
}
func TestValidateString_QuestionMark(t *testing.T) {
pattern := "ab?ba"
value := "abbba"
assert.Assert(t, validateString(value, pattern, operator.Equal))
assert.Assert(t, validateString(log.Log, value, pattern, operator.Equal))
value = "abbbba"
assert.Assert(t, !validateString(value, pattern, operator.Equal))
assert.Assert(t, !validateString(log.Log, value, pattern, operator.Equal))
}
func TestValidateValueWithPattern_BoolInJson(t *testing.T) {
@ -76,7 +77,7 @@ func TestValidateValueWithPattern_BoolInJson(t *testing.T) {
err = json.Unmarshal(rawValue, &value)
assert.Assert(t, err)
assert.Assert(t, ValidateValueWithPattern(value["key"], pattern["key"]))
assert.Assert(t, ValidateValueWithPattern(log.Log, value["key"], pattern["key"]))
}
func TestValidateValueWithPattern_NullPatternStringValue(t *testing.T) {
@ -98,7 +99,7 @@ func TestValidateValueWithPattern_NullPatternStringValue(t *testing.T) {
err = json.Unmarshal(rawValue, &value)
assert.Assert(t, err)
assert.Assert(t, !ValidateValueWithPattern(value["key"], pattern["key"]))
assert.Assert(t, !ValidateValueWithPattern(log.Log, value["key"], pattern["key"]))
}
func TestValidateValueWithPattern_NullPatternDefaultString(t *testing.T) {
@ -120,7 +121,7 @@ func TestValidateValueWithPattern_NullPatternDefaultString(t *testing.T) {
err = json.Unmarshal(rawValue, &value)
assert.Assert(t, err)
assert.Assert(t, ValidateValueWithPattern(value["key"], pattern["key"]))
assert.Assert(t, ValidateValueWithPattern(log.Log, value["key"], pattern["key"]))
}
func TestValidateValueWithPattern_NullPatternDefaultFloat(t *testing.T) {
@ -142,7 +143,7 @@ func TestValidateValueWithPattern_NullPatternDefaultFloat(t *testing.T) {
err = json.Unmarshal(rawValue, &value)
assert.Assert(t, err)
assert.Assert(t, ValidateValueWithPattern(value["key"], pattern["key"]))
assert.Assert(t, ValidateValueWithPattern(log.Log, value["key"], pattern["key"]))
}
func TestValidateValueWithPattern_NullPatternDefaultInt(t *testing.T) {
@ -164,7 +165,7 @@ func TestValidateValueWithPattern_NullPatternDefaultInt(t *testing.T) {
err = json.Unmarshal(rawValue, &value)
assert.Assert(t, err)
assert.Assert(t, ValidateValueWithPattern(value["key"], pattern["key"]))
assert.Assert(t, ValidateValueWithPattern(log.Log, value["key"], pattern["key"]))
}
func TestValidateValueWithPattern_NullPatternDefaultBool(t *testing.T) {
@ -186,77 +187,77 @@ func TestValidateValueWithPattern_NullPatternDefaultBool(t *testing.T) {
err = json.Unmarshal(rawValue, &value)
assert.Assert(t, err)
assert.Assert(t, ValidateValueWithPattern(value["key"], pattern["key"]))
assert.Assert(t, ValidateValueWithPattern(log.Log, value["key"], pattern["key"]))
}
func TestValidateValueWithPattern_StringsLogicalOr(t *testing.T) {
pattern := "192.168.88.1 | 10.100.11.*"
value := "10.100.11.54"
assert.Assert(t, ValidateValueWithPattern(value, pattern))
assert.Assert(t, ValidateValueWithPattern(log.Log, value, pattern))
}
func TestValidateValueWithPattern_EqualTwoFloats(t *testing.T) {
assert.Assert(t, ValidateValueWithPattern(7.0, 7.000))
assert.Assert(t, ValidateValueWithPattern(log.Log, 7.0, 7.000))
}
func TestValidateValueWithNilPattern_NullPatternStringValue(t *testing.T) {
assert.Assert(t, !validateValueWithNilPattern("value"))
assert.Assert(t, !validateValueWithNilPattern(log.Log, "value"))
}
func TestValidateValueWithNilPattern_NullPatternDefaultString(t *testing.T) {
assert.Assert(t, validateValueWithNilPattern(""))
assert.Assert(t, validateValueWithNilPattern(log.Log, ""))
}
func TestValidateValueWithNilPattern_NullPatternDefaultFloat(t *testing.T) {
assert.Assert(t, validateValueWithNilPattern(0.0))
assert.Assert(t, validateValueWithNilPattern(log.Log, 0.0))
}
func TestValidateValueWithNilPattern_NullPatternFloat(t *testing.T) {
assert.Assert(t, !validateValueWithNilPattern(0.1))
assert.Assert(t, !validateValueWithNilPattern(log.Log, 0.1))
}
func TestValidateValueWithNilPattern_NullPatternDefaultInt(t *testing.T) {
assert.Assert(t, validateValueWithNilPattern(0))
assert.Assert(t, validateValueWithNilPattern(log.Log, 0))
}
func TestValidateValueWithNilPattern_NullPatternInt(t *testing.T) {
assert.Assert(t, !validateValueWithNilPattern(1))
assert.Assert(t, !validateValueWithNilPattern(log.Log, 1))
}
func TestValidateValueWithNilPattern_NullPatternDefaultBool(t *testing.T) {
assert.Assert(t, validateValueWithNilPattern(false))
assert.Assert(t, validateValueWithNilPattern(log.Log, false))
}
func TestValidateValueWithNilPattern_NullPatternTrueBool(t *testing.T) {
assert.Assert(t, !validateValueWithNilPattern(true))
assert.Assert(t, !validateValueWithNilPattern(log.Log, true))
}
func TestValidateValueWithFloatPattern_FloatValue(t *testing.T) {
assert.Assert(t, validateValueWithFloatPattern(7.9914, 7.9914))
assert.Assert(t, validateValueWithFloatPattern(log.Log, 7.9914, 7.9914))
}
func TestValidateValueWithFloatPattern_FloatValueNotPass(t *testing.T) {
assert.Assert(t, !validateValueWithFloatPattern(7.9914, 7.99141))
assert.Assert(t, !validateValueWithFloatPattern(log.Log, 7.9914, 7.99141))
}
func TestValidateValueWithFloatPattern_FloatPatternWithoutFractionIntValue(t *testing.T) {
assert.Assert(t, validateValueWithFloatPattern(7, 7.000000))
assert.Assert(t, validateValueWithFloatPattern(log.Log, 7, 7.000000))
}
func TestValidateValueWithFloatPattern_FloatPatternWithoutFraction(t *testing.T) {
assert.Assert(t, validateValueWithFloatPattern(7.000000, 7.000000))
assert.Assert(t, validateValueWithFloatPattern(log.Log, 7.000000, 7.000000))
}
func TestValidateValueWithIntPattern_FloatValueWithoutFraction(t *testing.T) {
assert.Assert(t, validateValueWithFloatPattern(7.000000, 7))
assert.Assert(t, validateValueWithFloatPattern(log.Log, 7.000000, 7))
}
func TestValidateValueWithIntPattern_FloatValueWitFraction(t *testing.T) {
assert.Assert(t, !validateValueWithFloatPattern(7.000001, 7))
assert.Assert(t, !validateValueWithFloatPattern(log.Log, 7.000001, 7))
}
func TestValidateValueWithIntPattern_NotPass(t *testing.T) {
assert.Assert(t, !validateValueWithFloatPattern(8, 7))
assert.Assert(t, !validateValueWithFloatPattern(log.Log, 8, 7))
}
func TestGetNumberAndStringPartsFromPattern_NumberAndString(t *testing.T) {
@ -290,35 +291,35 @@ func TestGetNumberAndStringPartsFromPattern_Empty(t *testing.T) {
}
func TestValidateNumberWithStr_LessFloatAndInt(t *testing.T) {
assert.Assert(t, validateNumberWithStr(7.00001, "7.000001", operator.More))
assert.Assert(t, validateNumberWithStr(7.00001, "7", operator.NotEqual))
assert.Assert(t, validateNumberWithStr(log.Log, 7.00001, "7.000001", operator.More))
assert.Assert(t, validateNumberWithStr(log.Log, 7.00001, "7", operator.NotEqual))
assert.Assert(t, validateNumberWithStr(7.0000, "7", operator.Equal))
assert.Assert(t, !validateNumberWithStr(6.000000001, "6", operator.Less))
assert.Assert(t, validateNumberWithStr(log.Log, 7.0000, "7", operator.Equal))
assert.Assert(t, !validateNumberWithStr(log.Log, 6.000000001, "6", operator.Less))
}
func TestValidateQuantity_InvalidQuantity(t *testing.T) {
assert.Assert(t, !validateNumberWithStr("1024Gi", "", operator.Equal))
assert.Assert(t, !validateNumberWithStr("gii", "1024Gi", operator.Equal))
assert.Assert(t, !validateNumberWithStr(log.Log, "1024Gi", "", operator.Equal))
assert.Assert(t, !validateNumberWithStr(log.Log, "gii", "1024Gi", operator.Equal))
}
func TestValidateQuantity_Equal(t *testing.T) {
assert.Assert(t, validateNumberWithStr("1024Gi", "1024Gi", operator.Equal))
assert.Assert(t, validateNumberWithStr("1024Mi", "1Gi", operator.Equal))
assert.Assert(t, validateNumberWithStr("0.2", "200m", operator.Equal))
assert.Assert(t, validateNumberWithStr("500", "500", operator.Equal))
assert.Assert(t, !validateNumberWithStr("2048", "1024", operator.Equal))
assert.Assert(t, validateNumberWithStr(1024, "1024", operator.Equal))
assert.Assert(t, validateNumberWithStr(log.Log, "1024Gi", "1024Gi", operator.Equal))
assert.Assert(t, validateNumberWithStr(log.Log, "1024Mi", "1Gi", operator.Equal))
assert.Assert(t, validateNumberWithStr(log.Log, "0.2", "200m", operator.Equal))
assert.Assert(t, validateNumberWithStr(log.Log, "500", "500", operator.Equal))
assert.Assert(t, !validateNumberWithStr(log.Log, "2048", "1024", operator.Equal))
assert.Assert(t, validateNumberWithStr(log.Log, 1024, "1024", operator.Equal))
}
func TestValidateQuantity_Operation(t *testing.T) {
assert.Assert(t, validateNumberWithStr("1Gi", "1000Mi", operator.More))
assert.Assert(t, validateNumberWithStr("1G", "1Gi", operator.Less))
assert.Assert(t, validateNumberWithStr("500m", "0.5", operator.MoreEqual))
assert.Assert(t, validateNumberWithStr("1", "500m", operator.MoreEqual))
assert.Assert(t, validateNumberWithStr("0.5", ".5", operator.LessEqual))
assert.Assert(t, validateNumberWithStr("0.2", ".5", operator.LessEqual))
assert.Assert(t, validateNumberWithStr("0.2", ".5", operator.NotEqual))
assert.Assert(t, validateNumberWithStr(log.Log, "1Gi", "1000Mi", operator.More))
assert.Assert(t, validateNumberWithStr(log.Log, "1G", "1Gi", operator.Less))
assert.Assert(t, validateNumberWithStr(log.Log, "500m", "0.5", operator.MoreEqual))
assert.Assert(t, validateNumberWithStr(log.Log, "1", "500m", operator.MoreEqual))
assert.Assert(t, validateNumberWithStr(log.Log, "0.5", ".5", operator.LessEqual))
assert.Assert(t, validateNumberWithStr(log.Log, "0.2", ".5", operator.LessEqual))
assert.Assert(t, validateNumberWithStr(log.Log, "0.2", ".5", operator.NotEqual))
}
func TestGetOperatorFromStringPattern_OneChar(t *testing.T) {

60
pkg/engine/validate/validate.go
View file

@ -3,20 +3,20 @@ package validate
import (
"errors"
"fmt"
"path/filepath"
"path"
"reflect"
"strconv"
"strings"
"github.com/golang/glog"
"github.com/go-logr/logr"
"github.com/nirmata/kyverno/pkg/engine/anchor"
"github.com/nirmata/kyverno/pkg/engine/operator"
)
// ValidateResourceWithPattern is a start of element-by-element validation process
// It assumes that validation is started from root, so "/" is passed
func ValidateResourceWithPattern(resource, pattern interface{}) (string, error) {
path, err := validateResourceElement(resource, pattern, pattern, "/")
func ValidateResourceWithPattern(log logr.Logger, resource, pattern interface{}) (string, error) {
path, err := validateResourceElement(log, resource, pattern, pattern, "/")
if err != nil {
return path, err
}
@ -27,44 +27,44 @@ func ValidateResourceWithPattern(resource, pattern interface{}) (string, error)
// validateResourceElement detects the element type (map, array, nil, string, int, bool, float)
// and calls corresponding handler
// Pattern tree and resource tree can have different structure. In this case validation fails
func validateResourceElement(resourceElement, patternElement, originPattern interface{}, path string) (string, error) {
func validateResourceElement(log logr.Logger, resourceElement, patternElement, originPattern interface{}, path string) (string, error) {
var err error
switch typedPatternElement := patternElement.(type) {
// map
case map[string]interface{}:
typedResourceElement, ok := resourceElement.(map[string]interface{})
if !ok {
glog.V(4).Infof("Pattern and resource have different structures. Path: %s. Expected %T, found %T", path, patternElement, resourceElement)
log.V(4).Info("Pattern and resource have different structures.", "path", path, "expected", fmt.Sprintf("%T", patternElement), "current", fmt.Sprintf("%T", resourceElement))
return path, fmt.Errorf("Pattern and resource have different structures. Path: %s. Expected %T, found %T", path, patternElement, resourceElement)
}
return validateMap(typedResourceElement, typedPatternElement, originPattern, path)
return validateMap(log, typedResourceElement, typedPatternElement, originPattern, path)
// array
case []interface{}:
typedResourceElement, ok := resourceElement.([]interface{})
if !ok {
glog.V(4).Infof("Pattern and resource have different structures. Path: %s. Expected %T, found %T", path, patternElement, resourceElement)
log.V(4).Info("Pattern and resource have different structures.", "path", path, "expected", fmt.Sprintf("%T", patternElement), "current", fmt.Sprintf("%T", resourceElement))
return path, fmt.Errorf("Validation rule Failed at path %s, resource does not satisfy the expected overlay pattern", path)
}
return validateArray(typedResourceElement, typedPatternElement, originPattern, path)
return validateArray(log, typedResourceElement, typedPatternElement, originPattern, path)
// elementary values
case string, float64, int, int64, bool, nil:
/*Analyze pattern */
if checkedPattern := reflect.ValueOf(patternElement); checkedPattern.Kind() == reflect.String {
if isStringIsReference(checkedPattern.String()) { //check for $ anchor
patternElement, err = actualizePattern(originPattern, checkedPattern.String(), path)
patternElement, err = actualizePattern(log, originPattern, checkedPattern.String(), path)
if err != nil {
return path, err
}
}
}
if !ValidateValueWithPattern(resourceElement, patternElement) {
if !ValidateValueWithPattern(log, resourceElement, patternElement) {
return path, fmt.Errorf("Validation rule failed at '%s' to validate value '%v' with pattern '%v'", path, resourceElement, patternElement)
}
default:
glog.V(4).Infof("Pattern contains unknown type %T. Path: %s", patternElement, path)
log.V(4).Info("Pattern contains unknown type", "path", path, "current", fmt.Sprintf("%T", patternElement))
return path, fmt.Errorf("Validation rule failed at '%s', pattern contains unknown type", path)
}
return "", nil
@ -72,7 +72,7 @@ func validateResourceElement(resourceElement, patternElement, originPattern inte
// If validateResourceElement detects map element inside resource and pattern trees, it goes to validateMap
// For each element of the map we must detect the type again, so we pass these elements to validateResourceElement
func validateMap(resourceMap, patternMap map[string]interface{}, origPattern interface{}, path string) (string, error) {
func validateMap(log logr.Logger, resourceMap, patternMap map[string]interface{}, origPattern interface{}, path string) (string, error) {
// check if there is anchor in pattern
// Phase 1 : Evaluate all the anchors
// Phase 2 : Evaluate non-anchors
@ -91,7 +91,7 @@ func validateMap(resourceMap, patternMap map[string]interface{}, origPattern int
if err != nil {
// If Conditional anchor fails then we dont process the resources
if anchor.IsConditionAnchor(key) {
glog.V(4).Infof("condition anchor did not satisfy, wont process the resources: %s", err)
log.Error(err, "condition anchor did not satisfy, wont process the resource")
return "", nil
}
return handlerPath, err
@ -109,7 +109,7 @@ func validateMap(resourceMap, patternMap map[string]interface{}, origPattern int
return "", nil
}
func validateArray(resourceArray, patternArray []interface{}, originPattern interface{}, path string) (string, error) {
func validateArray(log logr.Logger, resourceArray, patternArray []interface{}, originPattern interface{}, path string) (string, error) {
if 0 == len(patternArray) {
return path, fmt.Errorf("Pattern Array empty")
@ -119,7 +119,7 @@ func validateArray(resourceArray, patternArray []interface{}, originPattern inte
case map[string]interface{}:
// This is special case, because maps in arrays can have anchors that must be
// processed with the special way affecting the entire array
path, err := validateArrayOfMaps(resourceArray, typedPatternElement, originPattern, path)
path, err := validateArrayOfMaps(log, resourceArray, typedPatternElement, originPattern, path)
if err != nil {
return path, err
}
@ -127,7 +127,7 @@ func validateArray(resourceArray, patternArray []interface{}, originPattern inte
// In all other cases - detect type and handle each array element with validateResourceElement
for i, patternElement := range patternArray {
currentPath := path + strconv.Itoa(i) + "/"
path, err := validateResourceElement(resourceArray[i], patternElement, originPattern, currentPath)
path, err := validateResourceElement(log, resourceArray[i], patternElement, originPattern, currentPath)
if err != nil {
return path, err
}
@ -137,7 +137,7 @@ func validateArray(resourceArray, patternArray []interface{}, originPattern inte
return "", nil
}
func actualizePattern(origPattern interface{}, referencePattern, absolutePath string) (interface{}, error) {
func actualizePattern(log logr.Logger, origPattern interface{}, referencePattern, absolutePath string) (interface{}, error) {
var foundValue interface{}
referencePattern = strings.Trim(referencePattern, "$()")
@ -155,7 +155,7 @@ func actualizePattern(origPattern interface{}, referencePattern, absolutePath st
// value :=
actualPath := formAbsolutePath(referencePattern, absolutePath)
valFromReference, err := getValueFromReference(origPattern, actualPath)
valFromReference, err := getValueFromReference(log, origPattern, actualPath)
if err != nil {
return err, nil
}
@ -188,23 +188,23 @@ func valFromReferenceToString(value interface{}, operator string) (string, error
// returns absolute path
func formAbsolutePath(referencePath, absolutePath string) string {
if filepath.IsAbs(referencePath) {
if path.IsAbs(referencePath) {
return referencePath
}
return filepath.Join(absolutePath, referencePath)
return path.Join(absolutePath, referencePath)
}
//Prepares original pattern, path to value, and call traverse function
func getValueFromReference(origPattern interface{}, reference string) (interface{}, error) {
func getValueFromReference(log logr.Logger, origPattern interface{}, reference string) (interface{}, error) {
originalPatternMap := origPattern.(map[string]interface{})
reference = reference[1:]
statements := strings.Split(reference, "/")
return getValueFromPattern(originalPatternMap, statements, 0)
return getValueFromPattern(log, originalPatternMap, statements, 0)
}
func getValueFromPattern(patternMap map[string]interface{}, keys []string, currentKeyIndex int) (interface{}, error) {
func getValueFromPattern(log logr.Logger, patternMap map[string]interface{}, keys []string, currentKeyIndex int) (interface{}, error) {
for key, pattern := range patternMap {
rawKey := getRawKeyIfWrappedWithAttributes(key)
@ -221,19 +221,21 @@ func getValueFromPattern(patternMap map[string]interface{}, keys []string, curre
for i, value := range typedPattern {
resourceMap, ok := value.(map[string]interface{})
if !ok {
glog.V(4).Infof("Pattern and resource have different structures. Expected %T, found %T", pattern, value)
log.V(4).Info("Pattern and resource have different structures.", "expected", fmt.Sprintf("%T", pattern), "current", fmt.Sprintf("%T", value))
return nil, fmt.Errorf("Validation rule failed, resource does not have expected pattern %v", patternMap)
}
if keys[currentKeyIndex+1] == strconv.Itoa(i) {
return getValueFromPattern(resourceMap, keys, currentKeyIndex+2)
return getValueFromPattern(log, resourceMap, keys, currentKeyIndex+2)
}
// TODO : SA4004: the surrounding loop is unconditionally terminated (staticcheck)
return nil, errors.New("Reference to non-existent place in the document")
}
return nil, nil // Just a hack to fix the lint
}
return nil, errors.New("Reference to non-existent place in the document")
case map[string]interface{}:
if keys[currentKeyIndex] == rawKey {
return getValueFromPattern(typedPattern, keys, currentKeyIndex+1)
return getValueFromPattern(log, typedPattern, keys, currentKeyIndex+1)
}
return nil, errors.New("Reference to non-existent place in the document")
case string, float64, int, int64, bool, nil:
@ -251,12 +253,12 @@ func getValueFromPattern(patternMap map[string]interface{}, keys []string, curre
// validateArrayOfMaps gets anchors from pattern array map element, applies anchors logic
// and then validates each map due to the pattern
func validateArrayOfMaps(resourceMapArray []interface{}, patternMap map[string]interface{}, originPattern interface{}, path string) (string, error) {
func validateArrayOfMaps(log logr.Logger, resourceMapArray []interface{}, patternMap map[string]interface{}, originPattern interface{}, path string) (string, error) {
for i, resourceElement := range resourceMapArray {
// check the types of resource element
// expect it to be map, but can be anything ?:(
currentPath := path + strconv.Itoa(i) + "/"
returnpath, err := validateResourceElement(resourceElement, patternMap, originPattern, currentPath)
returnpath, err := validateResourceElement(log, resourceElement, patternMap, originPattern, currentPath)
if err != nil {
return returnpath, err
}

52
pkg/engine/validate/validate_test.go
View file

@ -5,6 +5,7 @@ import (
"testing"
"gotest.tools/assert"
"sigs.k8s.io/controller-runtime/pkg/log"
)
func TestValidateMap(t *testing.T) {
@ -100,7 +101,7 @@ func TestValidateMap(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateMap(resource, pattern, pattern, "/")
path, err := validateMap(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.NilError(t, err)
}
@ -196,7 +197,7 @@ func TestValidateMap_AsteriskForInt(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateMap(resource, pattern, pattern, "/")
path, err := validateMap(log.Log, resource, pattern, pattern, "/")
t.Log(path)
assert.NilError(t, err)
}
@ -289,7 +290,7 @@ func TestValidateMap_AsteriskForMap(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateMap(resource, pattern, pattern, "/")
path, err := validateMap(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.NilError(t, err)
}
@ -377,7 +378,7 @@ func TestValidateMap_AsteriskForArray(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateMap(resource, pattern, pattern, "/")
path, err := validateMap(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.NilError(t, err)
}
@ -468,7 +469,7 @@ func TestValidateMap_AsteriskFieldIsMissing(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateMap(resource, pattern, pattern, "/")
path, err := validateMap(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "/spec/template/spec/containers/0/")
assert.Assert(t, err != nil)
}
@ -557,9 +558,10 @@ func TestValidateMap_livenessProbeIsNull(t *testing.T) {
var pattern, resource map[string]interface{}
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
json.Unmarshal(rawMap, &resource)
err := json.Unmarshal(rawMap, &resource)
assert.NilError(t, err)
path, err := validateMap(resource, pattern, pattern, "/")
path, err := validateMap(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.NilError(t, err)
}
@ -649,7 +651,7 @@ func TestValidateMap_livenessProbeIsMissing(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateMap(resource, pattern, pattern, "/")
path, err := validateMap(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.NilError(t, err)
}
@ -695,7 +697,7 @@ func TestValidateMapElement_TwoElementsInArrayOnePass(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
// assert.Equal(t, path, "/1/object/0/key2/")
// assert.NilError(t, err)
@ -730,7 +732,7 @@ func TestValidateMapElement_OneElementInArrayPass(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.NilError(t, err)
}
@ -784,7 +786,7 @@ func TestValidateMap_CorrectRelativePathInConfig(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.NilError(t, err)
}
@ -838,7 +840,7 @@ func TestValidateMap_RelativePathDoesNotExists(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "/spec/containers/0/resources/requests/memory/")
assert.Assert(t, err != nil)
}
@ -892,7 +894,7 @@ func TestValidateMap_OnlyAnchorsInPath(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "/spec/containers/0/resources/requests/memory/")
assert.Assert(t, err != nil)
}
@ -946,7 +948,7 @@ func TestValidateMap_MalformedReferenceOnlyDolarMark(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "/spec/containers/0/resources/requests/memory/")
assert.Assert(t, err != nil)
}
@ -1000,7 +1002,7 @@ func TestValidateMap_RelativePathWithParentheses(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.NilError(t, err)
}
@ -1054,7 +1056,7 @@ func TestValidateMap_MalformedPath(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "/spec/containers/0/resources/requests/memory/")
assert.Assert(t, err != nil)
}
@ -1108,7 +1110,7 @@ func TestValidateMap_AbosolutePathExists(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.Assert(t, err == nil)
}
@ -1149,7 +1151,7 @@ func TestValidateMap_AbsolutePathToMetadata(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "")
assert.Assert(t, err == nil)
}
@ -1191,7 +1193,7 @@ func TestValidateMap_AbsolutePathToMetadata_fail(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "/spec/containers/0/image/")
assert.Assert(t, err != nil)
}
@ -1245,7 +1247,7 @@ func TestValidateMap_AbosolutePathDoesNotExists(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
assert.Assert(t, json.Unmarshal(rawMap, &resource))
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "/spec/containers/0/resources/requests/memory/")
assert.Assert(t, err != nil)
}
@ -1276,7 +1278,7 @@ func TestActualizePattern_GivenRelativePathThatExists(t *testing.T) {
assert.Assert(t, json.Unmarshal(rawPattern, &pattern))
pattern, err := actualizePattern(pattern, referencePath, absolutePath)
pattern, err := actualizePattern(log.Log, pattern, referencePath, absolutePath)
assert.Assert(t, err == nil)
}
@ -1344,10 +1346,12 @@ func TestValidateMapElement_OneElementInArrayNotPass(t *testing.T) {
]`)
var pattern, resource interface{}
json.Unmarshal(rawPattern, &pattern)
json.Unmarshal(rawMap, &resource)
err := json.Unmarshal(rawPattern, &pattern)
assert.NilError(t, err)
err = json.Unmarshal(rawMap, &resource)
assert.NilError(t, err)
path, err := validateResourceElement(resource, pattern, pattern, "/")
path, err := validateResourceElement(log.Log, resource, pattern, pattern, "/")
assert.Equal(t, path, "/0/object/0/key2/")
assert.Assert(t, err != nil)
}

170
pkg/engine/validation.go
View file

@ -5,7 +5,7 @@ import (
"reflect"
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/engine/context"
"github.com/nirmata/kyverno/pkg/engine/response"
@ -13,6 +13,7 @@ import (
"github.com/nirmata/kyverno/pkg/engine/validate"
"github.com/nirmata/kyverno/pkg/engine/variables"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/log"
)
//Validate applies validation rules from policy on the resource
@ -23,42 +24,59 @@ func Validate(policyContext PolicyContext) (resp response.EngineResponse) {
oldR := policyContext.OldResource
ctx := policyContext.Context
admissionInfo := policyContext.AdmissionInfo
logger := log.Log.WithName("Validate").WithValues("policy", policy.Name)
// policy information
glog.V(4).Infof("started applying validation rules of policy %q (%v)", policy.Name, startTime)
// Process new & old resource
if reflect.DeepEqual(oldR, unstructured.Unstructured{}) {
// Create Mode
// Operate on New Resource only
resp := validateResource(ctx, policy, newR, admissionInfo)
startResultResponse(resp, policy, newR)
defer endResultResponse(resp, startTime)
// set PatchedResource with origin resource if empty
// in order to create policy violation
if reflect.DeepEqual(resp.PatchedResource, unstructured.Unstructured{}) {
resp.PatchedResource = newR
}
return *resp
if reflect.DeepEqual(newR, unstructured.Unstructured{}) {
logger = logger.WithValues("kind", oldR.GetKind(), "namespace", oldR.GetNamespace(), "name", oldR.GetName())
} else {
logger = logger.WithValues("kind", newR.GetKind(), "namespace", newR.GetNamespace(), "name", newR.GetName())
}
// Update Mode
// Operate on New and Old Resource only
// New resource
oldResponse := validateResource(ctx, policy, oldR, admissionInfo)
newResponse := validateResource(ctx, policy, newR, admissionInfo)
// if the old and new response is same then return empty response
if !isSameResponse(oldResponse, newResponse) {
// there are changes send response
startResultResponse(newResponse, policy, newR)
defer endResultResponse(newResponse, startTime)
if reflect.DeepEqual(newResponse.PatchedResource, unstructured.Unstructured{}) {
newResponse.PatchedResource = newR
logger.V(4).Info("start processing", "startTime", startTime)
defer func() {
if reflect.DeepEqual(resp, response.EngineResponse{}) {
return
}
var resource unstructured.Unstructured
if reflect.DeepEqual(resp.PatchedResource, unstructured.Unstructured{}) {
// for delete requests patched resource will be oldR since newR is empty
if reflect.DeepEqual(newR, unstructured.Unstructured{}) {
resource = oldR
} else {
resource = newR
}
}
for i := range resp.PolicyResponse.Rules {
messageInterface, err := variables.SubstituteVars(logger, ctx, resp.PolicyResponse.Rules[i].Message)
if err != nil {
continue
}
resp.PolicyResponse.Rules[i].Message, _ = messageInterface.(string)
}
resp.PatchedResource = resource
startResultResponse(&resp, policy, resource)
endResultResponse(logger, &resp, startTime)
}()
// If request is delete, newR will be empty
if reflect.DeepEqual(newR, unstructured.Unstructured{}) {
return *isRequestDenied(logger, ctx, policy, oldR, admissionInfo)
} else {
if denyResp := isRequestDenied(logger, ctx, policy, newR, admissionInfo); !denyResp.IsSuccesful() {
return *denyResp
}
}
if reflect.DeepEqual(oldR, unstructured.Unstructured{}) {
return *validateResource(logger, ctx, policy, newR, admissionInfo)
}
oldResponse := validateResource(logger, ctx, policy, oldR, admissionInfo)
newResponse := validateResource(logger, ctx, policy, newR, admissionInfo)
if !isSameResponse(oldResponse, newResponse) {
return *newResponse
}
// if there are no changes with old and new response then sent empty response
// skip processing
return response.EngineResponse{}
}
@ -73,10 +91,9 @@ func startResultResponse(resp *response.EngineResponse, policy kyverno.ClusterPo
resp.PolicyResponse.ValidationFailureAction = policy.Spec.ValidationFailureAction
}
func endResultResponse(resp *response.EngineResponse, startTime time.Time) {
func endResultResponse(log logr.Logger, resp *response.EngineResponse, startTime time.Time) {
resp.PolicyResponse.ProcessingTime = time.Since(startTime)
glog.V(4).Infof("Finished applying validation rules policy %v (%v)", resp.PolicyResponse.Policy, resp.PolicyResponse.ProcessingTime)
glog.V(4).Infof("Validation Rules appplied successfully count %v for policy %q", resp.PolicyResponse.RulesAppliedCount, resp.PolicyResponse.Policy)
log.V(4).Info("finshed processing", "processingTime", resp.PolicyResponse.ProcessingTime, "validationRulesApplied", resp.PolicyResponse.RulesAppliedCount)
}
func incrementAppliedCount(resp *response.EngineResponse) {
@ -84,37 +101,74 @@ func incrementAppliedCount(resp *response.EngineResponse) {
resp.PolicyResponse.RulesAppliedCount++
}
func validateResource(ctx context.EvalInterface, policy kyverno.ClusterPolicy, resource unstructured.Unstructured, admissionInfo kyverno.RequestInfo) *response.EngineResponse {
func isRequestDenied(log logr.Logger, ctx context.EvalInterface, policy kyverno.ClusterPolicy, resource unstructured.Unstructured, admissionInfo kyverno.RequestInfo) *response.EngineResponse {
resp := &response.EngineResponse{}
for _, rule := range policy.Spec.Rules {
if !rule.HasValidate() {
continue
}
if err := MatchesResourceDescription(resource, rule, admissionInfo); err != nil {
log.V(4).Info("resource fails the match description", "reason", err.Error())
continue
}
preconditionsCopy := copyConditions(rule.Conditions)
if !variables.EvaluateConditions(log, ctx, preconditionsCopy) {
log.V(4).Info("resource fails the preconditions")
continue
}
if rule.Validation.Deny != nil {
denyConditionsCopy := copyConditions(rule.Validation.Deny.Conditions)
if len(rule.Validation.Deny.Conditions) == 0 || variables.EvaluateConditions(log, ctx, denyConditionsCopy) {
ruleResp := response.RuleResponse{
Name: rule.Name,
Type: utils.Validation.String(),
Message: rule.Validation.Message,
Success: false,
}
resp.PolicyResponse.Rules = append(resp.PolicyResponse.Rules, ruleResp)
}
continue
}
}
return resp
}
func validateResource(log logr.Logger, ctx context.EvalInterface, policy kyverno.ClusterPolicy, resource unstructured.Unstructured, admissionInfo kyverno.RequestInfo) *response.EngineResponse {
resp := &response.EngineResponse{}
for _, rule := range policy.Spec.Rules {
if !rule.HasValidate() {
continue
}
startTime := time.Now()
glog.V(4).Infof("Time: Validate matchAdmissionInfo %v", time.Since(startTime))
// check if the resource satisfies the filter conditions defined in the rule
// TODO: this needs to be extracted, to filter the resource so that we can avoid passing resources that
// dont statisfy a policy rule resource description
// dont satisfy a policy rule resource description
if err := MatchesResourceDescription(resource, rule, admissionInfo); err != nil {
glog.V(4).Infof("resource %s/%s does not satisfy the resource description for the rule:\n%s", resource.GetNamespace(), resource.GetName(), err.Error())
log.V(4).Info("resource fails the match description", "reason", err.Error())
continue
}
// operate on the copy of the conditions, as we perform variable substitution
copyConditions := copyConditions(rule.Conditions)
preconditionsCopy := copyConditions(rule.Conditions)
// evaluate pre-conditions
// - handle variable subsitutions
if !variables.EvaluateConditions(ctx, copyConditions) {
glog.V(4).Infof("resource %s/%s does not satisfy the conditions for the rule ", resource.GetNamespace(), resource.GetName())
if !variables.EvaluateConditions(log, ctx, preconditionsCopy) {
log.V(4).Info("resource fails the preconditions")
continue
}
if rule.Validation.Pattern != nil || rule.Validation.AnyPattern != nil {
ruleResponse := validatePatterns(ctx, resource, rule)
ruleResponse := validatePatterns(log, ctx, resource, rule)
incrementAppliedCount(resp)
resp.PolicyResponse.Rules = append(resp.PolicyResponse.Rules, ruleResponse)
}
}
return resp
}
@ -159,15 +213,17 @@ func isSameRules(oldRules []response.RuleResponse, newRules []response.RuleRespo
}
// validatePatterns validate pattern and anyPattern
func validatePatterns(ctx context.EvalInterface, resource unstructured.Unstructured, rule kyverno.Rule) (resp response.RuleResponse) {
func validatePatterns(log logr.Logger, ctx context.EvalInterface, resource unstructured.Unstructured, rule kyverno.Rule) (resp response.RuleResponse) {
startTime := time.Now()
glog.V(4).Infof("started applying validation rule %q (%v)", rule.Name, startTime)
logger := log.WithValues("rule", rule.Name)
logger.V(4).Info("start processing rule", "startTime", startTime)
resp.Name = rule.Name
resp.Type = utils.Validation.String()
defer func() {
resp.RuleStats.ProcessingTime = time.Since(startTime)
glog.V(4).Infof("finished applying validation rule %q (%v)", resp.Name, resp.RuleStats.ProcessingTime)
logger.V(4).Info("finished processing rule", "processingTime", resp.RuleStats.ProcessingTime)
}()
// work on a copy of validation rule
validationRule := rule.Validation.DeepCopy()
@ -176,7 +232,7 @@ func validatePatterns(ctx context.EvalInterface, resource unstructured.Unstructu
// substitute variables in the pattern
pattern := validationRule.Pattern
var err error
if pattern, err = variables.SubstituteVars(ctx, pattern); err != nil {
if pattern, err = variables.SubstituteVars(logger, ctx, pattern); err != nil {
// variable subsitution failed
resp.Success = false
resp.Message = fmt.Sprintf("Validation error: %s; Validation rule '%s' failed. '%s'",
@ -184,15 +240,15 @@ func validatePatterns(ctx context.EvalInterface, resource unstructured.Unstructu
return resp
}
if path, err := validate.ValidateResourceWithPattern(resource.Object, pattern); err != nil {
if path, err := validate.ValidateResourceWithPattern(logger, resource.Object, pattern); err != nil {
// validation failed
resp.Success = false
resp.Message = fmt.Sprintf("Validation error: %s; Validation rule '%s' failed at path '%s'",
resp.Message = fmt.Sprintf("Validation error: %s; Validation rule %s failed at path %s",
rule.Validation.Message, rule.Name, path)
return resp
}
// rule application successful
glog.V(4).Infof("rule %s pattern validated successfully on resource %s/%s/%s", rule.Name, resource.GetKind(), resource.GetNamespace(), resource.GetName())
logger.V(4).Info("successfully processed rule")
resp.Success = true
resp.Message = fmt.Sprintf("Validation rule '%s' succeeded.", rule.Name)
return resp
@ -203,19 +259,18 @@ func validatePatterns(ctx context.EvalInterface, resource unstructured.Unstructu
var failedAnyPatternsErrors []error
var err error
for idx, pattern := range validationRule.AnyPattern {
if pattern, err = variables.SubstituteVars(ctx, pattern); err != nil {
if pattern, err = variables.SubstituteVars(logger, ctx, pattern); err != nil {
// variable subsitution failed
failedSubstitutionsErrors = append(failedSubstitutionsErrors, err)
continue
}
_, err := validate.ValidateResourceWithPattern(resource.Object, pattern)
_, err := validate.ValidateResourceWithPattern(logger, resource.Object, pattern)
if err == nil {
resp.Success = true
resp.Message = fmt.Sprintf("Validation rule '%s' anyPattern[%d] succeeded.", rule.Name, idx)
return resp
}
glog.V(4).Infof("Validation error: %s; Validation rule %s anyPattern[%d] for %s/%s/%s",
rule.Validation.Message, rule.Name, idx, resource.GetKind(), resource.GetNamespace(), resource.GetName())
logger.V(4).Info(fmt.Sprintf("validation rule failed for anyPattern[%d]", idx), "message", rule.Validation.Message)
patternErr := fmt.Errorf("anyPattern[%d] failed; %s", idx, err)
failedAnyPatternsErrors = append(failedAnyPatternsErrors, patternErr)
}
@ -234,7 +289,12 @@ func validatePatterns(ctx context.EvalInterface, resource unstructured.Unstructu
errorStr = append(errorStr, err.Error())
}
resp.Success = false
resp.Message = fmt.Sprintf("Validation rule '%s' failed. %s", rule.Name, errorStr)
log.V(4).Info(fmt.Sprintf("Validation rule '%s' failed. %s", rule.Name, errorStr))
if rule.Validation.Message == "" {
resp.Message = fmt.Sprintf("Validation rule '%s' has failed", rule.Name)
} else {
resp.Message = rule.Validation.Message
}
return resp
}
}

199
pkg/engine/validation_test.go
View file

File diff suppressed because one or more lines are too long

10
pkg/engine/variables/common.go Normal file
View file

@ -0,0 +1,10 @@
package variables
import "regexp"
//IsVariable returns true if the element contains a 'valid' variable {{}}
func IsVariable(element string) bool {
validRegex := regexp.MustCompile(variableRegex)
groups := validRegex.FindAllStringSubmatch(element, -1)
return len(groups) != 0
}

11
pkg/engine/variables/evaluate.go
View file

@ -1,16 +1,16 @@
package variables
import (
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/engine/context"
"github.com/nirmata/kyverno/pkg/engine/variables/operator"
)
//Evaluate evaluates the condition
func Evaluate(ctx context.EvalInterface, condition kyverno.Condition) bool {
func Evaluate(log logr.Logger, ctx context.EvalInterface, condition kyverno.Condition) bool {
// get handler for the operator
handle := operator.CreateOperatorHandler(ctx, condition.Operator, SubstituteVars)
handle := operator.CreateOperatorHandler(log, ctx, condition.Operator, SubstituteVars)
if handle == nil {
return false
}
@ -18,11 +18,10 @@ func Evaluate(ctx context.EvalInterface, condition kyverno.Condition) bool {
}
//EvaluateConditions evaluates multiple conditions
func EvaluateConditions(ctx context.EvalInterface, conditions []kyverno.Condition) bool {
func EvaluateConditions(log logr.Logger, ctx context.EvalInterface, conditions []kyverno.Condition) bool {
// AND the conditions
for _, condition := range conditions {
if !Evaluate(ctx, condition) {
glog.V(4).Infof("condition %v failed", condition)
if !Evaluate(log, ctx, condition) {
return false
}
}

61
pkg/engine/variables/evaluate_test.go
View file

@ -6,6 +6,7 @@ import (
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/engine/context"
"sigs.k8s.io/controller-runtime/pkg/log"
)
// STRINGS
@ -18,7 +19,7 @@ func Test_Eval_Equal_Const_String_Pass(t *testing.T) {
Value: "name",
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -32,7 +33,7 @@ func Test_Eval_Equal_Const_String_Fail(t *testing.T) {
Value: "name1",
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -46,7 +47,7 @@ func Test_Eval_NoEqual_Const_String_Pass(t *testing.T) {
Value: "name1",
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -60,7 +61,7 @@ func Test_Eval_NoEqual_Const_String_Fail(t *testing.T) {
Value: "name",
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -76,7 +77,7 @@ func Test_Eval_Equal_Const_Bool_Pass(t *testing.T) {
Value: true,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -90,7 +91,7 @@ func Test_Eval_Equal_Const_Bool_Fail(t *testing.T) {
Value: false,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -104,7 +105,7 @@ func Test_Eval_NoEqual_Const_Bool_Pass(t *testing.T) {
Value: false,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -118,7 +119,7 @@ func Test_Eval_NoEqual_Const_Bool_Fail(t *testing.T) {
Value: true,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -133,7 +134,7 @@ func Test_Eval_Equal_Const_int_Pass(t *testing.T) {
Value: 1,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -147,7 +148,7 @@ func Test_Eval_Equal_Const_int_Fail(t *testing.T) {
Value: 2,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -161,7 +162,7 @@ func Test_Eval_NoEqual_Const_int_Pass(t *testing.T) {
Value: 2,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -175,7 +176,7 @@ func Test_Eval_NoEqual_Const_int_Fail(t *testing.T) {
Value: 1,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -190,7 +191,7 @@ func Test_Eval_Equal_Const_int64_Pass(t *testing.T) {
Value: int64(1),
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -204,7 +205,7 @@ func Test_Eval_Equal_Const_int64_Fail(t *testing.T) {
Value: int64(2),
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -218,7 +219,7 @@ func Test_Eval_NoEqual_Const_int64_Pass(t *testing.T) {
Value: int64(2),
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -232,7 +233,7 @@ func Test_Eval_NoEqual_Const_int64_Fail(t *testing.T) {
Value: int64(1),
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -248,7 +249,7 @@ func Test_Eval_Equal_Const_float64_Pass(t *testing.T) {
Value: 1.5,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -262,7 +263,7 @@ func Test_Eval_Equal_Const_float64_Fail(t *testing.T) {
Value: 1.6,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -276,7 +277,7 @@ func Test_Eval_NoEqual_Const_float64_Pass(t *testing.T) {
Value: 1.6,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -290,7 +291,7 @@ func Test_Eval_NoEqual_Const_float64_Fail(t *testing.T) {
Value: 1.5,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -320,7 +321,7 @@ func Test_Eval_Equal_Const_object_Pass(t *testing.T) {
Value: obj2,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -348,7 +349,7 @@ func Test_Eval_Equal_Const_object_Fail(t *testing.T) {
Value: obj2,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -376,7 +377,7 @@ func Test_Eval_NotEqual_Const_object_Pass(t *testing.T) {
Value: obj2,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -404,7 +405,7 @@ func Test_Eval_NotEqual_Const_object_Fail(t *testing.T) {
Value: obj2,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -434,7 +435,7 @@ func Test_Eval_Equal_Const_list_Pass(t *testing.T) {
Value: obj2,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -460,7 +461,7 @@ func Test_Eval_Equal_Const_list_Fail(t *testing.T) {
Value: obj2,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -486,7 +487,7 @@ func Test_Eval_NotEqual_Const_list_Pass(t *testing.T) {
Value: obj2,
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -512,7 +513,7 @@ func Test_Eval_NotEqual_Const_list_Fail(t *testing.T) {
Value: obj2,
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}
@ -545,7 +546,7 @@ func Test_Eval_Equal_Var_Pass(t *testing.T) {
Value: "temp",
}
if !Evaluate(ctx, condition) {
if !Evaluate(log.Log, ctx, condition) {
t.Error("expected to pass")
}
}
@ -576,7 +577,7 @@ func Test_Eval_Equal_Var_Fail(t *testing.T) {
Value: "temp1",
}
if Evaluate(ctx, condition) {
if Evaluate(log.Log, ctx, condition) {
t.Error("expected to fail")
}
}

40
pkg/engine/variables/operator/equal.go
View file

@ -1,19 +1,21 @@
package operator
import (
"fmt"
"math"
"reflect"
"strconv"
"github.com/golang/glog"
"github.com/go-logr/logr"
"github.com/nirmata/kyverno/pkg/engine/context"
)
//NewEqualHandler returns handler to manage Equal operations
func NewEqualHandler(ctx context.EvalInterface, subHandler VariableSubstitutionHandler) OperatorHandler {
func NewEqualHandler(log logr.Logger, ctx context.EvalInterface, subHandler VariableSubstitutionHandler) OperatorHandler {
return EqualHandler{
ctx: ctx,
subHandler: subHandler,
log: log,
}
}
@ -21,6 +23,7 @@ func NewEqualHandler(ctx context.EvalInterface, subHandler VariableSubstitutionH
type EqualHandler struct {
ctx context.EvalInterface
subHandler VariableSubstitutionHandler
log logr.Logger
}
//Evaluate evaluates expression with Equal Operator
@ -28,14 +31,14 @@ func (eh EqualHandler) Evaluate(key, value interface{}) bool {
var err error
//TODO: decouple variables from evaluation
// substitute the variables
if key, err = eh.subHandler(eh.ctx, key); err != nil {
if key, err = eh.subHandler(eh.log, eh.ctx, key); err != nil {
// Failed to resolve the variable
glog.Infof("Failed to resolve variables in key: %s: %v", key, err)
eh.log.Error(err, "Failed to resolve variable", "variable", key)
return false
}
if value, err = eh.subHandler(eh.ctx, value); err != nil {
if value, err = eh.subHandler(eh.log, eh.ctx, value); err != nil {
// Failed to resolve the variable
glog.Infof("Failed to resolve variables in value: %s: %v", value, err)
eh.log.Error(err, "Failed to resolve variable", "variable", value)
return false
}
@ -56,7 +59,7 @@ func (eh EqualHandler) Evaluate(key, value interface{}) bool {
case []interface{}:
return eh.validateValueWithSlicePattern(typedKey, value)
default:
glog.Errorf("Unsupported type %v", typedKey)
eh.log.Info("Unsupported type", "value", typedKey, "type", fmt.Sprintf("%T", typedKey))
return false
}
}
@ -65,7 +68,7 @@ func (eh EqualHandler) validateValueWithSlicePattern(key []interface{}, value in
if val, ok := value.([]interface{}); ok {
return reflect.DeepEqual(key, val)
}
glog.Warningf("Expected []interface{}, %v is of type %T", value, value)
eh.log.Info("Expected type []interface{}", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
@ -73,7 +76,7 @@ func (eh EqualHandler) validateValueWithMapPattern(key map[string]interface{}, v
if val, ok := value.(map[string]interface{}); ok {
return reflect.DeepEqual(key, val)
}
glog.Warningf("Expected map[string]interface{}, %v is of type %T", value, value)
eh.log.Info("Expected type map[string]interface{}", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
@ -81,7 +84,8 @@ func (eh EqualHandler) validateValuewithStringPattern(key string, value interfac
if val, ok := value.(string); ok {
return key == val
}
glog.Warningf("Expected string, %v is of type %T", value, value)
eh.log.Info("Expected type string", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
@ -92,25 +96,25 @@ func (eh EqualHandler) validateValuewithFloatPattern(key float64, value interfac
if key == math.Trunc(key) {
return int(key) == typedValue
}
glog.Warningf("Expected float, found int: %d\n", typedValue)
eh.log.Info("Expected type float, found int", "typedValue", typedValue)
case int64:
// check that float has not fraction
if key == math.Trunc(key) {
return int64(key) == typedValue
}
glog.Warningf("Expected float, found int: %d\n", typedValue)
eh.log.Info("Expected type float, found int", "typedValue", typedValue)
case float64:
return typedValue == key
case string:
// extract float from string
float64Num, err := strconv.ParseFloat(typedValue, 64)
if err != nil {
glog.Warningf("Failed to parse float64 from string: %v", err)
eh.log.Error(err, "Failed to parse float64 from string")
return false
}
return float64Num == key
default:
glog.Warningf("Expected float, found: %T\n", value)
eh.log.Info("Expected type float", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
return false
@ -119,7 +123,7 @@ func (eh EqualHandler) validateValuewithFloatPattern(key float64, value interfac
func (eh EqualHandler) validateValuewithBoolPattern(key bool, value interface{}) bool {
typedValue, ok := value.(bool)
if !ok {
glog.Error("Expected bool, found %V", value)
eh.log.Info("Expected type bool", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
return key == typedValue
@ -136,18 +140,18 @@ func (eh EqualHandler) validateValuewithIntPattern(key int64, value interface{})
if typedValue == math.Trunc(typedValue) {
return int64(typedValue) == key
}
glog.Warningf("Expected int, found float: %f", typedValue)
eh.log.Info("Expected type int, found float", "value", typedValue, "type", fmt.Sprintf("%T", typedValue))
return false
case string:
// extract in64 from string
int64Num, err := strconv.ParseInt(typedValue, 10, 64)
if err != nil {
glog.Warningf("Failed to parse int64 from string: %v", err)
eh.log.Error(err, "Failed to parse int64 from string")
return false
}
return int64Num == key
default:
glog.Warningf("Expected int, %v is of type %T", value, value)
eh.log.Info("Expected type int", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
}

39
pkg/engine/variables/operator/notequal.go
View file

@ -1,19 +1,21 @@
package operator
import (
"fmt"
"math"
"reflect"
"strconv"
"github.com/golang/glog"
"github.com/go-logr/logr"
"github.com/nirmata/kyverno/pkg/engine/context"
)
//NewNotEqualHandler returns handler to manage NotEqual operations
func NewNotEqualHandler(ctx context.EvalInterface, subHandler VariableSubstitutionHandler) OperatorHandler {
func NewNotEqualHandler(log logr.Logger, ctx context.EvalInterface, subHandler VariableSubstitutionHandler) OperatorHandler {
return NotEqualHandler{
ctx: ctx,
subHandler: subHandler,
log: log,
}
}
@ -21,6 +23,7 @@ func NewNotEqualHandler(ctx context.EvalInterface, subHandler VariableSubstituti
type NotEqualHandler struct {
ctx context.EvalInterface
subHandler VariableSubstitutionHandler
log logr.Logger
}
//Evaluate evaluates expression with NotEqual Operator
@ -28,14 +31,14 @@ func (neh NotEqualHandler) Evaluate(key, value interface{}) bool {
var err error
//TODO: decouple variables from evaluation
// substitute the variables
if key, err = neh.subHandler(neh.ctx, key); err != nil {
if key, err = neh.subHandler(neh.log, neh.ctx, key); err != nil {
// Failed to resolve the variable
glog.Infof("Failed to resolve variables in key: %s: %v", key, err)
neh.log.Error(err, "Failed to resolve variable", "variable", key)
return false
}
if value, err = neh.subHandler(neh.ctx, value); err != nil {
if value, err = neh.subHandler(neh.log, neh.ctx, value); err != nil {
// Failed to resolve the variable
glog.Infof("Failed to resolve variables in value: %s: %v", value, err)
neh.log.Error(err, "Failed to resolve variable", "variable", value)
return false
}
// key and value need to be of same type
@ -55,7 +58,7 @@ func (neh NotEqualHandler) Evaluate(key, value interface{}) bool {
case []interface{}:
return neh.validateValueWithSlicePattern(typedKey, value)
default:
glog.Error("Unsupported type %V", typedKey)
neh.log.Info("Unsupported type", "value", typedKey, "type", fmt.Sprintf("%T", typedKey))
return false
}
}
@ -64,7 +67,7 @@ func (neh NotEqualHandler) validateValueWithSlicePattern(key []interface{}, valu
if val, ok := value.([]interface{}); ok {
return !reflect.DeepEqual(key, val)
}
glog.Warningf("Expected []interface{}, %v is of type %T", value, value)
neh.log.Info("Expected type []interface{}", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
@ -72,7 +75,7 @@ func (neh NotEqualHandler) validateValueWithMapPattern(key map[string]interface{
if val, ok := value.(map[string]interface{}); ok {
return !reflect.DeepEqual(key, val)
}
glog.Warningf("Expected map[string]interface{}, %v is of type %T", value, value)
neh.log.Info("Expected type map[string]interface{}", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
@ -80,7 +83,7 @@ func (neh NotEqualHandler) validateValuewithStringPattern(key string, value inte
if val, ok := value.(string); ok {
return key != val
}
glog.Warningf("Expected string, %v is of type %T", value, value)
neh.log.Info("Expected type string", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
@ -91,25 +94,25 @@ func (neh NotEqualHandler) validateValuewithFloatPattern(key float64, value inte
if key == math.Trunc(key) {
return int(key) != typedValue
}
glog.Warningf("Expected float, found int: %d\n", typedValue)
neh.log.Info("Expected type float, found int", "typedValue", typedValue)
case int64:
// check that float has not fraction
if key == math.Trunc(key) {
return int64(key) != typedValue
}
glog.Warningf("Expected float, found int: %d\n", typedValue)
neh.log.Info("Expected type float, found int", "typedValue", typedValue)
case float64:
return typedValue != key
case string:
// extract float from string
float64Num, err := strconv.ParseFloat(typedValue, 64)
if err != nil {
glog.Warningf("Failed to parse float64 from string: %v", err)
neh.log.Error(err, "Failed to parse float64 from string")
return false
}
return float64Num != key
default:
glog.Warningf("Expected float, found: %T\n", value)
neh.log.Info("Expected type float", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
return false
@ -118,7 +121,7 @@ func (neh NotEqualHandler) validateValuewithFloatPattern(key float64, value inte
func (neh NotEqualHandler) validateValuewithBoolPattern(key bool, value interface{}) bool {
typedValue, ok := value.(bool)
if !ok {
glog.Error("Expected bool, found %V", value)
neh.log.Info("Expected type bool", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
return key != typedValue
@ -135,18 +138,18 @@ func (neh NotEqualHandler) validateValuewithIntPattern(key int64, value interfac
if typedValue == math.Trunc(typedValue) {
return int64(typedValue) != key
}
glog.Warningf("Expected int, found float: %f\n", typedValue)
neh.log.Info("Expected type int, found float", "value", typedValue, "type", fmt.Sprintf("%T", typedValue))
return false
case string:
// extract in64 from string
int64Num, err := strconv.ParseInt(typedValue, 10, 64)
if err != nil {
glog.Warningf("Failed to parse int64 from string: %v", err)
neh.log.Error(err, "Failed to parse int64 from string")
return false
}
return int64Num != key
default:
glog.Warningf("Expected int, %v is of type %T", value, value)
neh.log.Info("Expected type int", "value", value, "type", fmt.Sprintf("%T", value))
return false
}
}

16
pkg/engine/variables/operator/operator.go
View file

@ -1,7 +1,7 @@
package operator
import (
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/engine/context"
)
@ -17,17 +17,21 @@ type OperatorHandler interface {
}
//VariableSubstitutionHandler defines the handler function for variable substitution
type VariableSubstitutionHandler = func(ctx context.EvalInterface, pattern interface{}) (interface{}, error)
type VariableSubstitutionHandler = func(log logr.Logger, ctx context.EvalInterface, pattern interface{}) (interface{}, error)
//CreateOperatorHandler returns the operator handler based on the operator used in condition
func CreateOperatorHandler(ctx context.EvalInterface, op kyverno.ConditionOperator, subHandler VariableSubstitutionHandler) OperatorHandler {
func CreateOperatorHandler(log logr.Logger, ctx context.EvalInterface, op kyverno.ConditionOperator, subHandler VariableSubstitutionHandler) OperatorHandler {
switch op {
case kyverno.Equal:
return NewEqualHandler(ctx, subHandler)
return NewEqualHandler(log, ctx, subHandler)
case kyverno.NotEqual:
return NewNotEqualHandler(ctx, subHandler)
return NewNotEqualHandler(log, ctx, subHandler)
case kyverno.Equals:
return NewEqualHandler(log, ctx, subHandler)
case kyverno.NotEquals:
return NewNotEqualHandler(log, ctx, subHandler)
default:
glog.Errorf("unsupported operator: %s", string(op))
log.Info("operator not supported", "operator", string(op))
}
return nil
}

19
pkg/engine/variables/variables_test.go
View file

@ -7,6 +7,7 @@ import (
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
authenticationv1 "k8s.io/api/authentication/v1"
"sigs.k8s.io/controller-runtime/pkg/log"
"github.com/nirmata/kyverno/pkg/engine/context"
)
@ -84,7 +85,7 @@ func Test_variablesub1(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err != nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err != nil {
t.Error(err)
}
resultRaw, err := json.Marshal(patternCopy)
@ -174,7 +175,7 @@ func Test_variablesub_multiple(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err != nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err != nil {
t.Error(err)
}
resultRaw, err := json.Marshal(patternCopy)
@ -261,7 +262,7 @@ func Test_variablesubstitution(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err != nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err != nil {
t.Error(err)
}
resultRaw, err := json.Marshal(patternCopy)
@ -322,7 +323,7 @@ func Test_variableSubstitutionValue(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err != nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err != nil {
t.Error(err)
}
resultRaw, err := json.Marshal(patternCopy)
@ -380,7 +381,7 @@ func Test_variableSubstitutionValueOperatorNotEqual(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err != nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err != nil {
t.Error(err)
}
resultRaw, err := json.Marshal(patternCopy)
@ -439,7 +440,7 @@ func Test_variableSubstitutionValueFail(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err == nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err == nil {
t.Log("expected to fails")
t.Fail()
}
@ -497,7 +498,7 @@ func Test_variableSubstitutionObject(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err != nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err != nil {
t.Error(err)
}
resultRaw, err := json.Marshal(patternCopy)
@ -561,7 +562,7 @@ func Test_variableSubstitutionObjectOperatorNotEqualFail(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err == nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err == nil {
t.Error(err)
}
@ -620,7 +621,7 @@ func Test_variableSubstitutionMultipleObject(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, patternCopy); err != nil {
if patternCopy, err = SubstituteVars(log.Log, ctx, patternCopy); err != nil {
t.Error(err)
}
resultRaw, err := json.Marshal(patternCopy)

187
pkg/engine/variables/vars.go
View file

@ -6,162 +6,111 @@ import (
"strconv"
"strings"
"github.com/golang/glog"
"github.com/go-logr/logr"
"github.com/nirmata/kyverno/pkg/engine/context"
)
const (
variableRegex = `\{\{([^{}]*)\}\}`
singleVarRegex = `^\{\{([^{}]*)\}\}$`
variableRegex = `\{\{([^{}]*)\}\}`
)
//SubstituteVars replaces the variables with the values defined in the context
// - if any variable is invaid or has nil value, it is considered as a failed varable substitution
func SubstituteVars(ctx context.EvalInterface, pattern interface{}) (interface{}, error) {
errs := []error{}
pattern = subVars(ctx, pattern, "", &errs)
if len(errs) == 0 {
// no error while parsing the pattern
return pattern, nil
func SubstituteVars(log logr.Logger, ctx context.EvalInterface, pattern interface{}) (interface{}, error) {
pattern, err := subVars(log, ctx, pattern, "")
if err != nil {
return pattern, err
}
return pattern, fmt.Errorf("%v", errs)
return pattern, nil
}
func subVars(ctx context.EvalInterface, pattern interface{}, path string, errs *[]error) interface{} {
func subVars(log logr.Logger, ctx context.EvalInterface, pattern interface{}, path string) (interface{}, error) {
switch typedPattern := pattern.(type) {
case map[string]interface{}:
return subMap(ctx, typedPattern, path, errs)
mapCopy := make(map[string]interface{})
for k, v := range typedPattern {
mapCopy[k] = v
}
return subMap(log, ctx, mapCopy, path)
case []interface{}:
return subArray(ctx, typedPattern, path, errs)
sliceCopy := make([]interface{}, len(typedPattern))
copy(sliceCopy, typedPattern)
return subArray(log, ctx, sliceCopy, path)
case string:
return subValR(ctx, typedPattern, path, errs)
return subValR(ctx, typedPattern, path)
default:
return pattern
return pattern, nil
}
}
func subMap(ctx context.EvalInterface, patternMap map[string]interface{}, path string, errs *[]error) map[string]interface{} {
func subMap(log logr.Logger, ctx context.EvalInterface, patternMap map[string]interface{}, path string) (map[string]interface{}, error) {
for key, patternElement := range patternMap {
curPath := path + "/" + key
value := subVars(ctx, patternElement, curPath, errs)
value, err := subVars(log, ctx, patternElement, curPath)
if err != nil {
return nil, err
}
patternMap[key] = value
}
return patternMap
return patternMap, nil
}
func subArray(ctx context.EvalInterface, patternList []interface{}, path string, errs *[]error) []interface{} {
func subArray(log logr.Logger, ctx context.EvalInterface, patternList []interface{}, path string) ([]interface{}, error) {
for idx, patternElement := range patternList {
curPath := path + "/" + strconv.Itoa(idx)
value := subVars(ctx, patternElement, curPath, errs)
value, err := subVars(log, ctx, patternElement, curPath)
if err != nil {
return nil, err
}
patternList[idx] = value
}
return patternList
return patternList, nil
}
type NotFoundVariableErr struct {
variable string
path string
}
func (n NotFoundVariableErr) Error() string {
return fmt.Sprintf("could not find variable %v at path %v", n.variable, n.path)
}
// subValR resolves the variables if defined
func subValR(ctx context.EvalInterface, valuePattern string, path string, errs *[]error) interface{} {
func subValR(ctx context.EvalInterface, valuePattern string, path string) (interface{}, error) {
originalPattern := valuePattern
// variable values can be scalar values(string,int, float) or they can be obects(map,slice)
// - {{variable}}
// there is a single variable resolution so the value can be scalar or object
// - {{variable1--{{variable2}}}}}
// variable2 is evaluted first as an individual variable and can be have scalar or object values
// but resolving the outer variable, {{variable--<value>}}
// if <value> is scalar then it can replaced, but for object types its tricky
// as object cannot be directy replaced, if the object is stringyfied then it loses it structure.
// since this might be a potential place for error, required better error reporting and handling
// object values are only suported for single variable substitution
if ok, retVal := processIfSingleVariable(ctx, valuePattern, path, errs); ok {
return retVal
}
// var emptyInterface interface{}
var failedVars []string
// process type string
regex := regexp.MustCompile(`\{\{([^{}]*)\}\}`)
for {
valueStr := valuePattern
if len(failedVars) != 0 {
glog.Info("some failed variables short-circuiting")
if vars := regex.FindAllString(valuePattern, -1); len(vars) > 0 {
for _, variable := range vars {
underlyingVariable := strings.ReplaceAll(strings.ReplaceAll(variable, "}}", ""), "{{", "")
substitutedVar, err := ctx.Query(underlyingVariable)
if err != nil {
return nil, fmt.Errorf("failed to resolve %v at path %s", underlyingVariable, path)
}
if val, ok := substitutedVar.(string); ok {
valuePattern = strings.Replace(valuePattern, variable, val, -1)
} else {
if substitutedVar != nil {
if originalPattern == variable {
return substitutedVar, nil
}
return nil, fmt.Errorf("failed to resolve %v at path %s", underlyingVariable, path)
}
return nil, NotFoundVariableErr{
variable: underlyingVariable,
path: path,
}
}
}
} else {
break
}
// get variables at this level
validRegex := regexp.MustCompile(variableRegex)
groups := validRegex.FindAllStringSubmatch(valueStr, -1)
if len(groups) == 0 {
// there was no match
// not variable defined
break
}
subs := map[string]interface{}{}
for _, group := range groups {
if _, ok := subs[group[0]]; ok {
// value has already been substituted
continue
}
// here we do the querying of the variables from the context
variable, err := ctx.Query(group[1])
if err != nil {
// error while evaluating
failedVars = append(failedVars, group[1])
continue
}
// path not found in context and value stored in null/nill
if variable == nil {
failedVars = append(failedVars, group[1])
continue
}
// get values for each and replace
subs[group[0]] = variable
}
// perform substitutions
newVal := valueStr
for k, v := range subs {
// if value is of type string then cast else consider it as direct replacement
if val, ok := v.(string); ok {
newVal = strings.Replace(newVal, k, val, -1)
continue
}
// if type is not scalar then consider this as a failed variable
glog.Infof("variable %s resolves to non-scalar value %v. Non-Scalar values are not supported for nested variables", k, v)
failedVars = append(failedVars, k)
}
valuePattern = newVal
}
// update errors if any
if len(failedVars) > 0 {
*errs = append(*errs, fmt.Errorf("failed to resolve %v at path %s", failedVars, path))
}
return valuePattern
}
// processIfSingleVariable will process the evaluation of single variables
// {{variable-{{variable}}}} -> compound/nested variables
// {{variable}}{{variable}} -> multiple variables
// {{variable}} -> single variable
// if the value can be evaluted return the value
// -> return value can be scalar or object type
// -> if the variable is not present in the context then add an error and dont process further
func processIfSingleVariable(ctx context.EvalInterface, valuePattern interface{}, path string, errs *[]error) (bool, interface{}) {
valueStr, ok := valuePattern.(string)
if !ok {
glog.Infof("failed to convert %v to string", valuePattern)
return false, nil
}
// get variables at this level
validRegex := regexp.MustCompile(singleVarRegex)
groups := validRegex.FindAllStringSubmatch(valueStr, -1)
if len(groups) == 0 {
return false, nil
}
// as there will be exactly one variable based on the above regex
group := groups[0]
variable, err := ctx.Query(group[1])
if err != nil || variable == nil {
*errs = append(*errs, fmt.Errorf("failed to resolve %v at path %s", group[1], path))
// return the same value pattern, and add un-resolvable variable error
return true, valuePattern
}
return true, variable
return valuePattern, nil
}

8
pkg/engine/variables/vars_test.go
View file

@ -6,6 +6,7 @@ import (
"github.com/nirmata/kyverno/pkg/engine/context"
"gotest.tools/assert"
"sigs.k8s.io/controller-runtime/pkg/log"
)
func Test_subVars_success(t *testing.T) {
@ -64,7 +65,7 @@ func Test_subVars_success(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, pattern); err != nil {
if _, err := SubstituteVars(log.Log, ctx, pattern); err != nil {
t.Error(err)
}
}
@ -125,7 +126,7 @@ func Test_subVars_failed(t *testing.T) {
t.Error(err)
}
if _, err := SubstituteVars(ctx, pattern); err == nil {
if _, err := SubstituteVars(log.Log, ctx, pattern); err == nil {
t.Error("error is expected")
}
}
@ -151,6 +152,5 @@ func Test_SubvarRecursive(t *testing.T) {
ctx := context.NewContext()
assert.Assert(t, ctx.AddResource(resourceRaw))
errs := []error{}
subValR(ctx, string(patternRaw), "/", &errs)
subValR(ctx, string(patternRaw), "/")
}

58
pkg/event/controller.go
View file

@ -1,13 +1,12 @@
package event
import (
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
"github.com/nirmata/kyverno/pkg/client/clientset/versioned/scheme"
kyvernoinformer "github.com/nirmata/kyverno/pkg/client/informers/externalversions/kyverno/v1"
kyvernolister "github.com/nirmata/kyverno/pkg/client/listers/kyverno/v1"
"github.com/nirmata/kyverno/pkg/constant"
client "github.com/nirmata/kyverno/pkg/dclient"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
@ -17,6 +16,7 @@ import (
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/tools/record"
"k8s.io/client-go/util/workqueue"
"k8s.io/klog"
)
//Generator generate events
@ -34,6 +34,7 @@ type Generator struct {
admissionCtrRecorder record.EventRecorder
// events generated at namespaced policy controller to process 'generate' rule
genPolicyRecorder record.EventRecorder
log logr.Logger
}
//Interface to generate event
@ -42,32 +43,33 @@ type Interface interface {
}
//NewEventGenerator to generate a new event controller
func NewEventGenerator(client *client.Client, pInformer kyvernoinformer.ClusterPolicyInformer) *Generator {
func NewEventGenerator(client *client.Client, pInformer kyvernoinformer.ClusterPolicyInformer, log logr.Logger) *Generator {
gen := Generator{
client: client,
pLister: pInformer.Lister(),
queue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), eventWorkQueueName),
pSynced: pInformer.Informer().HasSynced,
policyCtrRecorder: initRecorder(client, PolicyController),
admissionCtrRecorder: initRecorder(client, AdmissionController),
genPolicyRecorder: initRecorder(client, GeneratePolicyController),
policyCtrRecorder: initRecorder(client, PolicyController, log),
admissionCtrRecorder: initRecorder(client, AdmissionController, log),
genPolicyRecorder: initRecorder(client, GeneratePolicyController, log),
log: log,
}
return &gen
}
func initRecorder(client *client.Client, eventSource Source) record.EventRecorder {
func initRecorder(client *client.Client, eventSource Source, log logr.Logger) record.EventRecorder {
// Initliaze Event Broadcaster
err := scheme.AddToScheme(scheme.Scheme)
if err != nil {
glog.Error(err)
log.Error(err, "failed to add to scheme")
return nil
}
eventBroadcaster := record.NewBroadcaster()
eventBroadcaster.StartLogging(glog.V(4).Infof)
eventBroadcaster.StartLogging(klog.V(5).Infof)
eventInterface, err := client.GetEventsInterface()
if err != nil {
glog.Error(err) // TODO: add more specific error
log.Error(err, "failed to get event interface for logging")
return nil
}
eventBroadcaster.StartRecordingToSink(
@ -81,11 +83,12 @@ func initRecorder(client *client.Client, eventSource Source) record.EventRecorde
//Add queues an event for generation
func (gen *Generator) Add(infos ...Info) {
logger := gen.log
for _, info := range infos {
if info.Name == "" {
// dont create event for resources with generateName
// as the name is not generated yet
glog.V(4).Infof("received info %v, not creating an event as the resource has not been assigned a name yet", info)
logger.V(4).Info("not creating an event as the resource has not been assigned a name yet", "kind", info.Kind, "name", info.Name, "namespace", info.Namespace)
continue
}
gen.queue.Add(info)
@ -94,16 +97,18 @@ func (gen *Generator) Add(infos ...Info) {
// Run begins generator
func (gen *Generator) Run(workers int, stopCh <-chan struct{}) {
logger := gen.log
defer utilruntime.HandleCrash()
glog.Info("Starting event generator")
defer glog.Info("Shutting down event generator")
logger.Info("start")
defer logger.Info("shutting down")
if !cache.WaitForCacheSync(stopCh, gen.pSynced) {
glog.Error("event generator: failed to sync informer cache")
logger.Info("failed to sync informer cache")
}
for i := 0; i < workers; i++ {
go wait.Until(gen.runWorker, time.Second, stopCh)
go wait.Until(gen.runWorker, constant.EventControllerResync, stopCh)
}
<-stopCh
}
@ -114,24 +119,25 @@ func (gen *Generator) runWorker() {
}
func (gen *Generator) handleErr(err error, key interface{}) {
logger := gen.log
if err == nil {
gen.queue.Forget(key)
return
}
// This controller retries if something goes wrong. After that, it stops trying.
if gen.queue.NumRequeues(key) < workQueueRetryLimit {
glog.Warningf("Error syncing events %v(re-queuing request, the resource might not have been created yet): %v", key, err)
logger.Error(err, "Error syncing events;re-queuing request,the resource might not have been created yet", "key", key)
// Re-enqueue the key rate limited. Based on the rate limiter on the
// queue and the re-enqueue history, the key will be processed later again.
gen.queue.AddRateLimited(key)
return
}
gen.queue.Forget(key)
glog.Error(err)
glog.Warningf("Dropping the key out of the queue: %v", err)
logger.Error(err, "dropping the key out of queue", "key", key)
}
func (gen *Generator) processNextWorkItem() bool {
logger := gen.log
obj, shutdown := gen.queue.Get()
if shutdown {
return false
@ -144,7 +150,7 @@ func (gen *Generator) processNextWorkItem() bool {
if key, ok = obj.(Info); !ok {
gen.queue.Forget(obj)
glog.Warningf("Expecting type info by got %v\n", obj)
logger.Info("Incorrect type; expected type 'info'", "obj", obj)
return nil
}
err := gen.syncHandler(key)
@ -152,13 +158,14 @@ func (gen *Generator) processNextWorkItem() bool {
return nil
}(obj)
if err != nil {
glog.Error(err)
logger.Error(err, "failed to process next work item")
return true
}
return true
}
func (gen *Generator) syncHandler(key Info) error {
logger := gen.log
var robj runtime.Object
var err error
switch key.Kind {
@ -166,13 +173,13 @@ func (gen *Generator) syncHandler(key Info) error {
//TODO: policy is clustered resource so wont need namespace
robj, err = gen.pLister.Get(key.Name)
if err != nil {
glog.V(4).Infof("Error creating event: unable to get policy %s, will retry ", key.Name)
logger.Error(err, "failed to get policy", "name", key.Name)
return err
}
default:
robj, err = gen.client.GetResource(key.Kind, key.Namespace, key.Name)
if err != nil {
glog.V(4).Infof("Error creating event: unable to get resource %s/%s/%s, will retry ", key.Kind, key.Namespace, key.Name)
logger.Error(err, "failed to get resource", "kind", key.Kind, "name", key.Name, "namespace", key.Namespace)
return err
}
}
@ -192,13 +199,14 @@ func (gen *Generator) syncHandler(key Info) error {
case GeneratePolicyController:
gen.genPolicyRecorder.Event(robj, eventType, key.Reason, key.Message)
default:
glog.Info("info.source not defined for the event generator request")
logger.Info("info.source not defined for the request")
}
return nil
}
//NewEvent builds a event creation request
func NewEvent(
log logr.Logger,
rkind,
rapiVersion,
rnamespace,
@ -209,7 +217,7 @@ func NewEvent(
args ...interface{}) Info {
msgText, err := getEventMsg(message, args...)
if err != nil {
glog.Error(err)
log.Error(err, "failed to get event message")
}
return Info{
Kind: rkind,

15
pkg/generate/cleanup/cleanup.go
View file

@ -1,19 +1,20 @@
package cleanup
import (
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
dclient "github.com/nirmata/kyverno/pkg/dclient"
apierrors "k8s.io/apimachinery/pkg/api/errors"
)
func (c *Controller) processGR(gr kyverno.GenerateRequest) error {
logger := c.log.WithValues("kind", gr.Kind, "namespace", gr.Namespace, "name", gr.Name)
// 1- Corresponding policy has been deleted
// then we dont delete the generated resources
// 2- The trigger resource is deleted, then delete the generated resources
if !ownerResourceExists(c.client, gr) {
if err := deleteGeneratedResources(c.client, gr); err != nil {
if !ownerResourceExists(logger, c.client, gr) {
if err := deleteGeneratedResources(logger, c.client, gr); err != nil {
return err
}
// - trigger-resource is deleted
@ -24,25 +25,25 @@ func (c *Controller) processGR(gr kyverno.GenerateRequest) error {
return nil
}
func ownerResourceExists(client *dclient.Client, gr kyverno.GenerateRequest) bool {
func ownerResourceExists(log logr.Logger, client *dclient.Client, gr kyverno.GenerateRequest) bool {
_, err := client.GetResource(gr.Spec.Resource.Kind, gr.Spec.Resource.Namespace, gr.Spec.Resource.Name)
// trigger resources has been deleted
if apierrors.IsNotFound(err) {
return false
}
if err != nil {
glog.V(4).Infof("Failed to get resource %s/%s/%s: error : %s", gr.Spec.Resource.Kind, gr.Spec.Resource.Namespace, gr.Spec.Resource.Name, err)
log.Error(err, "failed to get resource", "genKind", gr.Spec.Resource.Kind, "genNamespace", gr.Spec.Resource.Namespace, "genName", gr.Spec.Resource.Name)
}
// if there was an error while querying the resources we dont delete the generated resources
// but expect the deletion in next reconciliation loop
return true
}
func deleteGeneratedResources(client *dclient.Client, gr kyverno.GenerateRequest) error {
func deleteGeneratedResources(log logr.Logger, client *dclient.Client, gr kyverno.GenerateRequest) error {
for _, genResource := range gr.Status.GeneratedResources {
err := client.DeleteResource(genResource.Kind, genResource.Namespace, genResource.Name, false)
if apierrors.IsNotFound(err) {
glog.V(4).Infof("resource %s/%s/%s not found, will no delete", genResource.Kind, genResource.Namespace, genResource.Name)
log.Error(err, "resource not foundl will not delete", "genKind", gr.Spec.Resource.Kind, "genNamespace", gr.Spec.Resource.Namespace, "genName", gr.Spec.Resource.Name)
continue
}
if err != nil {

54
pkg/generate/cleanup/controller.go
View file

@ -1,15 +1,14 @@
package cleanup
import (
"fmt"
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
kyvernoclient "github.com/nirmata/kyverno/pkg/client/clientset/versioned"
kyvernoinformer "github.com/nirmata/kyverno/pkg/client/informers/externalversions/kyverno/v1"
kyvernolister "github.com/nirmata/kyverno/pkg/client/listers/kyverno/v1"
"github.com/nirmata/kyverno/pkg/constant"
dclient "github.com/nirmata/kyverno/pkg/dclient"
"k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@ -53,6 +52,7 @@ type Controller struct {
//TODO: list of generic informers
// only support Namespaces for deletion of resource
nsInformer informers.GenericInformer
log logr.Logger
}
//NewController returns a new controller instance to manage generate-requests
@ -62,6 +62,7 @@ func NewController(
pInformer kyvernoinformer.ClusterPolicyInformer,
grInformer kyvernoinformer.GenerateRequestInformer,
dynamicInformer dynamicinformer.DynamicSharedInformerFactory,
log logr.Logger,
) *Controller {
c := Controller{
kyvernoClient: kyvernoclient,
@ -70,6 +71,7 @@ func NewController(
// as we dont want a deleted GR to be re-queue
queue: workqueue.NewNamedRateLimitingQueue(workqueue.NewItemExponentialFailureRateLimiter(1, 30), "generate-request-cleanup"),
dynamicInformer: dynamicInformer,
log: log,
}
c.control = Control{client: kyvernoclient}
c.enqueueGR = c.enqueue
@ -102,10 +104,11 @@ func NewController(
}
func (c *Controller) deleteGenericResource(obj interface{}) {
logger := c.log
r := obj.(*unstructured.Unstructured)
grs, err := c.grLister.GetGenerateRequestsForResource(r.GetKind(), r.GetNamespace(), r.GetName())
if err != nil {
glog.Errorf("failed to Generate Requests for resource %s/%s/%s: %v", r.GetKind(), r.GetNamespace(), r.GetName(), err)
logger.Error(err, "failed to get generate request CR for resource", "kind", r.GetKind(), "namespace", r.GetNamespace(), "name", r.GetName())
return
}
// re-evaluate the GR as the resource was deleted
@ -115,26 +118,27 @@ func (c *Controller) deleteGenericResource(obj interface{}) {
}
func (c *Controller) deletePolicy(obj interface{}) {
logger := c.log
p, ok := obj.(*kyverno.ClusterPolicy)
if !ok {
tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
if !ok {
glog.Info(fmt.Errorf("Couldn't get object from tombstone %#v", obj))
logger.Info("ouldn't get object from tombstone", "obj", obj)
return
}
_, ok = tombstone.Obj.(*kyverno.ClusterPolicy)
if !ok {
glog.Info(fmt.Errorf("Tombstone contained object that is not a Generate Request %#v", obj))
logger.Info("Tombstone contained object that is not a Generate Request", "obj", obj)
return
}
}
glog.V(4).Infof("Deleting Policy %s", p.Name)
logger.V(4).Info("deleting policy", "name", p.Name)
// clean up the GR
// Get the corresponding GR
// get the list of GR for the current Policy version
grs, err := c.grLister.GetGenerateRequestsForClusterPolicy(p.Name)
if err != nil {
glog.Errorf("failed to Generate Requests for policy %s: %v", p.Name, err)
logger.Error(err, "failed to generate request CR for the policy", "name", p.Name)
return
}
for _, gr := range grs {
@ -153,48 +157,50 @@ func (c *Controller) updateGR(old, cur interface{}) {
}
func (c *Controller) deleteGR(obj interface{}) {
logger := c.log
gr, ok := obj.(*kyverno.GenerateRequest)
if !ok {
tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
if !ok {
glog.Info(fmt.Errorf("Couldn't get object from tombstone %#v", obj))
logger.Info("Couldn't get object from tombstone", "obj", obj)
return
}
_, ok = tombstone.Obj.(*kyverno.GenerateRequest)
if !ok {
glog.Info(fmt.Errorf("Tombstone contained object that is not a Generate Request %#v", obj))
logger.Info("ombstone contained object that is not a Generate Request", "obj", obj)
return
}
}
glog.V(4).Infof("Deleting GR %s", gr.Name)
logger.V(4).Info("deleting Generate Request CR", "name", gr.Name)
// sync Handler will remove it from the queue
c.enqueueGR(gr)
}
func (c *Controller) enqueue(gr *kyverno.GenerateRequest) {
logger := c.log
key, err := cache.MetaNamespaceKeyFunc(gr)
if err != nil {
glog.Error(err)
logger.Error(err, "failed to extract key")
return
}
glog.V(4).Infof("cleanup enqueu: %v", gr.Name)
logger.V(4).Info("eneque generate request", "name", gr.Name)
c.queue.Add(key)
}
//Run starts the generate-request re-conciliation loop
func (c *Controller) Run(workers int, stopCh <-chan struct{}) {
logger := c.log
defer utilruntime.HandleCrash()
defer c.queue.ShutDown()
glog.Info("Starting generate-policy-cleanup controller")
defer glog.Info("Shutting down generate-policy-cleanup controller")
logger.Info("starting")
defer logger.Info("shutting down")
if !cache.WaitForCacheSync(stopCh, c.pSynced, c.grSynced) {
glog.Error("generate-policy-cleanup controller: failed to sync informer cache")
logger.Info("failed to sync informer cache")
return
}
for i := 0; i < workers; i++ {
go wait.Until(c.worker, time.Second, stopCh)
go wait.Until(c.worker, constant.GenerateRequestControllerResync, stopCh)
}
<-stopCh
}
@ -219,31 +225,33 @@ func (c *Controller) processNextWorkItem() bool {
}
func (c *Controller) handleErr(err error, key interface{}) {
logger := c.log
if err == nil {
c.queue.Forget(key)
return
}
if c.queue.NumRequeues(key) < maxRetries {
glog.Errorf("Error syncing Generate Request %v: %v", key, err)
logger.Error(err, "failed to sync generate request", "key", key)
c.queue.AddRateLimited(key)
return
}
utilruntime.HandleError(err)
glog.Infof("Dropping generate request %q out of the queue: %v", key, err)
logger.Error(err, "dropping generate request out of the queue", "key", key)
c.queue.Forget(key)
}
func (c *Controller) syncGenerateRequest(key string) error {
logger := c.log.WithValues("key", key)
var err error
startTime := time.Now()
glog.V(4).Infof("Started syncing GR %q (%v)", key, startTime)
logger.Info("started syncing generate request", "startTime", startTime)
defer func() {
glog.V(4).Infof("Finished syncing GR %q (%v)", key, time.Since(startTime))
logger.V(4).Info("finished syncying generate request", "processingTIme", time.Since(startTime))
}()
_, grName, err := cache.SplitMetaNamespaceKey(key)
if errors.IsNotFound(err) {
glog.Infof("Generate Request %s has been deleted", key)
logger.Info("generate request has been deleted")
return nil
}
if err != nil {

48
pkg/generate/controller.go
View file

@ -1,14 +1,14 @@
package generate
import (
"fmt"
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
kyvernoclient "github.com/nirmata/kyverno/pkg/client/clientset/versioned"
kyvernoinformer "github.com/nirmata/kyverno/pkg/client/informers/externalversions/kyverno/v1"
kyvernolister "github.com/nirmata/kyverno/pkg/client/listers/kyverno/v1"
"github.com/nirmata/kyverno/pkg/constant"
dclient "github.com/nirmata/kyverno/pkg/dclient"
"github.com/nirmata/kyverno/pkg/event"
"github.com/nirmata/kyverno/pkg/policystatus"
@ -57,9 +57,9 @@ type Controller struct {
dynamicInformer dynamicinformer.DynamicSharedInformerFactory
//TODO: list of generic informers
// only support Namespaces for re-evalutation on resource updates
nsInformer informers.GenericInformer
nsInformer informers.GenericInformer
policyStatusListener policystatus.Listener
log logr.Logger
}
//NewController returns an instance of the Generate-Request Controller
@ -72,6 +72,7 @@ func NewController(
pvGenerator policyviolation.GeneratorInterface,
dynamicInformer dynamicinformer.DynamicSharedInformerFactory,
policyStatus policystatus.Listener,
log logr.Logger,
) *Controller {
c := Controller{
client: client,
@ -82,6 +83,7 @@ func NewController(
// as we dont want a deleted GR to be re-queue
queue: workqueue.NewNamedRateLimitingQueue(workqueue.NewItemExponentialFailureRateLimiter(1, 30), "generate-request"),
dynamicInformer: dynamicInformer,
log: log,
policyStatusListener: policyStatus,
}
c.statusControl = StatusControl{client: kyvernoclient}
@ -117,11 +119,12 @@ func NewController(
}
func (c *Controller) updateGenericResource(old, cur interface{}) {
logger := c.log
curR := cur.(*unstructured.Unstructured)
grs, err := c.grLister.GetGenerateRequestsForResource(curR.GetKind(), curR.GetNamespace(), curR.GetName())
if err != nil {
glog.Errorf("failed to Generate Requests for resource %s/%s/%s: %v", curR.GetKind(), curR.GetNamespace(), curR.GetName(), err)
logger.Error(err, "failed to get generate request CR for the resoource", "kind", curR.GetKind(), "name", curR.GetName(), "namespace", curR.GetNamespace())
return
}
// re-evaluate the GR as the resource was updated
@ -134,13 +137,14 @@ func (c *Controller) updateGenericResource(old, cur interface{}) {
func (c *Controller) enqueue(gr *kyverno.GenerateRequest) {
key, err := cache.MetaNamespaceKeyFunc(gr)
if err != nil {
glog.Error(err)
c.log.Error(err, "failed to extract name")
return
}
c.queue.Add(key)
}
func (c *Controller) updatePolicy(old, cur interface{}) {
logger := c.log
oldP := old.(*kyverno.ClusterPolicy)
curP := cur.(*kyverno.ClusterPolicy)
if oldP.ResourceVersion == curP.ResourceVersion {
@ -148,11 +152,11 @@ func (c *Controller) updatePolicy(old, cur interface{}) {
// Two different versions of the same replica set will always have different RVs.
return
}
glog.V(4).Infof("Updating Policy %s", oldP.Name)
logger.V(4).Info("updating policy", "name", oldP.Name)
// get the list of GR for the current Policy version
grs, err := c.grLister.GetGenerateRequestsForClusterPolicy(curP.Name)
if err != nil {
glog.Errorf("failed to Generate Requests for policy %s: %v", curP.Name, err)
logger.Error(err, "failed to generate request for policy", "name", curP.Name)
return
}
// re-evaluate the GR as the policy was updated
@ -183,38 +187,40 @@ func (c *Controller) updateGR(old, cur interface{}) {
}
func (c *Controller) deleteGR(obj interface{}) {
logger := c.log
gr, ok := obj.(*kyverno.GenerateRequest)
if !ok {
tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
if !ok {
glog.Info(fmt.Errorf("Couldn't get object from tombstone %#v", obj))
logger.Info("Couldn't get object from tombstone", "obj", obj)
return
}
_, ok = tombstone.Obj.(*kyverno.GenerateRequest)
if !ok {
glog.Info(fmt.Errorf("Tombstone contained object that is not a Generate Request %#v", obj))
logger.Info("tombstone contained object that is not a Generate Request CR", "obj", obj)
return
}
}
glog.V(4).Infof("Deleting GR %s", gr.Name)
logger.Info("deleting generate request", "name", gr.Name)
// sync Handler will remove it from the queue
c.enqueueGR(gr)
}
//Run ...
func (c *Controller) Run(workers int, stopCh <-chan struct{}) {
logger := c.log
defer utilruntime.HandleCrash()
defer c.queue.ShutDown()
glog.Info("Starting generate-policy controller")
defer glog.Info("Shutting down generate-policy controller")
logger.Info("starting")
defer logger.Info("shutting down")
if !cache.WaitForCacheSync(stopCh, c.pSynced, c.grSynced) {
glog.Error("generate-policy controller: failed to sync informer cache")
logger.Info("failed to sync informer cache")
return
}
for i := 0; i < workers; i++ {
go wait.Until(c.worker, time.Second, stopCh)
go wait.Until(c.worker, constant.GenerateControllerResync, stopCh)
}
<-stopCh
}
@ -239,27 +245,29 @@ func (c *Controller) processNextWorkItem() bool {
}
func (c *Controller) handleErr(err error, key interface{}) {
logger := c.log
if err == nil {
c.queue.Forget(key)
return
}
if c.queue.NumRequeues(key) < maxRetries {
glog.Errorf("Error syncing Generate Request %v: %v", key, err)
logger.Error(err, "failed to sync generate request", "key", key)
c.queue.AddRateLimited(key)
return
}
utilruntime.HandleError(err)
glog.Infof("Dropping generate request %q out of the queue: %v", key, err)
logger.Error(err, "Dropping generate request from the queue", "key", key)
c.queue.Forget(key)
}
func (c *Controller) syncGenerateRequest(key string) error {
logger := c.log
var err error
startTime := time.Now()
glog.V(4).Infof("Started syncing GR %q (%v)", key, startTime)
logger.Info("started sync", "key", key, "startTime", startTime)
defer func() {
glog.V(4).Infof("Finished syncing GR %q (%v)", key, time.Since(startTime))
logger.V(4).Info("finished sync", "key", key, "processingTime", time.Since(startTime))
}()
_, grName, err := cache.SplitMetaNamespaceKey(key)
if err != nil {
@ -268,7 +276,7 @@ func (c *Controller) syncGenerateRequest(key string) error {
gr, err := c.grLister.Get(grName)
if err != nil {
glog.V(4).Info(err)
logger.Error(err, "failed to list generate requests")
return err
}
return c.processGR(gr)

71
pkg/generate/generate.go
View file

@ -5,7 +5,7 @@ import (
"fmt"
"time"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
dclient "github.com/nirmata/kyverno/pkg/dclient"
"github.com/nirmata/kyverno/pkg/engine"
@ -17,6 +17,7 @@ import (
)
func (c *Controller) processGR(gr *kyverno.GenerateRequest) error {
logger := c.log.WithValues("name", gr.Name, "policy", gr.Spec.Policy, "kind", gr.Spec.Resource.Kind, "namespace", gr.Spec.Resource.Namespace, "name", gr.Spec.Resource.Name)
var err error
var resource *unstructured.Unstructured
var genResources []kyverno.ResourceSpec
@ -24,45 +25,46 @@ func (c *Controller) processGR(gr *kyverno.GenerateRequest) error {
resource, err = getResource(c.client, gr.Spec.Resource)
if err != nil {
// Dont update status
glog.V(4).Infof("resource does not exist or is yet to be created, requeuing: %v", err)
logger.Error(err, "resource does not exist or is yet to be created, requeueing")
return err
}
// 2 - Apply the generate policy on the resource
genResources, err = c.applyGenerate(*resource, *gr)
// 3 - Report Events
reportEvents(err, c.eventGen, *gr, *resource)
reportEvents(logger, err, c.eventGen, *gr, *resource)
// 4 - Update Status
return updateStatus(c.statusControl, *gr, err, genResources)
}
func (c *Controller) applyGenerate(resource unstructured.Unstructured, gr kyverno.GenerateRequest) ([]kyverno.ResourceSpec, error) {
logger := c.log.WithValues("name", gr.Name, "policy", gr.Spec.Policy, "kind", gr.Spec.Resource.Kind, "namespace", gr.Spec.Resource.Namespace, "name", gr.Spec.Resource.Name)
// Get the list of rules to be applied
// get policy
policy, err := c.pLister.Get(gr.Spec.Policy)
if err != nil {
glog.V(4).Infof("policy %s not found: %v", gr.Spec.Policy, err)
logger.Error(err, "policy not found")
return nil, nil
}
// build context
ctx := context.NewContext()
resourceRaw, err := resource.MarshalJSON()
if err != nil {
glog.V(4).Infof("failed to marshal resource: %v", err)
logger.Error(err, "failed to marshal resource")
return nil, err
}
err = ctx.AddResource(resourceRaw)
if err != nil {
glog.Infof("Failed to load resource in context: %v", err)
logger.Error(err, "failed to load resource in context")
return nil, err
}
err = ctx.AddUserInfo(gr.Spec.Context.UserRequestInfo)
if err != nil {
glog.Infof("Failed to load userInfo in context: %v", err)
logger.Error(err, "failed to load SA in context")
return nil, err
}
err = ctx.AddSA(gr.Spec.Context.UserRequestInfo.AdmissionUserInfo.Username)
if err != nil {
glog.Infof("Failed to load serviceAccount in context: %v", err)
logger.Error(err, "failed to load UserInfo in context")
return nil, err
}
@ -76,12 +78,12 @@ func (c *Controller) applyGenerate(resource unstructured.Unstructured, gr kyvern
// check if the policy still applies to the resource
engineResponse := engine.Generate(policyContext)
if len(engineResponse.PolicyResponse.Rules) == 0 {
glog.V(4).Infof("policy %s, dont not apply to resource %v", gr.Spec.Policy, gr.Spec.Resource)
logger.V(4).Info("policy does not apply to resource")
return nil, fmt.Errorf("policy %s, dont not apply to resource %v", gr.Spec.Policy, gr.Spec.Resource)
}
// Apply the generate rule on resource
return c.applyGeneratePolicy(policyContext, gr)
return c.applyGeneratePolicy(logger, policyContext, gr)
}
func updateStatus(statusControl StatusControlInterface, gr kyverno.GenerateRequest, err error, genResources []kyverno.ResourceSpec) error {
@ -93,7 +95,7 @@ func updateStatus(statusControl StatusControlInterface, gr kyverno.GenerateReque
return statusControl.Success(gr, genResources)
}
func (c *Controller) applyGeneratePolicy(policyContext engine.PolicyContext, gr kyverno.GenerateRequest) ([]kyverno.ResourceSpec, error) {
func (c *Controller) applyGeneratePolicy(log logr.Logger, policyContext engine.PolicyContext, gr kyverno.GenerateRequest) ([]kyverno.ResourceSpec, error) {
// List of generatedResources
var genResources []kyverno.ResourceSpec
// Get the response as the actions to be performed on the resource
@ -113,9 +115,8 @@ func (c *Controller) applyGeneratePolicy(policyContext engine.PolicyContext, gr
if !rule.HasGenerate() {
continue
}
startTime := time.Now()
genResource, err := applyRule(c.client, rule, resource, ctx, processExisting)
genResource, err := applyRule(log, c.client, rule, resource, ctx, processExisting)
if err != nil {
return nil, err
}
@ -172,7 +173,7 @@ func updateGenerateExecutionTime(newTime time.Duration, oldAverageTimeString str
return time.Duration(newAverageTimeInNanoSeconds) * time.Nanosecond
}
func applyRule(client *dclient.Client, rule kyverno.Rule, resource unstructured.Unstructured, ctx context.EvalInterface, processExisting bool) (kyverno.ResourceSpec, error) {
func applyRule(log logr.Logger, client *dclient.Client, rule kyverno.Rule, resource unstructured.Unstructured, ctx context.EvalInterface, processExisting bool) (kyverno.ResourceSpec, error) {
var rdata map[string]interface{}
var err error
var mode ResourceMode
@ -187,9 +188,12 @@ func applyRule(client *dclient.Client, rule kyverno.Rule, resource unstructured.
// format : {{<variable_name}}
// - if there is variables that are not defined the context -> results in error and rule is not applied
// - valid variables are replaced with the values
if _, err := variables.SubstituteVars(ctx, genUnst.Object); err != nil {
object, err := variables.SubstituteVars(log, ctx, genUnst.Object)
if err != nil {
return noGenResource, err
}
genUnst.Object, _ = object.(map[string]interface{})
genKind, _, err := unstructured.NestedString(genUnst.Object, "kind")
if err != nil {
return noGenResource, err
@ -219,9 +223,9 @@ func applyRule(client *dclient.Client, rule kyverno.Rule, resource unstructured.
}
if genData != nil {
rdata, mode, err = manageData(genKind, genNamespace, genName, genData, client, resource)
rdata, mode, err = manageData(log, genKind, genNamespace, genName, genData, client, resource)
} else {
rdata, mode, err = manageClone(genKind, genNamespace, genName, genCopy, client, resource)
rdata, mode, err = manageClone(log, genKind, genNamespace, genName, genCopy, client, resource)
}
if err != nil {
return noGenResource, err
@ -243,43 +247,46 @@ func applyRule(client *dclient.Client, rule kyverno.Rule, resource unstructured.
newResource.SetUnstructuredContent(rdata)
newResource.SetName(genName)
newResource.SetNamespace(genNamespace)
if newResource.GetKind() == "" {
newResource.SetKind(genKind)
}
// manage labels
// - app.kubernetes.io/managed-by: kyverno
// - kyverno.io/generated-by: kind/namespace/name (trigger resource)
manageLabels(newResource, resource)
logger := log.WithValues("genKind", genKind, "genNamespace", genNamespace, "genName", genName)
if mode == Create {
// Reset resource version
newResource.SetResourceVersion("")
// Create the resource
glog.V(4).Infof("Creating new resource %s/%s/%s", genKind, genNamespace, genName)
logger.V(4).Info("creating new resource")
_, err = client.CreateResource(genKind, genNamespace, newResource, false)
if err != nil {
// Failed to create resource
return noGenResource, err
}
glog.V(4).Infof("Created new resource %s/%s/%s", genKind, genNamespace, genName)
logger.V(4).Info("created new resource")
} else if mode == Update {
glog.V(4).Infof("Updating existing resource %s/%s/%s", genKind, genNamespace, genName)
logger.V(4).Info("updating existing resource")
// Update the resource
_, err := client.UpdateResource(genKind, genNamespace, newResource, false)
if err != nil {
// Failed to update resource
return noGenResource, err
}
glog.V(4).Infof("Updated existing resource %s/%s/%s", genKind, genNamespace, genName)
logger.V(4).Info("updated new resource")
}
return newGenResource, nil
}
func manageData(kind, namespace, name string, data map[string]interface{}, client *dclient.Client, resource unstructured.Unstructured) (map[string]interface{}, ResourceMode, error) {
func manageData(log logr.Logger, kind, namespace, name string, data map[string]interface{}, client *dclient.Client, resource unstructured.Unstructured) (map[string]interface{}, ResourceMode, error) {
// check if resource to be generated exists
obj, err := client.GetResource(kind, namespace, name)
if apierrors.IsNotFound(err) {
glog.V(4).Infof("Resource %s/%s/%s does not exists, will try to create", kind, namespace, name)
log.Error(err, "resource does not exist, will try to create", "genKind", kind, "genNamespace", namespace, "genName", name)
return data, Create, nil
}
if err != nil {
@ -288,18 +295,17 @@ func manageData(kind, namespace, name string, data map[string]interface{}, clien
return nil, Skip, err
}
// Resource exists; verfiy the content of the resource
err = checkResource(data, obj)
err = checkResource(log, data, obj)
if err == nil {
// Existing resource does contain the mentioned configuration in spec, skip processing the resource as it is already in expected state
return nil, Skip, nil
}
glog.V(4).Infof("Resource %s/%s/%s exists but missing required configuration, will try to update", kind, namespace, name)
log.Info("to be generated resoruce already exists, but is missing the specifeid configurations, will try to update", "genKind", kind, "genNamespace", namespace, "genName", name)
return data, Update, nil
}
func manageClone(kind, namespace, name string, clone map[string]interface{}, client *dclient.Client, resource unstructured.Unstructured) (map[string]interface{}, ResourceMode, error) {
func manageClone(log logr.Logger, kind, namespace, name string, clone map[string]interface{}, client *dclient.Client, resource unstructured.Unstructured) (map[string]interface{}, ResourceMode, error) {
// check if resource to be generated exists
_, err := client.GetResource(kind, namespace, name)
if err == nil {
@ -308,6 +314,7 @@ func manageClone(kind, namespace, name string, clone map[string]interface{}, cli
}
//TODO: check this
if !apierrors.IsNotFound(err) {
log.Error(err, "reference/clone resource is not found", "genKind", kind, "genNamespace", namespace, "genName", name)
//something wrong while fetching resource
return nil, Skip, err
}
@ -325,8 +332,6 @@ func manageClone(kind, namespace, name string, clone map[string]interface{}, cli
// attempting to clone it self, this will fail -> short-ciruit it
return nil, Skip, nil
}
glog.V(4).Infof("check if resource %s/%s/%s exists", kind, newRNs, newRName)
// check if the resource as reference in clone exists?
obj, err := client.GetResource(kind, newRNs, newRName)
if err != nil {
@ -349,10 +354,10 @@ const (
Update = "UPDATE"
)
func checkResource(newResourceSpec interface{}, resource *unstructured.Unstructured) error {
func checkResource(log logr.Logger, newResourceSpec interface{}, resource *unstructured.Unstructured) error {
// check if the resource spec if a subset of the resource
if path, err := validate.ValidateResourceWithPattern(resource.Object, newResourceSpec); err != nil {
glog.V(4).Infof("Failed to match the resource at path %s: err %v", path, err)
if path, err := validate.ValidateResourceWithPattern(log, resource.Object, newResourceSpec); err != nil {
log.Error(err, "Failed to match the resource ", "path", path)
return err
}
return nil

6
pkg/generate/labels.go
View file

@ -3,8 +3,8 @@ package generate
import (
"fmt"
"github.com/golang/glog"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/log"
)
func manageLabels(unstr *unstructured.Unstructured, triggerResource unstructured.Unstructured) {
@ -30,7 +30,7 @@ func managedBy(labels map[string]string) {
val, ok := labels[key]
if ok {
if val != value {
glog.Infof("resource managed by %s, kyverno wont over-ride the label", val)
log.Log.Info(fmt.Sprintf("resource managed by %s, kyverno wont over-ride the label", val))
return
}
}
@ -46,7 +46,7 @@ func generatedBy(labels map[string]string, triggerResource unstructured.Unstruct
val, ok := labels[key]
if ok {
if val != value {
glog.Infof("resource generated by %s, kyverno wont over-ride the label", val)
log.Log.Info(fmt.Sprintf("resource generated by %s, kyverno wont over-ride the label", val))
return
}
}

2
pkg/generate/policyStatus_test.go
View file

@ -15,7 +15,7 @@ func Test_Stats(t *testing.T) {
expectedOutput []byte
existingStatus map[string]v1.PolicyStatus
}{
expectedOutput: []byte(`{"policy1":{"averageExecutionTime":"","resourcesGeneratedCount":2,"ruleStatus":[{"ruleName":"rule1","averageExecutionTime":"23ns","resourcesGeneratedCount":1},{"ruleName":"rule2","averageExecutionTime":"44ns","resourcesGeneratedCount":1},{"ruleName":"rule3"}]}}`),
expectedOutput: []byte(`{"policy1":{"resourcesGeneratedCount":2,"ruleStatus":[{"ruleName":"rule1","averageExecutionTime":"23ns","resourcesGeneratedCount":1},{"ruleName":"rule2","averageExecutionTime":"44ns","resourcesGeneratedCount":1},{"ruleName":"rule3"}]}}`),
generatedSyncStats: []generateSyncStats{
{
policyName: "policy1",

5
pkg/generate/report.go
View file

@ -3,13 +3,13 @@ package generate
import (
"fmt"
"github.com/golang/glog"
"github.com/go-logr/logr"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/nirmata/kyverno/pkg/event"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)
func reportEvents(err error, eventGen event.Interface, gr kyverno.GenerateRequest, resource unstructured.Unstructured) {
func reportEvents(log logr.Logger, err error, eventGen event.Interface, gr kyverno.GenerateRequest, resource unstructured.Unstructured) {
if err == nil {
// Success Events
// - resource -> policy rule applied successfully
@ -18,7 +18,6 @@ func reportEvents(err error, eventGen event.Interface, gr kyverno.GenerateReques
eventGen.Add(events...)
return
}
glog.V(4).Infof("reporing events for %v", err)
events := failedEvents(err, gr, resource)
eventGen.Add(events...)
}

10
pkg/generate/status.go
View file

@ -1,9 +1,9 @@
package generate
import (
"github.com/golang/glog"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
kyvernoclient "github.com/nirmata/kyverno/pkg/client/clientset/versioned"
"sigs.k8s.io/controller-runtime/pkg/log"
)
//StatusControlInterface provides interface to update status subresource
@ -25,10 +25,10 @@ func (sc StatusControl) Failed(gr kyverno.GenerateRequest, message string, genRe
gr.Status.GeneratedResources = genResources
_, err := sc.client.KyvernoV1().GenerateRequests("kyverno").UpdateStatus(&gr)
if err != nil {
glog.V(4).Infof("FAILED: updated gr %s status to %s", gr.Name, string(kyverno.Failed))
log.Log.Error(err, "failed to update generate request status", "name", gr.Name)
return err
}
glog.V(4).Infof("updated gr %s status to %s", gr.Name, string(kyverno.Failed))
log.Log.Info("updated generate request status", "name", gr.Name, "status", string(kyverno.Failed))
return nil
}
@ -41,9 +41,9 @@ func (sc StatusControl) Success(gr kyverno.GenerateRequest, genResources []kyver
_, err := sc.client.KyvernoV1().GenerateRequests("kyverno").UpdateStatus(&gr)
if err != nil {
glog.V(4).Infof("FAILED: updated gr %s status to %s", gr.Name, string(kyverno.Completed))
log.Log.Error(err, "failed to update generate request status", "name", gr.Name)
return err
}
glog.V(4).Infof("updated gr %s status to %s", gr.Name, string(kyverno.Completed))
log.Log.Info("updated generate request status", "name", gr.Name, "status", string(kyverno.Completed))
return nil
}

62
pkg/kyverno/apply/command.go
View file

@ -6,17 +6,21 @@ import (
"io/ioutil"
"os"
"path/filepath"
"regexp"
"time"
client "github.com/nirmata/kyverno/pkg/dclient"
"github.com/nirmata/kyverno/pkg/utils"
"github.com/nirmata/kyverno/pkg/openapi"
"github.com/nirmata/kyverno/pkg/kyverno/sanitizedError"
policy2 "github.com/nirmata/kyverno/pkg/policy"
"github.com/golang/glog"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/client-go/discovery"
"k8s.io/apimachinery/pkg/util/yaml"
"github.com/nirmata/kyverno/pkg/engine"
@ -32,6 +36,7 @@ import (
yamlv2 "gopkg.in/yaml.v2"
"k8s.io/cli-runtime/pkg/genericclioptions"
"k8s.io/client-go/kubernetes/scheme"
log "sigs.k8s.io/controller-runtime/pkg/log"
)
func Command() *cobra.Command {
@ -49,7 +54,7 @@ func Command() *cobra.Command {
defer func() {
if err != nil {
if !sanitizedError.IsErrorSanitized(err) {
glog.V(4).Info(err)
log.Log.Error(err, "failed to sanitize")
err = fmt.Errorf("Internal error")
}
}
@ -68,18 +73,30 @@ func Command() *cobra.Command {
}
}
openAPIController, err := openapi.NewOpenAPIController()
if err != nil {
return err
}
for _, policy := range policies {
err := policy2.Validate(*policy)
err := policy2.Validate(utils.MarshalPolicy(*policy), nil, true, openAPIController)
if err != nil {
return sanitizedError.New(fmt.Sprintf("Policy %v is not valid", policy.Name))
}
if policyHasVariables(*policy) {
return sanitizedError.New(fmt.Sprintf("Policy %v is not valid - 'apply' does not support policies with variables", policy.Name))
}
}
var dClient discovery.CachedDiscoveryInterface
var dClient *client.Client
if cluster {
dClient, err = kubernetesConfig.ToDiscoveryClient()
restConfig, err := kubernetesConfig.ToRESTConfig()
if err != nil {
return sanitizedError.New(fmt.Errorf("Issues with kubernetes Config").Error())
return err
}
dClient, err = client.NewClient(restConfig, 5*time.Minute, make(chan struct{}), log.Log)
if err != nil {
return err
}
}
@ -111,7 +128,7 @@ func Command() *cobra.Command {
return cmd
}
func getResources(policies []*v1.ClusterPolicy, resourcePaths []string, dClient discovery.CachedDiscoveryInterface) ([]*unstructured.Unstructured, error) {
func getResources(policies []*v1.ClusterPolicy, resourcePaths []string, dClient *client.Client) ([]*unstructured.Unstructured, error) {
var resources []*unstructured.Unstructured
var err error
@ -148,26 +165,11 @@ func getResources(policies []*v1.ClusterPolicy, resourcePaths []string, dClient
return resources, nil
}
func getResourcesOfTypeFromCluster(resourceTypes []string, dClient discovery.CachedDiscoveryInterface) ([]*unstructured.Unstructured, error) {
func getResourcesOfTypeFromCluster(resourceTypes []string, dClient *client.Client) ([]*unstructured.Unstructured, error) {
var resources []*unstructured.Unstructured
for _, kind := range resourceTypes {
endpoint, err := getListEndpointForKind(kind)
if err != nil {
return nil, err
}
listObjectRaw, err := dClient.RESTClient().Get().RequestURI(endpoint).Do().Raw()
if err != nil {
return nil, err
}
listObject, err := engineutils.ConvertToUnstructured(listObjectRaw)
if err != nil {
return nil, err
}
resourceList, err := listObject.ToList()
resourceList, err := dClient.ListResource(kind, "", nil)
if err != nil {
return nil, err
}
@ -378,3 +380,9 @@ func applyPolicyOnResource(policy *v1.ClusterPolicy, resource *unstructured.Unst
return nil
}
func policyHasVariables(policy v1.ClusterPolicy) bool {
policyRaw, _ := json.Marshal(policy)
regex := regexp.MustCompile(`\{\{([^{}]*)\}\}`)
return len(regex.FindAllStringSubmatch(string(policyRaw), -1)) > 0
}

37
pkg/kyverno/apply/helper.go
View file

@ -1,37 +0,0 @@
package apply
import (
"fmt"
"strings"
"github.com/nirmata/kyverno/pkg/openapi"
)
func getListEndpointForKind(kind string) (string, error) {
definitionName := openapi.GetDefinitionNameFromKind(kind)
definitionNameWithoutPrefix := strings.Replace(definitionName, "io.k8s.", "", -1)
parts := strings.Split(definitionNameWithoutPrefix, ".")
definitionPrefix := strings.Join(parts[:len(parts)-1], ".")
defPrefixToApiPrefix := map[string]string{
"api.core.v1": "/api/v1",
"api.apps.v1": "/apis/apps/v1",
"api.batch.v1": "/apis/batch/v1",
"api.admissionregistration.v1": "/apis/admissionregistration.k8s.io/v1",
"kube-aggregator.pkg.apis.apiregistration.v1": "/apis/apiregistration.k8s.io/v1",
"apiextensions-apiserver.pkg.apis.apiextensions.v1": "/apis/apiextensions.k8s.io/v1",
"api.autoscaling.v1": "/apis/autoscaling/v1/",
"api.storage.v1": "/apis/storage.k8s.io/v1",
"api.coordination.v1": "/apis/coordination.k8s.io/v1",
"api.scheduling.v1": "/apis/scheduling.k8s.io/v1",
"api.rbac.v1": "/apis/rbac.authorization.k8s.io/v1",
}
if defPrefixToApiPrefix[definitionPrefix] == "" {
return "", fmt.Errorf("Unsupported resource type %v", kind)
}
return defPrefixToApiPrefix[definitionPrefix] + "/" + strings.ToLower(kind) + "s", nil
}

11
pkg/kyverno/main.go
View file

@ -9,6 +9,9 @@ import (
"github.com/nirmata/kyverno/pkg/kyverno/apply"
"github.com/nirmata/kyverno/pkg/kyverno/version"
"k8s.io/klog"
"k8s.io/klog/klogr"
log "sigs.k8s.io/controller-runtime/pkg/log"
"github.com/spf13/cobra"
)
@ -19,7 +22,7 @@ func CLI() {
Short: "kyverno manages native policies of Kubernetes",
}
configureGlog(cli)
configurelog(cli)
commands := []*cobra.Command{
version.Command(),
@ -36,9 +39,9 @@ func CLI() {
}
}
func configureGlog(cli *cobra.Command) {
flag.Parse()
_ = flag.Set("logtostderr", "true")
func configurelog(cli *cobra.Command) {
klog.InitFlags(nil)
log.SetLogger(klogr.New())
cli.PersistentFlags().AddGoFlagSet(flag.CommandLine)
_ = cli.PersistentFlags().MarkHidden("alsologtostderr")

16
pkg/kyverno/validate/command.go
View file

@ -7,15 +7,18 @@ import (
"os"
"path/filepath"
"github.com/nirmata/kyverno/pkg/kyverno/sanitizedError"
"github.com/nirmata/kyverno/pkg/utils"
"github.com/golang/glog"
"github.com/nirmata/kyverno/pkg/openapi"
"github.com/nirmata/kyverno/pkg/kyverno/sanitizedError"
policyvalidate "github.com/nirmata/kyverno/pkg/policy"
v1 "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
"github.com/spf13/cobra"
"k8s.io/apimachinery/pkg/util/yaml"
log "sigs.k8s.io/controller-runtime/pkg/log"
)
func Command() *cobra.Command {
@ -27,7 +30,7 @@ func Command() *cobra.Command {
defer func() {
if err != nil {
if !sanitizedError.IsErrorSanitized(err) {
glog.V(4).Info(err)
log.Log.Error(err, "failed to sanitize")
err = fmt.Errorf("Internal error")
}
}
@ -42,8 +45,13 @@ func Command() *cobra.Command {
}
}
openAPIController, err := openapi.NewOpenAPIController()
if err != nil {
return err
}
for _, policy := range policies {
err = policyvalidate.Validate(*policy)
err = policyvalidate.Validate(utils.MarshalPolicy(*policy), nil, true, openAPIController)
if err != nil {
fmt.Println("Policy " + policy.Name + " is invalid")
} else {

165
pkg/openapi/crdSync.go
View file

@ -2,23 +2,46 @@ package openapi
import (
"encoding/json"
"time"
"errors"
"fmt"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
runtimeSchema "k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"github.com/golang/glog"
"gopkg.in/yaml.v2"
"github.com/googleapis/gnostic/compiler"
openapi_v2 "github.com/googleapis/gnostic/OpenAPIv2"
"sigs.k8s.io/controller-runtime/pkg/log"
"github.com/nirmata/kyverno/pkg/constant"
client "github.com/nirmata/kyverno/pkg/dclient"
"k8s.io/apimachinery/pkg/util/wait"
)
type crdDefinition struct {
type crdSync struct {
client *client.Client
controller *Controller
}
// crdDefinitionPrior represents CRDs version prior to 1.16
var crdDefinitionPrior struct {
Spec struct {
Names struct {
Kind string `json:"kind"`
} `json:"names"`
Validation struct {
OpenAPIV3Schema interface{} `json:"openAPIV3Schema"`
} `json:"validation"`
} `json:"spec"`
}
// crdDefinitionNew represents CRDs version 1.16+
var crdDefinitionNew struct {
Spec struct {
Names struct {
Kind string `json:"kind"`
@ -27,86 +50,152 @@ type crdDefinition struct {
Schema struct {
OpenAPIV3Schema interface{} `json:"openAPIV3Schema"`
} `json:"schema"`
Storage bool `json:"storage"`
} `json:"versions"`
} `json:"spec"`
}
type crdSync struct {
client *client.Client
}
func NewCRDSync(client *client.Client, controller *Controller) *crdSync {
if controller == nil {
panic(fmt.Errorf("nil controller sent into crd sync"))
}
func NewCRDSync(client *client.Client) *crdSync {
return &crdSync{
client: client,
controller: controller,
client: client,
}
}
func (c *crdSync) Run(workers int, stopCh <-chan struct{}) {
newDoc, err := c.client.DiscoveryClient.OpenAPISchema()
if err != nil {
glog.V(4).Infof("cannot get openapi schema: %v", err)
log.Log.Error(err, "cannot get OpenAPI schema")
}
err = useOpenApiDocument(newDoc)
err = c.controller.useOpenApiDocument(newDoc)
if err != nil {
glog.V(4).Infof("Could not set custom OpenApi document: %v\n", err)
log.Log.Error(err, "Could not set custom OpenAPI document")
}
// Sync CRD before kyverno starts
c.sync()
for i := 0; i < workers; i++ {
go wait.Until(c.sync, time.Second*10, stopCh)
go wait.Until(c.sync, constant.CRDControllerResync, stopCh)
}
<-stopCh
}
func (c *crdSync) sync() {
openApiGlobalState.mutex.Lock()
defer openApiGlobalState.mutex.Unlock()
crds, err := c.client.ListResource("CustomResourceDefinition", "", nil)
crds, err := c.client.GetDynamicInterface().Resource(runtimeSchema.GroupVersionResource{
Group: "apiextensions.k8s.io",
Version: "v1beta1",
Resource: "customresourcedefinitions",
}).List(v1.ListOptions{})
if err != nil {
glog.V(4).Infof("could not fetch crd's from server: %v", err)
log.Log.Error(err, "could not fetch crd's from server")
return
}
deleteCRDFromPreviousSync()
c.controller.mutex.Lock()
defer c.controller.mutex.Unlock()
c.controller.deleteCRDFromPreviousSync()
for _, crd := range crds.Items {
parseCRD(crd)
c.controller.parseCRD(crd)
}
}
func deleteCRDFromPreviousSync() {
for _, crd := range openApiGlobalState.crdList {
delete(openApiGlobalState.kindToDefinitionName, crd)
delete(openApiGlobalState.definitions, crd)
func (o *Controller) deleteCRDFromPreviousSync() {
for k := range o.kindToDefinitionName {
delete(o.kindToDefinitionName, k)
}
openApiGlobalState.crdList = []string{}
o.kindToDefinitionName = make(map[string]string, 0)
for k := range o.definitions {
delete(o.definitions, k)
}
o.definitions = make(map[string]*openapi_v2.Schema, 0)
}
func parseCRD(crd unstructured.Unstructured) {
var crdDefinition crdDefinition
func (o *Controller) parseCRD(crd unstructured.Unstructured) {
var err error
crdRaw, _ := json.Marshal(crd.Object)
_ = json.Unmarshal(crdRaw, &crdDefinition)
_ = json.Unmarshal(crdRaw, &crdDefinitionPrior)
crdName := crdDefinition.Spec.Names.Kind
if len(crdDefinition.Spec.Versions) < 1 {
glog.V(4).Infof("could not parse crd schema, no versions present")
openV3schema := crdDefinitionPrior.Spec.Validation.OpenAPIV3Schema
crdName := crdDefinitionPrior.Spec.Names.Kind
if openV3schema == nil {
_ = json.Unmarshal(crdRaw, &crdDefinitionNew)
for _, crdVersion := range crdDefinitionNew.Spec.Versions {
if crdVersion.Storage {
openV3schema = crdVersion.Schema.OpenAPIV3Schema
crdName = crdDefinitionNew.Spec.Names.Kind
break
}
}
}
if openV3schema == nil {
log.Log.V(3).Info("skip adding schema, CRD has no properties", "name", crdName)
return
}
schemaRaw, _ := json.Marshal(openV3schema)
if len(schemaRaw) < 1 {
log.Log.V(3).Info("could not parse crd schema", "name", crdName)
return
}
schemaRaw, err = addingDefaultFieldsToSchema(schemaRaw)
if err != nil {
log.Log.Error(err, "could not parse crd schema", "name", crdName)
return
}
var schema yaml.MapSlice
schemaRaw, _ := json.Marshal(crdDefinition.Spec.Versions[0].Schema.OpenAPIV3Schema)
_ = yaml.Unmarshal(schemaRaw, &schema)
parsedSchema, err := openapi_v2.NewSchema(schema, compiler.NewContext("schema", nil))
if err != nil {
glog.V(4).Infof("could not parse crd schema:%v", err)
log.Log.Error(err, "could not parse crd schema", "name", crdName)
return
}
openApiGlobalState.crdList = append(openApiGlobalState.crdList, crdName)
openApiGlobalState.kindToDefinitionName[crdName] = crdName
openApiGlobalState.definitions[crdName] = parsedSchema
o.kindToDefinitionName[crdName] = crdName
o.definitions[crdName] = parsedSchema
}
// addingDefaultFieldsToSchema will add any default missing fields like apiVersion, metadata
func addingDefaultFieldsToSchema(schemaRaw []byte) ([]byte, error) {
var schema struct {
Properties map[string]interface{} `json:"properties"`
}
_ = json.Unmarshal(schemaRaw, &schema)
if len(schema.Properties) < 1 {
return nil, errors.New("crd schema has no properties")
}
if schema.Properties["apiVersion"] == nil {
apiVersionDefRaw := `{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources","type":"string"}`
apiVersionDef := make(map[string]interface{})
_ = json.Unmarshal([]byte(apiVersionDefRaw), &apiVersionDef)
schema.Properties["apiVersion"] = apiVersionDef
}
if schema.Properties["metadata"] == nil {
metadataDefRaw := `{"$ref":"#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta","description":"Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata"}`
metadataDef := make(map[string]interface{})
_ = json.Unmarshal([]byte(metadataDefRaw), &metadataDef)
schema.Properties["metadata"] = metadataDef
}
schemaWithDefaultFields, _ := json.Marshal(schema)
return schemaWithDefaultFields, nil
}

180
pkg/openapi/validation.go
View file

@ -1,14 +1,15 @@
package openapi
import (
"encoding/json"
"errors"
"fmt"
"strconv"
"strings"
"sync"
"github.com/nirmata/kyverno/data"
"github.com/golang/glog"
data "github.com/nirmata/kyverno/api"
"github.com/nirmata/kyverno/pkg/engine/utils"
"github.com/nirmata/kyverno/pkg/engine"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@ -19,84 +20,67 @@ import (
"github.com/googleapis/gnostic/compiler"
"k8s.io/kube-openapi/pkg/util/proto"
"k8s.io/kube-openapi/pkg/util/proto/validation"
log "sigs.k8s.io/controller-runtime/pkg/log"
"gopkg.in/yaml.v2"
)
var openApiGlobalState struct {
type Controller struct {
mutex sync.RWMutex
document *openapi_v2.Document
definitions map[string]*openapi_v2.Schema
kindToDefinitionName map[string]string
crdList []string
models proto.Models
}
func init() {
func NewOpenAPIController() (*Controller, error) {
controller := &Controller{}
defaultDoc, err := getSchemaDocument()
if err != nil {
panic(err)
return nil, err
}
err = useOpenApiDocument(defaultDoc)
err = controller.useOpenApiDocument(defaultDoc)
if err != nil {
panic(err)
return nil, err
}
return controller, nil
}
func ValidatePolicyMutation(policy v1.ClusterPolicy) error {
openApiGlobalState.mutex.RLock()
defer openApiGlobalState.mutex.RUnlock()
func (o *Controller) ValidatePolicyFields(policyRaw []byte) error {
o.mutex.RLock()
defer o.mutex.RUnlock()
var kindToRules = make(map[string][]v1.Rule)
for _, rule := range policy.Spec.Rules {
if rule.HasMutate() {
rule.MatchResources = v1.MatchResources{
UserInfo: v1.UserInfo{},
ResourceDescription: v1.ResourceDescription{
Kinds: rule.MatchResources.Kinds,
},
}
rule.ExcludeResources = v1.ExcludeResources{}
for _, kind := range rule.MatchResources.Kinds {
kindToRules[kind] = append(kindToRules[kind], rule)
}
}
var policy v1.ClusterPolicy
err := json.Unmarshal(policyRaw, &policy)
if err != nil {
return err
}
for kind, rules := range kindToRules {
newPolicy := *policy.DeepCopy()
newPolicy.Spec.Rules = rules
resource, _ := generateEmptyResource(openApiGlobalState.definitions[openApiGlobalState.kindToDefinitionName[kind]]).(map[string]interface{})
if resource == nil {
glog.V(4).Infof("Cannot Validate policy: openApi definition now found for %v", kind)
return nil
}
newResource := unstructured.Unstructured{Object: resource}
newResource.SetKind(kind)
patchedResource, err := engine.ForceMutate(nil, *newPolicy.DeepCopy(), newResource)
if err != nil {
return err
}
err = ValidateResource(*patchedResource.DeepCopy(), kind)
if err != nil {
return err
}
policyUnst, err := utils.ConvertToUnstructured(policyRaw)
if err != nil {
return err
}
return nil
err = o.ValidateResource(*policyUnst.DeepCopy(), "ClusterPolicy")
if err != nil {
return err
}
return o.ValidatePolicyMutation(policy)
}
func ValidateResource(patchedResource unstructured.Unstructured, kind string) error {
openApiGlobalState.mutex.RLock()
defer openApiGlobalState.mutex.RUnlock()
func (o *Controller) ValidateResource(patchedResource unstructured.Unstructured, kind string) error {
o.mutex.RLock()
defer o.mutex.RUnlock()
var err error
kind = openApiGlobalState.kindToDefinitionName[kind]
schema := openApiGlobalState.models.LookupModel(kind)
kind = o.kindToDefinitionName[kind]
schema := o.models.LookupModel(kind)
if schema == nil {
schema, err = getSchemaFromDefinitions(kind)
// Check if kind is a CRD
schema, err = o.getCRDSchema(kind)
if err != nil || schema == nil {
return fmt.Errorf("pre-validation: couldn't find model %s", kind)
}
@ -115,28 +99,63 @@ func ValidateResource(patchedResource unstructured.Unstructured, kind string) er
return nil
}
func GetDefinitionNameFromKind(kind string) string {
openApiGlobalState.mutex.RLock()
defer openApiGlobalState.mutex.RUnlock()
return openApiGlobalState.kindToDefinitionName[kind]
func (o *Controller) GetDefinitionNameFromKind(kind string) string {
o.mutex.RLock()
defer o.mutex.RUnlock()
return o.kindToDefinitionName[kind]
}
func useOpenApiDocument(customDoc *openapi_v2.Document) error {
openApiGlobalState.mutex.Lock()
defer openApiGlobalState.mutex.Unlock()
func (o *Controller) ValidatePolicyMutation(policy v1.ClusterPolicy) error {
o.mutex.RLock()
defer o.mutex.RUnlock()
openApiGlobalState.document = customDoc
var kindToRules = make(map[string][]v1.Rule)
for _, rule := range policy.Spec.Rules {
if rule.HasMutate() {
for _, kind := range rule.MatchResources.Kinds {
kindToRules[kind] = append(kindToRules[kind], rule)
}
}
}
openApiGlobalState.definitions = make(map[string]*openapi_v2.Schema)
openApiGlobalState.kindToDefinitionName = make(map[string]string)
for _, definition := range openApiGlobalState.document.GetDefinitions().AdditionalProperties {
openApiGlobalState.definitions[definition.GetName()] = definition.GetValue()
for kind, rules := range kindToRules {
newPolicy := *policy.DeepCopy()
newPolicy.Spec.Rules = rules
resource, _ := o.generateEmptyResource(o.definitions[o.kindToDefinitionName[kind]]).(map[string]interface{})
if resource == nil {
log.Log.V(4).Info(fmt.Sprintf("Cannot Validate policy: openApi definition now found for %v", kind))
return nil
}
newResource := unstructured.Unstructured{Object: resource}
newResource.SetKind(kind)
patchedResource, err := engine.ForceMutate(nil, *newPolicy.DeepCopy(), newResource)
if err != nil {
return err
}
err = o.ValidateResource(*patchedResource.DeepCopy(), kind)
if err != nil {
return err
}
}
return nil
}
func (o *Controller) useOpenApiDocument(doc *openapi_v2.Document) error {
o.mutex.Lock()
defer o.mutex.Unlock()
o.definitions = make(map[string]*openapi_v2.Schema)
o.kindToDefinitionName = make(map[string]string)
for _, definition := range doc.GetDefinitions().AdditionalProperties {
o.definitions[definition.GetName()] = definition.GetValue()
path := strings.Split(definition.GetName(), ".")
openApiGlobalState.kindToDefinitionName[path[len(path)-1]] = definition.GetName()
o.kindToDefinitionName[path[len(path)-1]] = definition.GetName()
}
var err error
openApiGlobalState.models, err = proto.NewOpenAPIData(openApiGlobalState.document)
o.models, err = proto.NewOpenAPIData(doc)
if err != nil {
return err
}
@ -155,17 +174,32 @@ func getSchemaDocument() (*openapi_v2.Document, error) {
}
// For crd, we do not store definition in document
func getSchemaFromDefinitions(kind string) (proto.Schema, error) {
func (o *Controller) getCRDSchema(kind string) (proto.Schema, error) {
if kind == "" {
return nil, errors.New("invalid kind")
}
path := proto.NewPath(kind)
return (&proto.Definitions{}).ParseSchema(openApiGlobalState.definitions[kind], &path)
definition := o.definitions[kind]
if definition == nil {
return nil, errors.New("could not find definition")
}
// This was added so crd's can access
// normal definitions from existing schema such as
// `metadata` - this maybe a breaking change.
// Removing this may cause policy validate to stop working
existingDefinitions, _ := o.models.(*proto.Definitions)
return (existingDefinitions).ParseSchema(definition, &path)
}
func generateEmptyResource(kindSchema *openapi_v2.Schema) interface{} {
func (o *Controller) generateEmptyResource(kindSchema *openapi_v2.Schema) interface{} {
types := kindSchema.GetType().GetValue()
if kindSchema.GetXRef() != "" {
return generateEmptyResource(openApiGlobalState.definitions[strings.TrimPrefix(kindSchema.GetXRef(), "#/definitions/")])
return o.generateEmptyResource(o.definitions[strings.TrimPrefix(kindSchema.GetXRef(), "#/definitions/")])
}
if len(types) != 1 {
@ -189,7 +223,7 @@ func generateEmptyResource(kindSchema *openapi_v2.Schema) interface{} {
wg.Add(len(properties))
for _, property := range properties {
go func(property *openapi_v2.NamedSchema) {
prop := generateEmptyResource(property.GetValue())
prop := o.generateEmptyResource(property.GetValue())
mutex.Lock()
props[property.GetName()] = prop
mutex.Unlock()
@ -201,7 +235,7 @@ func generateEmptyResource(kindSchema *openapi_v2.Schema) interface{} {
case "array":
var array []interface{}
for _, schema := range kindSchema.GetItems().GetSchema() {
array = append(array, generateEmptyResource(schema))
array = append(array, o.generateEmptyResource(schema))
}
return array
case "string":

9
pkg/openapi/validation_test.go
View file

@ -47,12 +47,14 @@ func Test_ValidateMutationPolicy(t *testing.T) {
},
}
o, _ := NewOpenAPIController()
for i, tc := range tcs {
policy := v1.ClusterPolicy{}
_ = json.Unmarshal(tc.policy, &policy)
var errMessage string
err := ValidatePolicyMutation(policy)
err := o.ValidatePolicyMutation(policy)
if err != nil {
errMessage = err.Error()
}
@ -63,3 +65,8 @@ func Test_ValidateMutationPolicy(t *testing.T) {
}
}
func Test_addDefaultFieldsToSchema(t *testing.T) {
addingDefaultFieldsToSchema([]byte(`null`))
addingDefaultFieldsToSchema(nil)
}

61
pkg/policy/actions.go Normal file
View file

@ -0,0 +1,61 @@
package policy
import (
"fmt"
kyverno "github.com/nirmata/kyverno/pkg/api/kyverno/v1"
dclient "github.com/nirmata/kyverno/pkg/dclient"
"github.com/nirmata/kyverno/pkg/policy/generate"
"github.com/nirmata/kyverno/pkg/policy/mutate"
"github.com/nirmata/kyverno/pkg/policy/validate"
"sigs.k8s.io/controller-runtime/pkg/log"
)
//Validation provides methods to validate a rule
type Validation interface {
Validate() (string, error)
}
//validateAction performs validation on the rule actions
// - Mutate
// - Validation
// - Generate
func validateActions(idx int, rule kyverno.Rule, client *dclient.Client, mock bool) error {
var checker Validation
// Mutate
if rule.HasMutate() {
checker = mutate.NewMutateFactory(rule.Mutation)
if path, err := checker.Validate(); err != nil {
return fmt.Errorf("path: spec.rules[%d].mutate.%s.: %v", idx, path, err)
}
}
// Validate
if rule.HasValidate() {
checker = validate.NewValidateFactory(rule.Validation)
if path, err := checker.Validate(); err != nil {
return fmt.Errorf("path: spec.rules[%d].validate.%s.: %v", idx, path, err)
}
}
// Generate
if rule.HasGenerate() {
//TODO: this check is there to support offline validations
// generate uses selfSubjectReviews to verify actions
// this need to modified to use different implementation for online and offline mode
if mock {
checker = generate.NewFakeGenerate(rule.Generation)
if path, err := checker.Validate(); err != nil {
return fmt.Errorf("path: spec.rules[%d].generate.%s.: %v", idx, path, err)
}
} else {
checker = generate.NewGenerateFactory(client, rule.Generation, log.Log)
if path, err := checker.Validate(); err != nil {
return fmt.Errorf("path: spec.rules[%d].generate.%s.: %v", idx, path, err)
}
}
}
return nil
}

Some files were not shown because too many files have changed in this diff Show more