1
0
Fork 0
mirror of https://github.com/kyverno/kyverno.git synced 2025-03-13 03:16:55 +00:00
Commit graph

93 commits

Author SHA1 Message Date
Charles-Edouard Brétéché
e0ab72bb9a
feat: reports v2 implementation ()
This PR refactors the reports generation code.
It removes RCR and CRCR crds and replaces them with AdmissionReport, ClusterAdmissionReport, BackgroundScanReport and ClusterBackgroundScanReport crds.

The new reports system is based on 4 controllers:

Admission reports controller is responsible for cleaning up admission reports and attaching admission reports to their corresponding resource in case of a creation
Background scan reports controller is responsible for creating background scan reports when a resource and/or policy changes
Aggregation controller takes care of aggregation per resource reports into higher level reports (per namespace)
Resources controller is responsible for watching reports that need background scan reports
I added two new flags to disable admission reports and/or background scan reports, the whole reporting system can be disabled if something goes wrong.

I also added a flag to split reports in chunks to avoid creating too large resources.

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>
Co-authored-by: prateekpandey14 <prateek.pandey@nirmata.com>
2022-09-28 17:15:16 +05:30
Prateek Pandey
71404df826
refactor: update log based on the policy types ()
Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>
Co-authored-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-09-19 10:56:55 +00:00
Charles-Edouard Brétéché
42a2df56c1
refactor: add a couple of constants in api ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-09-19 09:11:12 +00:00
Charles-Edouard Brétéché
b9baae9746
fix: incorrect namespace in report controller ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Co-authored-by: Prateek Pandey <prateek.pandey@nirmata.com>
2022-09-19 07:33:34 +00:00
Charles-Edouard Brétéché
e516fb868e
fix: lock in policy report mapper ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-09-13 16:30:14 +08:00
Charles-Edouard Brétéché
da5312c177
refactor: simplify RCR creator queue ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-09-12 14:48:27 +00:00
Charles-Edouard Brétéché
7c74e40b86
refactor: info in policyreport package ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
Co-authored-by: shuting <shuting@nirmata.com>
2022-09-12 09:00:56 +00:00
Charles-Edouard Brétéché
6a0fff7c9f
fix: split policy report flag ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Co-authored-by: Prateek Pandey <prateek.pandey@nirmata.com>
2022-09-10 15:29:48 +00:00
Charles-Edouard Brétéché
a95d61b9d7
refactor: client wrappers ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-09-07 12:01:43 +08:00
Charles-Edouard Brétéché
f243a7dd84
refactor: make toggles easier to define and use ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
Co-authored-by: shuting <shuting@nirmata.com>
2022-08-31 06:41:14 +00:00
shuting
3bf3dcc1af
Add the metric "kyverno_client_queries_total" ()
* Add metric "kyverno_kube_client_queries_total"

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* publish metric for missing queries

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Refactor the way Kyverno registers QPS metric

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Move clientsets to a dedicated folder

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Wrap Kyverno client and policyreport client to register client query metric

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* address linter comments

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* address linter comments

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Switch to use wrapper clients

Signed-off-by: ShutingZhao <shuting@nirmata.com>

Signed-off-by: ShutingZhao <shuting@nirmata.com>
Co-authored-by: Vyankatesh Kudtarkar <vyankateshkd@gmail.com>
2022-08-31 11:33:47 +05:30
Charles-Edouard Brétéché
144985ee5a
chore: fix golangcilint timeout ()
* chore: fix golangcilint timeout

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* fix commit sha

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* add .gitattributes

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-08-24 21:08:24 +08:00
Anutosh Bhat
d92e16526f
Added appropriate logging levels to log.Info() calls wherever necessary ()
* Added appropriate logging levels to log.Info() calls wherever necessary

Signed-off-by: anutosh491 <andersonbhat491@gmail.com>

* Changed logging levels to 2

Signed-off-by: anutosh491 <andersonbhat491@gmail.com>

Signed-off-by: anutosh491 <andersonbhat491@gmail.com>
Co-authored-by: shuting <shuting@nirmata.com>
2022-08-18 13:24:59 +00:00
Jim Bugwadia
943c3a1929
use failurePolicy to block or allow requests, on policy errors ()
* use failurePolicy to block or allow requests, on policy errors

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* add warnings

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* codegen

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix linter issues

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* add unit tests

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* handle network errors

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix linter issues

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix test

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix title conversion

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix path in generated file

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix test

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix fake metrics

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix tests

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* add check for klog flag initialization

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* check for flag reinitialization

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* check for flag reinitialization

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix spelling

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix flag init

Signed-off-by: Jim Bugwadia <jim@nirmata.com>
2022-08-02 20:24:02 +05:30
Prateek Pandey
3f1997c0e8
fix split policyreport name with background scan ()
- fix split policyreport name with background scan
- fix the label selector initialising
- refactor the generatePolicyName func

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>
2022-07-21 14:31:42 +05:30
Prateek Pandey
808e6ae8b7
delete policy reports on policy deletion ()
Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>
2022-06-30 15:45:02 +00:00
Prateek Pandey
9226873e68
feat: split policy report per policy bases ()
* feat: split policy report per policy bases

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

* add policy name as a handler key

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

* update merge change request logic

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

* handle the delete resource update on policy report

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

* add splitPolicyReport feature gate

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

* delete old reports if splitPolicyReport feature enable

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

* use trim policyname as label and create name

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>

* fix change request result

Signed-off-by: prateekpandey14 <prateek.pandey@nirmata.com>
2022-06-28 15:27:57 +00:00
shuting
77fb10a430
Clean up RCRs if the count exceeds the threshold ()
* Clean up RCRs if the count exceeds the limit

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Sets reports to inactive on resourceExhausted error

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* fix linter

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Add a container flag changeRequestLimit

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Skip generating RCRs if resourceExhausted error occurs

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* set default RCR limit to 1000

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Update log messages and CHANGELOG.md

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Address review comments

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* Extract mapper to a separate file

Signed-off-by: ShutingZhao <shuting@nirmata.com>
2022-06-28 06:18:57 +00:00
shuting
cd2d89bf55
Wait for informers' cache to be synced before starting controllers ()
Signed-off-by: ShutingZhao <shuting@nirmata.com>
2022-06-28 04:55:52 +00:00
Charles-Edouard Brétéché
41a3f6c388
chore: make kyverno informers and listers import aliases consistent ()
* chore: make kyverno api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make apimachinery api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make dclient api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make clients import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make kube informers and listers import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make kyverno informers and listers import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Co-authored-by: shuting <shuting@nirmata.com>
2022-05-18 04:02:31 +00:00
Charles-Edouard Brétéché
572a76ce33
chore: make kube informers and listers import aliases consistent ()
* chore: make kyverno api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make apimachinery api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make dclient api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make clients import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make kube informers and listers import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-17 17:51:03 +02:00
Charles-Edouard Brétéché
a6d4cd186d
chore: make clients import aliases consistent ()
* chore: make kyverno api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make apimachinery api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make dclient api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make clients import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-17 15:26:32 +00:00
Charles-Edouard Brétéché
666bcb3c15
chore: make k8s api import aliases consistent ()
* chore: make kyverno api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* chore: make apimachinery api import aliases consistent

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-17 22:14:31 +08:00
Charles-Edouard Brétéché
5aaf2d8770
chore: make kyverno api import aliases consistent ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-17 13:12:43 +02:00
Charles-Edouard Brétéché
0099ef54ad
chore: enable gofmt and gofumpt linters ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-17 06:19:03 +00:00
Charles-Edouard Brétéché
26a5402747
chore: enable wastedassign linter ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-12 11:03:26 +00:00
Charles-Edouard Brétéché
2064a69b8a
refactor: make config vars private ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-11 06:14:30 +00:00
Charles-Edouard Brétéché
97e5e64fd4
chore: enable whitespace linter ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Co-authored-by: Sambhav Kothari <sambhavs.email@gmail.com>
2022-05-10 17:01:29 +00:00
Moritz Johner
4d2ec26c90
CLI should respect scored annotation for warnings ()
Co-authored-by: Sambhav Kothari <skothari44@bloomberg.net>
2022-05-07 13:33:50 +00:00
Jim Bugwadia
76608e315e
handle duplicate images; use container name as key ()
* handle duplicate images; use container name as key

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* use OldObject for modify requests

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* use unique image names

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* merge main

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* create a single annotation patch across rules and images

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fmt and change annotation key name

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

* fix linter issues

Signed-off-by: Jim Bugwadia <jim@nirmata.com>

Co-authored-by: Vyankatesh Kudtarkar <vyankateshkd@gmail.com>
Co-authored-by: Sambhav Kothari <sambhavs.email@gmail.com>
2022-05-05 14:06:18 -07:00
Charles-Edouard Brétéché
32789d1c0d
fix: add missing tombstone calls ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Co-authored-by: shuting <shuting@nirmata.com>
2022-05-03 16:58:20 +00:00
Charles-Edouard Brétéché
c79223393b
refactor: dclient package ()
* refactor: replace clientset by inteface

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* refactor: dclient package

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-03 13:30:04 +08:00
Charles-Edouard Brétéché
6e07acdd87
refactor: replace clientset by inteface ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-02 20:30:07 +00:00
Charles-Edouard Brétéché
18af55ed49
refactor: wait for cache sync ()
Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-05-03 01:41:39 +08:00
Charles-Edouard Brétéché
68c35b2f2e
refactor: remove some api unnecessary pointers (2) ()
* refactor: remove some api unnecessary pointers

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* refactor: remove some api unnecessary pointers (2)

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-04-28 17:11:14 +08:00
Charles-Edouard Brétéché
c8275b7c00
refactor: make response type (RuleType) typed ()
* refactor: move common utils

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* refactor: make response type (RuleType) typed

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

* fix: merge

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>

Co-authored-by: shuting <shuting@nirmata.com>
Co-authored-by: Vyankatesh Kudtarkar <vyankateshkd@gmail.com>
2022-04-06 19:04:08 +00:00
Charles-Edouard Brétéché
83343697b9
refactor: make use of policy interface ()
- refactor: make use of policy interface

Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
2022-03-30 18:34:30 +05:30
shuting
a30493e550
Fix policy report OwnerReference ()
* add namespaces/finalizers to clusterrole kyverno:generate

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* set policy report's owner to Kyverno namespace

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* address comments

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* address comments

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* remove BlockOwnerDeletion

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* remove namespaces/finalizers permission

Signed-off-by: ShutingZhao <shuting@nirmata.com>

Co-authored-by: Jim Bugwadia <jim@nirmata.com>
2022-02-18 00:50:18 -08:00
shuting
2eb9660aee
Reduce throttling requests for Kyverno resources ()
* remove resoureCache from the event controller

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* create rcr using typed client to reduce PUT throttling request

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* use typed client for report/rcr operations

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* clarify naming patterns for Kyverno ClusterRoles/ClusterRoleBindings  ()

* clarify naming patterns for Kyverno ClusterRoles/ClusterRoleBindings ()

* fix comment

* fix comment

Co-authored-by: Vyankatesh Kudtarkar <vyankateshkd@gmail.com>
2022-01-22 05:36:42 +00:00
shuting
376a8d3b22
Reduce throttling requests for Kyverno managed resources ()
* remove resoureCache from the event controller

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* create rcr using typed client to reduce PUT throttling request

Signed-off-by: ShutingZhao <shuting@nirmata.com>
2022-01-21 18:36:05 +08:00
Kumar Mallikarjuna
3f61e2dd3a
Added report generation for verifyImage rules ()
* Add report generation for verifyImage rules

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Add flag comment

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Mutation: handleDelete()

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Remove redundant delete

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Test validation failure

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Validation force rules test

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Default validation behaviour

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Manual rules

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Update Config Manager

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

* Move Delete check

Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com>

Co-authored-by: Jim Bugwadia <jim@nirmata.com>
2022-01-05 07:07:44 +00:00
shuting
045a58e2ef
Don't create ReportChangeRequest on managed pods/jobs deletion ()
* don't generate policy report on managed pod/job

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* - don't generate rcr for managed pods/jobs; - add debug info

Signed-off-by: ShutingZhao <shuting@nirmata.com>

* address linter issues

Signed-off-by: ShutingZhao <shuting@nirmata.com>

Co-authored-by: Vyankatesh Kudtarkar <vyankateshkd@gmail.com>
2022-01-03 15:43:38 +00:00
shuting
2c9319ea87
don't generate policy report on managed pod/job ()
Signed-off-by: ShutingZhao <shuting@nirmata.com>
2021-12-30 00:34:43 +08:00
Frank Jogeleit
abb5bd2947
Add SelectorLabel to (Cluster)PolicyReporter resources ()
Signed-off-by: Frank Jogeleit <frank.jogeleit@lovoo.com>

Co-authored-by: shuting <shutting06@gmail.com>
2021-12-17 05:03:52 +00:00
Marcus Noble
8690f8b142
Handle reports with missing result property ()
* Handle reports with missing result property

Signed-off-by: Marcus Noble <github@marcusnoble.co.uk>

* Make use of type structs

Signed-off-by: Marcus Noble <github@marcusnoble.co.uk>

* Fix import

Signed-off-by: Marcus Noble <github@marcusnoble.co.uk>

* Fix cast from map to struct

Signed-off-by: Marcus Noble <github@marcusnoble.co.uk>
2021-11-09 12:03:15 +01:00
shuting
0f0c070072
Fix memory issue - RCR conversion () 2021-11-08 15:53:21 -08:00
Jose Armesto
831a9826d1
Restructure project to follow standards ()
Signed-off-by: Jose Armesto <github@armesto.net>
2021-10-29 18:13:20 +02:00
Marcus Noble
1966c82c6d
Fix various go lint issues ()
* Fix various go lint issues

Signed-off-by: Marcus Noble <github@marcusnoble.co.uk>

* Fix if mistake

Signed-off-by: Marcus Noble <github@marcusnoble.co.uk>

* Simplified returns

Signed-off-by: Marcus Noble <github@marcusnoble.co.uk>
2021-10-29 17:06:03 +02:00
shuting
4835157cc4
fix policy report reconciliation on resource/policy deletion () 2021-10-27 22:59:59 -07:00
Bricktop
3f15ec5a1e
Remove dead code and unused variables ()
* Remove dead code and unused variables

Signed-off-by: Marcel Mueller <marcel.mueller1@rwth-aachen.de>

* Remove unnecessary definitions

Signed-off-by: Marcel Mueller <marcel.mueller1@rwth-aachen.de>
2021-10-13 22:45:23 +02:00