Skip to content

Commit 8d618e4

Browse files
chen-keinanchen-keinan
authored
feat(k8s)!: node-collector dynamic commands support (#6861)
Signed-off-by: chenk <[email protected]>
1 parent a76e328 commit 8d618e4

File tree

18 files changed

+353
-69
lines changed

18 files changed

+353
-69
lines changed

‎docs/docs/compliance/compliance.md

Lines changed: 223 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,231 @@ to specify a built-in compliance report, select it by ID like `trivy --complianc
3535
For the list of built-in compliance reports, please see the relevant section:
3636

3737
- [Docker compliance](../target/container_image.md#compliance)
38-
- [Kubernetes compliance](../target/kubernetes.md#compliance)
38+
- [Kubernetes compliance](../target/kubernetes.md#compliance)
3939
- [AWS compliance](../target/aws.md#compliance)
4040

41+
## Contribute a Built-in Compliance Report
42+
43+
### Define a Compliance spec, based on CIS benchmark or other specs
44+
45+
Here is an example for CIS compliance report:
46+
47+
```yaml
48+
---
49+
spec:
50+
id: k8s-cis-1.23
51+
title: CIS Kubernetes Benchmarks v1.23
52+
description: CIS Kubernetes Benchmarks
53+
platform: k8s
54+
type: cis
55+
version: '1.23'
56+
relatedResources:
57+
- https://www.cisecurity.org/benchmark/kubernetes
58+
controls:
59+
- id: 1.1.1
60+
name: Ensure that the API server pod specification file permissions are set to
61+
600 or more restrictive
62+
description: Ensure that the API server pod specification file has permissions
63+
of 600 or more restrictive
64+
checks:
65+
- id: AVD-KCV-0073
66+
commands:
67+
- id: CMD-0001
68+
severity: HIGH
69+
70+
```
71+
72+
### Compliance ID
73+
74+
ID field is the name used to execute the compliance scan via trivy
75+
example:
76+
77+
```sh
78+
trivy k8s --compliance k8s-cis-1.23
79+
```
80+
81+
ID naming convention: {platform}-{type}-{version}
82+
83+
### Compliance Platform
84+
85+
The platform field specifies the type of platform on which to run this compliance report.
86+
Supported platforms:
87+
88+
- k8s (native kubernetes cluster)
89+
- eks (elastic kubernetes service)
90+
- aks (azure kubernetes service)
91+
- gke (google kubernetes engine)
92+
- rke2 (rancher kubernetes engine v2)
93+
- ocp (OpenShift Container Platform)
94+
- docker (docker engine)
95+
- aws (amazon web services)
96+
97+
### Compliance Type
98+
99+
The type field specifies the kind compliance report.
100+
101+
- cis (Center for Internet Security)
102+
- nsa (National Security Agency)
103+
- pss (Pod Security Standards)
104+
105+
### Compliance Version
106+
107+
The version field specifies the version of the compliance report.
108+
109+
- 1.23
110+
111+
### Compliance Check ID
112+
113+
Specify the check ID that needs to be evaluated based on the information collected from the command data output to assess the control.
114+
115+
Example of how to define check data under [checks folder](https://github.com/aquasecurity/trivy-checks/tree/main/checks):
116+
117+
```sh
118+
# METADATA
119+
# title: "Ensure that the --kubeconfig kubelet.conf file permissions are set to 600 or more restrictive"
120+
# description: "Ensure that the kubelet.conf file has permissions of 600 or more restrictive."
121+
# scope: package
122+
# schemas:
123+
# - input: schema["kubernetes"]
124+
# related_resources:
125+
# - https://www.cisecurity.org/benchmark/kubernetes
126+
# custom:
127+
# id: KCV0073
128+
# avd_id: AVD-KCV-0073
129+
# severity: HIGH
130+
# short_code: ensure-kubelet.conf-file-permissions-600-or-more-restrictive.
131+
# recommended_action: "Change the kubelet.conf file permissions to 600 or more restrictive if exist"
132+
# input:
133+
# selector:
134+
# - type: kubernetes
135+
package builtin.kubernetes.KCV0073
136+
137+
import data.lib.kubernetes
138+
139+
types := ["master", "worker"]
140+
141+
validate_kubelet_file_permission(sp) := {"kubeletConfFilePermissions": violation} {
142+
sp.kind == "NodeInfo"
143+
sp.type == types[_]
144+
violation := {permission | permission = sp.info.kubeletConfFilePermissions.values[_]; permission > 600}
145+
count(violation) > 0
146+
}
147+
148+
deny[res] {
149+
output := validate_kubelet_file_permission(input)
150+
msg := "Ensure that the --kubeconfig kubelet.conf file permissions are set to 600 or more restrictive"
151+
res := result.new(msg, output)
152+
}
153+
```
154+
155+
### Compliance Command ID
156+
157+
***Note:*** This field is not mandatory, it is relevant to k8s compliance report when node-collector is in use
158+
159+
Specify the command ID (#ref) that needs to be executed to collect the information required to evaluate the control.
160+
161+
Example of how to define command data under [commands folder](https://github.com/aquasecurity/trivy-checks/tree/main/commands)
162+
163+
```yaml
164+
---
165+
- id: CMD-0001
166+
key: kubeletConfFilePermissions
167+
title: kubelet.conf file permissions
168+
nodeType: worker
169+
audit: stat -c %a $kubelet.kubeconfig
170+
platfroms:
171+
- k8s
172+
- aks
173+
```
174+
175+
#### Command ID
176+
177+
Find the next command ID by running the command on [trivy-checks project](https://github.com/aquasecurity/trivy-checks).
178+
179+
```sh
180+
make command-id
181+
```
182+
183+
#### Command Key
184+
185+
- Re-use an existing key or specifiy a new one (make sure key name has no spaces)
186+
187+
Note: The key value should match the key name evaluated by the Rego check.
188+
189+
### Command Title
190+
191+
Represent the purpose of the command
192+
193+
### Command NodeType
194+
195+
Specify the node type on which the command is supposed to run.
196+
197+
- worker
198+
- master
199+
200+
### Command Audit
201+
202+
Specify here the shell command to be used please make sure to add error supression (2>/dev/null)
203+
204+
### Command Platforms
205+
206+
The list of platforms that support this command. Name should be taken from this list [Platforms](#compliance-platform)
207+
208+
### Command Config Files
209+
210+
The commands use a configuration file that helps obtain the paths to binaries and configuration files based on different platforms (e.g., Rancher, native Kubernetes, etc.).
211+
212+
For example:
213+
214+
```yaml
215+
kubelet:
216+
bins:
217+
- kubelet
218+
- hyperkube kubelet
219+
confs:
220+
- /etc/kubernetes/kubelet-config.yaml
221+
- /var/lib/kubelet/config.yaml
222+
```
223+
224+
### Commands Files Location
225+
226+
Currently checks files location are :`https://github.com/aquasecurity/trivy-checks/tree/main/checks`
227+
228+
Command files location: `https://github.com/aquasecurity/trivy-checks/tree/main/commands`
229+
under command file
230+
231+
Note: command config files will be located under `https://github.com/aquasecurity/trivy-checks/tree/main/commands` as well
232+
233+
### Node-collector output
234+
235+
The node collector will read commands and execute each command, and incorporate the output into the NodeInfo resource.
236+
237+
example:
238+
239+
```json
240+
{
241+
"apiVersion": "v1",
242+
"kind": "NodeInfo",
243+
"metadata": {
244+
"creationTimestamp": "2023-01-04T11:37:11+02:00"
245+
},
246+
"type": "master",
247+
"info": {
248+
"adminConfFileOwnership": {
249+
"values": [
250+
"root:root"
251+
]
252+
},
253+
"adminConfFilePermissions": {
254+
"values": [
255+
600
256+
]
257+
}
258+
...
259+
}
260+
}
261+
```
262+
41263
## Custom compliance
42264

43265
You can create your own custom compliance report. A compliance report is a simple YAML document in the following format:

‎docs/docs/references/configuration/cli/trivy_image.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ trivy image [flags] IMAGE_NAME
3838
--cache-ttl duration cache TTL when using redis as cache backend
3939
--check-namespaces strings Rego namespaces
4040
--checks-bundle-repository string OCI registry URL to retrieve checks bundle from (default "ghcr.io/aquasecurity/trivy-checks:0")
41-
--compliance string compliance report to generate (docker-cis)
41+
--compliance string compliance report to generate (docker-cis-1.6.0)
4242
--config-check strings specify the paths to the Rego check files or to the directories containing them, applying config files
4343
--config-data strings specify paths from which data for the Rego checks will be recursively loaded
4444
--custom-headers strings custom headers in client mode

‎docs/docs/references/configuration/cli/trivy_kubernetes.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ trivy kubernetes [flags] [CONTEXT]
3434
--cache-ttl duration cache TTL when using redis as cache backend
3535
--check-namespaces strings Rego namespaces
3636
--checks-bundle-repository string OCI registry URL to retrieve checks bundle from (default "ghcr.io/aquasecurity/trivy-checks:0")
37-
--compliance string compliance report to generate (k8s-nsa,k8s-cis,k8s-pss-baseline,k8s-pss-restricted)
37+
--compliance string compliance report to generate (k8s-nsa-1.0,k8s-cis-1.23,eks-cis-1.4,rke2-cis-1.24,k8s-pss-baseline-0.1,k8s-pss-restricted-0.1)
3838
--config-check strings specify the paths to the Rego check files or to the directories containing them, applying config files
3939
--config-data strings specify paths from which data for the Rego checks will be recursively loaded
4040
--db-repository string OCI repository to retrieve trivy-db from (default "ghcr.io/aquasecurity/trivy-db:2")
@@ -71,7 +71,7 @@ trivy kubernetes [flags] [CONTEXT]
7171
--list-all-pkgs output all packages in the JSON report regardless of vulnerability
7272
--misconfig-scanners strings comma-separated list of misconfig scanners to use for misconfiguration scanning (default [azure-arm,cloudformation,dockerfile,helm,kubernetes,terraform,terraformplan-json,terraformplan-snapshot])
7373
--no-progress suppress progress bar
74-
--node-collector-imageref string indicate the image reference for the node-collector scan job (default "ghcr.io/aquasecurity/node-collector:0.2.1")
74+
--node-collector-imageref string indicate the image reference for the node-collector scan job (default "ghcr.io/aquasecurity/node-collector:0.3.1")
7575
--node-collector-namespace string specify the namespace in which the node-collector job should be deployed (default "trivy-temp")
7676
--offline-scan do not issue API requests to identify dependencies
7777
-o, --output string output file name

‎docs/docs/target/container_image.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -436,14 +436,14 @@ The following reports are available out of the box:
436436

437437
| Compliance | Version | Name for command | More info |
438438
|----------------------------------------|---------|------------------|---------------------------------------------------------------------------------------------|
439-
| CIS Docker Community Edition Benchmark | 1.1.0 | `docker-cis` | [Link](https://www.aquasec.com/cloud-native-academy/docker-container/docker-cis-benchmark/) |
439+
| CIS Docker Community Edition Benchmark | 1.1.0 | `docker-cis-1.6.0` | [Link](https://www.aquasec.com/cloud-native-academy/docker-container/docker-cis-benchmark/) |
440440

441441
### Examples
442442

443443
Scan a container image configuration and generate a compliance summary report:
444444

445445
```
446-
$ trivy image --compliance docker-cis [YOUR_IMAGE_NAME]
446+
trivy image --compliance docker-cis-1.6.0 [YOUR_IMAGE_NAME]
447447
```
448448

449449
!!! note

‎docs/docs/target/kubernetes.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -355,12 +355,14 @@ For an overview of Trivy's Compliance feature, including working with custom com
355355
356356
The following reports are available out of the box:
357357
358-
| Compliance | Name for command | More info |
359-
|----------------------------------------------|----------------------|---------------------------------------------------------------------------------------------------------------------|
360-
| NSA, CISA Kubernetes Hardening Guidance v1.2 | `k8s-nsa` | [Link](https://media.defense.gov/2022/Aug/29/2003066362/-1/-1/0/CTR_KUBERNETES_HARDENING_GUIDANCE_1.2_20220829.PDF) |
361-
| CIS Benchmark for Kubernetes v1.23 | `k8s-cis` | [Link](https://www.cisecurity.org/benchmark/kubernetes) |
362-
| Pod Security Standards, Baseline | `k8s-pss-baseline` | [Link](https://kubernetes.io/docs/concepts/security/pod-security-standards/#baseline) |
363-
| Pod Security Standards, Restricted | `k8s-pss-restricted` | [Link](https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted) |
358+
| Compliance | Name for command | More info |
359+
|----------------------------------------------|--------------------------|---------------------------------------------------------------------------------------------------------------------|
360+
| NSA, CISA Kubernetes Hardening Guidance v1.0 | `k8s-nsa-1.0` | [Link](https://media.defense.gov/2022/Aug/29/2003066362/-1/-1/0/CTR_KUBERNETES_HARDENING_GUIDANCE_1.2_20220829.PDF) |
361+
| CIS Benchmark for Kubernetes v1.23 | `k8s-cis-1.23` | [Link](https://www.cisecurity.org/benchmark/kubernetes) |
362+
| CIS Benchmark for RKE2 v1.24 | `rke2-cis-1.24` | [Link](https://www.cisecurity.org/benchmark/kubernetes) |
363+
| CIS Benchmark for EKS v1.4 | `eks-cis-1.4` | [Link](https://www.cisecurity.org/benchmark/kubernetes) |
364+
| Pod Security Standards, Baseline | `k8s-pss-baseline-0.1` | [Link](https://kubernetes.io/docs/concepts/security/pod-security-standards/#baseline) |
365+
| Pod Security Standards, Restricted | `k8s-pss-restricted-0.1` | [Link](https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted) |
364366
365367
Examples:
366368
@@ -376,23 +378,23 @@ Get the detailed report for checks:
376378
377379
```
378380
379-
trivy k8s --compliance=k8s-cis --report all
381+
trivy k8s --compliance=k8s-cis-1.23 --report all
380382
381383
```
382384
383385
Get summary report in JSON format:
384386
385387
```
386388
387-
trivy k8s --compliance=k8s-cis --report summary --format json
389+
trivy k8s --compliance=k8s-cis-1.23 --report summary --format json
388390
389391
```
390392
391393
Get detailed report in JSON format:
392394
393395
```
394396
395-
trivy k8s --compliance=k8s-cis --report all --format json
397+
trivy k8s --compliance=k8s-cis-1.23 --report all --format json
396398
397399
```
398400

‎go.mod

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,10 @@ require (
2525
github.com/aquasecurity/table v1.8.0
2626
github.com/aquasecurity/testdocker v0.0.0-20240613070307-2c3868d658ac
2727
github.com/aquasecurity/tml v0.6.1
28-
github.com/aquasecurity/trivy-checks v0.11.0
28+
github.com/aquasecurity/trivy-checks v0.13.0
2929
github.com/aquasecurity/trivy-db v0.0.0-20231005141211-4fc651f7ac8d
3030
github.com/aquasecurity/trivy-java-db v0.0.0-20240109071736-184bd7481d48
31-
github.com/aquasecurity/trivy-kubernetes v0.6.7-0.20240516051533-4c5a4aad13b7
31+
github.com/aquasecurity/trivy-kubernetes v0.6.7-0.20240625102549-87c0f9c7bcf4
3232
github.com/aws/aws-sdk-go-v2 v1.27.2
3333
github.com/aws/aws-sdk-go-v2/config v1.27.18
3434
github.com/aws/aws-sdk-go-v2/credentials v1.17.18
@@ -167,7 +167,7 @@ require (
167167
github.com/antchfx/xpath v1.3.0 // indirect
168168
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
169169
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
170-
github.com/aws/aws-sdk-go v1.53.0 // indirect
170+
github.com/aws/aws-sdk-go v1.53.16 // indirect
171171
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.5 // indirect
172172
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.9 // indirect
173173
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.9 // indirect
@@ -205,6 +205,7 @@ require (
205205
github.com/docker/go-metrics v0.0.1 // indirect
206206
github.com/docker/go-units v0.5.0 // indirect
207207
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 // indirect
208+
github.com/dsnet/compress v0.0.1 // indirect
208209
github.com/dustin/go-humanize v1.0.1 // indirect
209210
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
210211
github.com/emirpasic/gods v1.18.1 // indirect
@@ -369,12 +370,12 @@ require (
369370
k8s.io/apiextensions-apiserver v0.30.0 // indirect
370371
k8s.io/apimachinery v0.30.1 // indirect
371372
k8s.io/apiserver v0.30.0 // indirect
372-
k8s.io/cli-runtime v0.30.0 // indirect
373-
k8s.io/client-go v0.30.0 // indirect
374-
k8s.io/component-base v0.30.0 // indirect
373+
k8s.io/cli-runtime v0.30.1 // indirect
374+
k8s.io/client-go v0.30.1 // indirect
375+
k8s.io/component-base v0.30.1 // indirect
375376
k8s.io/klog/v2 v2.120.1 // indirect
376377
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
377-
k8s.io/kubectl v0.30.0 // indirect
378+
k8s.io/kubectl v0.30.1 // indirect
378379
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
379380
modernc.org/libc v1.50.9 // indirect
380381
modernc.org/mathutil v1.6.0 // indirect

0 commit comments

Comments
 (0)