Skip to content

Commit 7c3da8d

Browse files
sjbermansjberman
authored
Support NginxProxy CRD and global tracing settings (#1870)
Problem: As a user of NGF I want to set the collection point for my traces for my installation of NGF So that I can ensure all my traces are sent to the same collection platform. Solution: Implement the NginxProxy CRD which contains the fields required to configure the collection point for tracing. This resource is attached to the GatewayClass. If the resource is not found, a condition will be set on the GatewayClass to indicate this. The GatewayClass will continue to be Accepted even if the parametersRef is invalid. This configuration sets the `http` context-level otel directives. The otel module is loaded conditionally based on the existence of this configuration. Note: tracing is not enabled by this configuration, this only sets high level options. #1828 is required to actually enable tracing on a per-route basis.
1 parent c7fd089 commit 7c3da8d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+1793
-61
lines changed

‎build/Dockerfile.nginx

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# syntax=docker/dockerfile:1.6
2-
FROM nginx:1.25.5-alpine
2+
FROM nginx:1.25.5-alpine-otel
33

44
ARG NJS_DIR
55
ARG NGINX_CONF_DIR

‎build/Dockerfile.nginxplus

+1-1
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ RUN --mount=type=secret,id=nginx-repo.crt,dst=/etc/apk/cert.pem,mode=0644 \
1818
addgroup -g 1001 -S nginx \
1919
&& adduser -S -D -H -u 101 -h /var/cache/nginx -s /sbin/nologin -G nginx -g nginx nginx \
2020
&& printf "%s\n" "https://pkgs.nginx.com/plus/${NGINX_PLUS_VERSION}/alpine/v$(grep -E -o '^[0-9]+\.[0-9]+' /etc/alpine-release)/main" >> /etc/apk/repositories \
21-
&& apk add --no-cache nginx-plus nginx-plus-module-njs libcap \
21+
&& apk add --no-cache nginx-plus nginx-plus-module-njs nginx-plus-module-otel libcap \
2222
&& mkdir -p /var/lib/nginx /usr/lib/nginx/modules \
2323
&& setcap 'cap_net_bind_service=+ep' /usr/sbin/nginx \
2424
&& setcap -v 'cap_net_bind_service=+ep' /usr/sbin/nginx \

‎charts/nginx-gateway-fabric/README.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -224,7 +224,7 @@ To uninstall/delete the release `ngf`:
224224
```shell
225225
helm uninstall ngf -n nginx-gateway
226226
kubectl delete ns nginx-gateway
227-
kubectl delete crd nginxgateways.gateway.nginx.org
227+
kubectl delete crd nginxgateways.gateway.nginx.org nginxproxies.gateway.nginx.org
228228
```
229229

230230
These commands remove all the Kubernetes components associated with the release and deletes the release.
@@ -269,6 +269,7 @@ The following tables lists the configurable parameters of the NGINX Gateway Fabr
269269
| `nginx.image.tag` | The tag for the NGINX image. | edge |
270270
| `nginx.image.pullPolicy` | The `imagePullPolicy` for the NGINX image. | Always |
271271
| `nginx.plus` | Is NGINX Plus image being used | false |
272+
| `nginx.config` | The configuration for the data plane that is contained in the NginxProxy resource. | [See nginx.config section](values.yaml) |
272273
| `nginx.usage.secretName` | The namespace/name of the Secret containing the credentials for NGINX Plus usage reporting. | |
273274
| `nginx.usage.serverURL` | The base server URL of the NGINX Plus usage reporting server. | |
274275
| `nginx.usage.clusterName` | The display name of the Kubernetes cluster in the NGINX Plus usage reporting server. | |

‎charts/nginx-gateway-fabric/templates/_helpers.tpl

+8
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,14 @@ Create control plane config name.
3131
{{- printf "%s-config" $name | trunc 63 | trimSuffix "-" }}
3232
{{- end }}
3333

34+
{{/*
35+
Create data plane config name.
36+
*/}}
37+
{{- define "nginx-gateway.proxy-config-name" -}}
38+
{{- $name := default .Release.Name .Values.nameOverride }}
39+
{{- printf "%s-proxy-config" $name | trunc 63 | trimSuffix "-" }}
40+
{{- end }}
41+
3442
{{/*
3543
Create chart name and version as used by the chart label.
3644
*/}}

‎charts/nginx-gateway-fabric/templates/deployment.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,8 @@ spec:
118118
volumeMounts:
119119
- name: nginx-conf
120120
mountPath: /etc/nginx/conf.d
121+
- name: module-includes
122+
mountPath: /etc/nginx/module-includes
121123
- name: nginx-secrets
122124
mountPath: /etc/nginx/secrets
123125
- name: nginx-run
@@ -149,6 +151,8 @@ spec:
149151
volumeMounts:
150152
- name: nginx-conf
151153
mountPath: /etc/nginx/conf.d
154+
- name: module-includes
155+
mountPath: /etc/nginx/module-includes
152156
- name: nginx-secrets
153157
mountPath: /etc/nginx/secrets
154158
- name: nginx-run
@@ -181,6 +185,8 @@ spec:
181185
volumes:
182186
- name: nginx-conf
183187
emptyDir: {}
188+
- name: module-includes
189+
emptyDir: {}
184190
- name: nginx-secrets
185191
emptyDir: {}
186192
- name: nginx-run

‎charts/nginx-gateway-fabric/templates/gatewayclass.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,9 @@ metadata:
66
{{- include "nginx-gateway.labels" . | nindent 4 }}
77
spec:
88
controllerName: {{ .Values.nginxGateway.gatewayControllerName }}
9+
{{- if .Values.nginx.config }}
10+
parametersRef:
11+
group: gateway.nginx.org
12+
kind: NginxProxy
13+
name: {{ include "nginx-gateway.proxy-config-name" . }}
14+
{{- end }}

‎charts/nginx-gateway-fabric/templates/nginxproxy.yaml

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{{- if .Values.nginx.config }}
2+
apiVersion: gateway.nginx.org/v1alpha1
3+
kind: NginxProxy
4+
metadata:
5+
name: {{ include "nginx-gateway.proxy-config-name" . }}
6+
labels:
7+
{{- include "nginx-gateway.labels" . | nindent 4 }}
8+
spec:
9+
{{- toYaml .Values.nginx.config | nindent 2 }}
10+
{{- end }}

‎charts/nginx-gateway-fabric/templates/rbac.yaml

+8-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ metadata:
1010
{{- if or .Values.serviceAccount.imagePullSecret .Values.serviceAccount.imagePullSecrets }}
1111
imagePullSecrets:
1212
{{- if .Values.serviceAccount.imagePullSecret }}
13-
- name: {{ .Values.serviceAccount.imagePullSecret}}
13+
- name: {{ .Values.serviceAccount.imagePullSecret }}
1414
{{- end }}
1515
{{- if .Values.serviceAccount.imagePullSecrets }}
1616
{{- range .Values.serviceAccount.imagePullSecrets }}
@@ -115,6 +115,13 @@ rules:
115115
- get
116116
- list
117117
- watch
118+
- apiGroups:
119+
- gateway.nginx.org
120+
resources:
121+
- nginxproxies
122+
verbs:
123+
- list
124+
- watch
118125
- apiGroups:
119126
- gateway.nginx.org
120127
resources:

‎charts/nginx-gateway-fabric/values.yaml

+11
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,17 @@ nginx:
7070
## Is NGINX Plus image being used
7171
plus: false
7272

73+
## The configuration for the data plane that is contained in the NginxProxy resource.
74+
config: {}
75+
# telemetry:
76+
# exporter:
77+
# endpoint: otel-collector.default.svc:4317
78+
# interval: 5s
79+
# batchSize: 512
80+
# batchCount: 4
81+
# serviceName: ""
82+
# spanAttributes: []
83+
7384
## Configuration for NGINX Plus usage reporting.
7485
usage:
7586
## The namespace/name of the Secret containing the credentials for NGINX Plus usage reporting.

‎conformance/provisioner/static-deployment.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,8 @@ spec:
7070
volumeMounts:
7171
- name: nginx-conf
7272
mountPath: /etc/nginx/conf.d
73+
- name: module-includes
74+
mountPath: /etc/nginx/module-includes
7375
- name: nginx-secrets
7476
mountPath: /etc/nginx/secrets
7577
- name: nginx-run
@@ -94,6 +96,8 @@ spec:
9496
volumeMounts:
9597
- name: nginx-conf
9698
mountPath: /etc/nginx/conf.d
99+
- name: module-includes
100+
mountPath: /etc/nginx/module-includes
97101
- name: nginx-secrets
98102
mountPath: /etc/nginx/secrets
99103
- name: nginx-run
@@ -111,6 +115,8 @@ spec:
111115
volumes:
112116
- name: nginx-conf
113117
emptyDir: {}
118+
- name: module-includes
119+
emptyDir: {}
114120
- name: nginx-secrets
115121
emptyDir: {}
116122
- name: nginx-run

‎deploy/manifests/nginx-gateway-experimental.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,13 @@ rules:
9797
- get
9898
- list
9999
- watch
100+
- apiGroups:
101+
- gateway.nginx.org
102+
resources:
103+
- nginxproxies
104+
verbs:
105+
- list
106+
- watch
100107
- apiGroups:
101108
- gateway.nginx.org
102109
resources:
@@ -213,6 +220,8 @@ spec:
213220
volumeMounts:
214221
- name: nginx-conf
215222
mountPath: /etc/nginx/conf.d
223+
- name: module-includes
224+
mountPath: /etc/nginx/module-includes
216225
- name: nginx-secrets
217226
mountPath: /etc/nginx/secrets
218227
- name: nginx-run
@@ -237,6 +246,8 @@ spec:
237246
volumeMounts:
238247
- name: nginx-conf
239248
mountPath: /etc/nginx/conf.d
249+
- name: module-includes
250+
mountPath: /etc/nginx/module-includes
240251
- name: nginx-secrets
241252
mountPath: /etc/nginx/secrets
242253
- name: nginx-run
@@ -254,6 +265,8 @@ spec:
254265
volumes:
255266
- name: nginx-conf
256267
emptyDir: {}
268+
- name: module-includes
269+
emptyDir: {}
257270
- name: nginx-secrets
258271
emptyDir: {}
259272
- name: nginx-run

‎deploy/manifests/nginx-gateway.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,13 @@ rules:
9494
- get
9595
- list
9696
- watch
97+
- apiGroups:
98+
- gateway.nginx.org
99+
resources:
100+
- nginxproxies
101+
verbs:
102+
- list
103+
- watch
97104
- apiGroups:
98105
- gateway.nginx.org
99106
resources:
@@ -209,6 +216,8 @@ spec:
209216
volumeMounts:
210217
- name: nginx-conf
211218
mountPath: /etc/nginx/conf.d
219+
- name: module-includes
220+
mountPath: /etc/nginx/module-includes
212221
- name: nginx-secrets
213222
mountPath: /etc/nginx/secrets
214223
- name: nginx-run
@@ -233,6 +242,8 @@ spec:
233242
volumeMounts:
234243
- name: nginx-conf
235244
mountPath: /etc/nginx/conf.d
245+
- name: module-includes
246+
mountPath: /etc/nginx/module-includes
236247
- name: nginx-secrets
237248
mountPath: /etc/nginx/secrets
238249
- name: nginx-run
@@ -250,6 +261,8 @@ spec:
250261
volumes:
251262
- name: nginx-conf
252263
emptyDir: {}
264+
- name: module-includes
265+
emptyDir: {}
253266
- name: nginx-secrets
254267
emptyDir: {}
255268
- name: nginx-run

‎deploy/manifests/nginx-plus-gateway-experimental.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,13 @@ rules:
103103
- get
104104
- list
105105
- watch
106+
- apiGroups:
107+
- gateway.nginx.org
108+
resources:
109+
- nginxproxies
110+
verbs:
111+
- list
112+
- watch
106113
- apiGroups:
107114
- gateway.nginx.org
108115
resources:
@@ -220,6 +227,8 @@ spec:
220227
volumeMounts:
221228
- name: nginx-conf
222229
mountPath: /etc/nginx/conf.d
230+
- name: module-includes
231+
mountPath: /etc/nginx/module-includes
223232
- name: nginx-secrets
224233
mountPath: /etc/nginx/secrets
225234
- name: nginx-run
@@ -244,6 +253,8 @@ spec:
244253
volumeMounts:
245254
- name: nginx-conf
246255
mountPath: /etc/nginx/conf.d
256+
- name: module-includes
257+
mountPath: /etc/nginx/module-includes
247258
- name: nginx-secrets
248259
mountPath: /etc/nginx/secrets
249260
- name: nginx-run
@@ -261,6 +272,8 @@ spec:
261272
volumes:
262273
- name: nginx-conf
263274
emptyDir: {}
275+
- name: module-includes
276+
emptyDir: {}
264277
- name: nginx-secrets
265278
emptyDir: {}
266279
- name: nginx-run

‎deploy/manifests/nginx-plus-gateway.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,13 @@ rules:
100100
- get
101101
- list
102102
- watch
103+
- apiGroups:
104+
- gateway.nginx.org
105+
resources:
106+
- nginxproxies
107+
verbs:
108+
- list
109+
- watch
103110
- apiGroups:
104111
- gateway.nginx.org
105112
resources:
@@ -216,6 +223,8 @@ spec:
216223
volumeMounts:
217224
- name: nginx-conf
218225
mountPath: /etc/nginx/conf.d
226+
- name: module-includes
227+
mountPath: /etc/nginx/module-includes
219228
- name: nginx-secrets
220229
mountPath: /etc/nginx/secrets
221230
- name: nginx-run
@@ -240,6 +249,8 @@ spec:
240249
volumeMounts:
241250
- name: nginx-conf
242251
mountPath: /etc/nginx/conf.d
252+
- name: module-includes
253+
mountPath: /etc/nginx/module-includes
243254
- name: nginx-secrets
244255
mountPath: /etc/nginx/secrets
245256
- name: nginx-run
@@ -257,6 +268,8 @@ spec:
257268
volumes:
258269
- name: nginx-conf
259270
emptyDir: {}
271+
- name: module-includes
272+
emptyDir: {}
260273
- name: nginx-secrets
261274
emptyDir: {}
262275
- name: nginx-run

‎docs/proposals/gateway-settings.md

+11-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Enhancement Proposal-1775: Gateway Settings
22

33
- Issue: https://github.com/nginxinc/nginx-gateway-fabric/issues/1775
4-
- Status: Implementable
4+
- Status: Completed
55

66
## Summary
77

@@ -93,7 +93,7 @@ type Telemetry struct {
9393
// SpanAttributes are custom key/value attributes that are added to each span.
9494
//
9595
// +optional
96-
SpanAttributes map[string]string `json:"spanAttributes,omitempty"`
96+
SpanAttributes []SpanAttribute `json:"spanAttributes,omitempty"`
9797
}
9898

9999
// TelemetryExporter specifies OpenTelemetry export parameters.
@@ -122,6 +122,15 @@ type TelemetryExporter struct {
122122
// The format is a subset of the syntax parsed by Golang time.ParseDuration.
123123
// Examples: 1h, 12m, 30s, 150ms.
124124
type Duration string
125+
126+
// SpanAttribute is a key value pair to be added to a tracing span.
127+
type SpanAttribute struct {
128+
// Key is the key for a span attribute.
129+
Key string `json:"key"`
130+
131+
// Value is the value for a span attribute.
132+
Value string `json:"value"`
133+
}
125134
```
126135

127136
### Status

‎internal/mode/static/manager.go

+8
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ func StartManager(cfg config.Config) error {
116116
Logger: cfg.Logger.WithName("changeProcessor"),
117117
Validators: validation.Validators{
118118
HTTPFieldsValidator: ngxvalidation.HTTPValidator{},
119+
GenericValidator: ngxvalidation.GenericValidator{},
119120
},
120121
EventRecorder: recorder,
121122
Scheme: scheme,
@@ -414,6 +415,12 @@ func registerControllers(
414415
),
415416
},
416417
},
418+
{
419+
objectType: &ngfAPI.NginxProxy{},
420+
options: []controller.Option{
421+
controller.WithK8sPredicate(k8spredicate.GenerationChangedPredicate{}),
422+
},
423+
},
417424
}
418425

419426
if cfg.ExperimentalFeatures {
@@ -592,6 +599,7 @@ func prepareFirstEventBatchPreparerArgs(
592599
&discoveryV1.EndpointSliceList{},
593600
&gatewayv1.HTTPRouteList{},
594601
&gatewayv1beta1.ReferenceGrantList{},
602+
&ngfAPI.NginxProxyList{},
595603
partialObjectMetadataList,
596604
}
597605

0 commit comments

Comments
 (0)