Skip to content

feat(conformance): Add HTTPRouteMultipleGatewaysDifferentPools test #838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 70 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
87d1756
copy of accepted inference pool test to start from.
SinaChavoshi May 14, 2025
a9dcb62
add yaml file for the test
SinaChavoshi May 14, 2025
c36e5f8
update time out
SinaChavoshi May 14, 2025
aaf23e2
update the yaml file to add port 9002
SinaChavoshi May 14, 2025
2a67e7d
read timeout config from local repo
SinaChavoshi May 14, 2025
8ac7d72
remove excess comments
SinaChavoshi May 14, 2025
c859896
copy from existing test to start with
SinaChavoshi May 14, 2025
fe540a7
update test and add gateway to main manifest
SinaChavoshi May 15, 2025
63fedab
fix spelling error
SinaChavoshi May 19, 2025
66f325d
correct spelling for scenarios
SinaChavoshi May 20, 2025
fd0fa10
check route condition on RouteConditionResolvedRefs
SinaChavoshi May 20, 2025
68812d7
check RouteConditionResolvedRefs instead of Reconciled
SinaChavoshi May 20, 2025
59b1f9c
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi May 27, 2025
8a078dc
remove empty lines in yaml
SinaChavoshi May 27, 2025
c55a803
set optional/defaulted fields as unspecified
SinaChavoshi May 27, 2025
5efaeb2
add new line at the end of yaml files.
SinaChavoshi May 27, 2025
2cd8dab
delete blank lines in yaml files.
SinaChavoshi May 27, 2025
26704a1
change variable names to use primary / secondary
SinaChavoshi May 27, 2025
bb62da7
add to do for name change across the project.
SinaChavoshi May 27, 2025
796b584
fix comment
SinaChavoshi May 27, 2025
403b588
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 27, 2025
6506d44
change all variable names to use primary/secondary consistantly
SinaChavoshi May 27, 2025
dafd21e
add to do for http route testing
SinaChavoshi May 27, 2025
ecf11e4
HTTPRouteMustBeAcceptedAndResolved
SinaChavoshi May 27, 2025
f1ce162
add helper method.
SinaChavoshi May 27, 2025
f3d8e8c
using helper method for pool route accepted.
SinaChavoshi May 28, 2025
1cd8790
combline pool and route accpetnace
SinaChavoshi May 28, 2025
44b1427
add http route validation
SinaChavoshi May 28, 2025
c6bacb4
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi May 28, 2025
d15c25e
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 28, 2025
1451a90
remove extra comments and yaml fields.
SinaChavoshi May 28, 2025
271b7a8
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi May 28, 2025
adcc104
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 28, 2025
1535ccc
remove duplicate method
SinaChavoshi May 28, 2025
c92a402
fix timeout
SinaChavoshi May 28, 2025
ff25051
fix timeout
SinaChavoshi May 28, 2025
bb257fd
fix boilerplate header
SinaChavoshi May 28, 2025
5a61a70
change varialbe names to use primary secondary consistently.
SinaChavoshi May 28, 2025
6fa352d
remove extra comments
SinaChavoshi May 28, 2025
5f1bcad
factor out common code
SinaChavoshi May 28, 2025
e5fc860
Add actual http traffic validation using echo-basic
SinaChavoshi May 28, 2025
62791b5
remove extra comments from manifest
SinaChavoshi May 28, 2025
2f08131
remove modifiedTimeoutConfig.HTTPRouteMustHaveCondition per review co…
SinaChavoshi May 28, 2025
cf587a6
remove modifiedTimeoutConfig.HTTPRouteMustHaveCondition per review co…
SinaChavoshi May 28, 2025
afc8bc3
remove excessive logging from conformance.go
SinaChavoshi May 28, 2025
8588183
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
8b6a873
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
efe3990
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
b63459d
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
b903d8f
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
2d52812
Remove unecessary logs for accpeted reason.
SinaChavoshi May 28, 2025
fc2a410
Merge remote-tracking branch 'upstream/main'
SinaChavoshi May 29, 2025
939ae57
Remove all comments from yaml and go files and fix yaml
SinaChavoshi May 29, 2025
d34457a
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 5, 2025
753917a
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 6, 2025
f4bcc9a
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi Jun 6, 2025
d5530dc
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 6, 2025
0ce459c
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi Jun 6, 2025
3b884d6
intermediate update
SinaChavoshi Jun 6, 2025
d90eae0
fix the test run
SinaChavoshi Jun 6, 2025
73731e3
factor out common code
SinaChavoshi Jun 6, 2025
b57fb8a
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 9, 2025
6ee73d5
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi Jun 9, 2025
b239930
move epp def to shared manifest
SinaChavoshi Jun 9, 2025
66b8426
remove extra comments
SinaChavoshi Jun 9, 2025
9432280
revert back to two epps
SinaChavoshi Jun 9, 2025
4b1f44a
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi Jun 9, 2025
450a5c5
Merge branch 'inf_pool_tests_11' into inf_pool_tests_12
SinaChavoshi Jun 9, 2025
5990c51
update to use epp filter
SinaChavoshi Jun 9, 2025
392171d
remove extra comments
SinaChavoshi Jun 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions conformance/conformance.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ import (
"sigs.k8s.io/yaml"

// Import necessary types and utilities from the core Gateway API conformance suite.
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" // Import core Gateway API types
confapis "sigs.k8s.io/gateway-api/conformance/apis/v1" // Report struct definition
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
confapis "sigs.k8s.io/gateway-api/conformance/apis/v1"
confconfig "sigs.k8s.io/gateway-api/conformance/utils/config"
confflags "sigs.k8s.io/gateway-api/conformance/utils/flags"
apikubernetes "sigs.k8s.io/gateway-api/conformance/utils/kubernetes"
Expand All @@ -68,8 +68,9 @@ import (

// Constants for the shared Gateway
const (
SharedGatewayName = "conformance-gateway" // Name of the Gateway in manifests.yaml
SharedGatewayNamespace = "gateway-conformance-infra" // Namespace of the Gateway
PrimaryGatewayName = "conformance-gateway"
SharedGatewayNamespace = "gateway-conformance-infra"
SecondaryGatewayName = "conformance-secondary-gateway"
)

// GatewayLayerProfileName defines the name for the conformance profile that tests
Expand Down Expand Up @@ -221,10 +222,13 @@ func RunConformanceWithOptions(t *testing.T, opts confsuite.ConformanceOptions)

cSuite.Setup(t, tests.ConformanceTests)

sharedGwNN := types.NamespacedName{Name: SharedGatewayName, Namespace: SharedGatewayNamespace}
sharedGwNN := types.NamespacedName{Name: PrimaryGatewayName, Namespace: SharedGatewayNamespace}
secondaryGwNN := types.NamespacedName{Name: SecondaryGatewayName, Namespace: SharedGatewayNamespace}

// Validate Gateway setup.
// Validate Gateway setup for both Gateways.
ensureGatewayAvailableAndReady(t, cSuite.Client, opts, sharedGwNN)
ensureGatewayAvailableAndReady(t, cSuite.Client, opts, secondaryGwNN)

t.Log("Running Inference Extension conformance tests against all registered tests")
err = cSuite.Run(t, tests.ConformanceTests)
require.NoError(t, err, "error running conformance tests")
Expand Down
104 changes: 90 additions & 14 deletions conformance/resources/manifests/manifests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,57 +4,45 @@
# individual test files or other resource directories (e.g., sample_backends).

---
# Namespace for core infrastructure like Gateways.
apiVersion: v1
kind: Namespace
metadata:
name: gateway-conformance-infra
labels:
gateway-conformance: infra
---
# Namespace for application backends (potentially simulating model servers
# or where InferencePools might reside in some tests).
apiVersion: v1
kind: Namespace
metadata:
name: gateway-conformance-app-backend
labels:
gateway-conformance: backend
---
# Namespace for simple web server backends. This is expected by
# the upstream conformance suite's Setup method.
apiVersion: v1
kind: Namespace
metadata:
name: gateway-conformance-web-backend
labels:
gateway-conformance: web-backend
---
# A basic Gateway resource that allows HTTPRoutes from the same namespace.
# Tests can use this as a parent reference for routes that target InferencePools.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: conformance-gateway
namespace: gateway-conformance-infra
spec:
# The conformance suite runner will replace this placeholder
# with the actual GatewayClass name provided via flags.
gatewayClassName: "{GATEWAY_CLASS_NAME}"
listeners:
- name: http # Standard listener name
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: All
kinds:
# Allows HTTPRoutes to attach, which can then reference InferencePools.
- group: gateway.networking.k8s.io
kind: HTTPRoute
---
# --- Conformance Secondary Gateway Definition ---
# A second generic Gateway resource for tests requiring multiple Gateways.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
Expand All @@ -66,7 +54,95 @@ spec:
- name: http
port: 80
protocol: HTTP
hostname: "secondary.example.com" # Distinct hostname to differentiate from conformance-gateway
hostname: "secondary.example.com"
allowedRoutes:
namespaces:
from: All
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: inference-model-1-deployment
namespace: gateway-conformance-app-backend
labels:
app: inference-model-1
spec:
replicas: 1
selector:
matchLabels:
app: inference-model-1
template:
metadata:
labels:
app: inference-model-1
spec:
containers:
- name: echo-basic-1
image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
ports:
- name: http
containerPort: 3000
readinessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: inference-model-2-deployment
namespace: gateway-conformance-app-backend
labels:
app: inference-model-2
spec:
replicas: 1
selector:
matchLabels:
app: inference-model-2
template:
metadata:
labels:
app: inference-model-2
spec:
containers:
- name: echo-basic-2
image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
ports:
- name: http
containerPort: 3000
readinessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
/*
Copyright 2025 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package basic

import (
"testing"

"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/gateway-api/conformance/utils/suite"

"sigs.k8s.io/gateway-api-inference-extension/conformance/tests"
k8sutils "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/kubernetes"
trafficutils "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/traffic"
)

func init() {
tests.ConformanceTests = append(tests.ConformanceTests, HTTPRouteMultipleGatewaysDifferentPools)
}

var HTTPRouteMultipleGatewaysDifferentPools = suite.ConformanceTest{
ShortName: "HTTPRouteMultipleGatewaysDifferentPools",
Description: "Validates two HTTPRoutes on different Gateways successfully referencing different InferencePools and routes traffic accordingly.",
Manifests: []string{"tests/basic/httproute_multiple_gateways_different_pools.yaml"},
Test: func(t *testing.T, s *suite.ConformanceTestSuite) {
const (
appBackendNamespace = "gateway-conformance-app-backend"
infraNamespace = "gateway-conformance-infra"
backendAppLabelKey = "app"

primaryGatewayName = "conformance-gateway"
routeForPrimaryGWName = "route-for-primary-gateway"
primaryPoolName = "primary-pool"
primaryBackendLabel = "inference-model-1"
primaryRoutePath = "/test-primary-gateway"

secondaryGatewayName = "conformance-secondary-gateway"
routeForSecondaryGWName = "route-for-secondary-gateway"
secondaryPoolName = "secondary-pool"
secondaryBackendLabel = "inference-model-2"
secondaryRoutePath = "/test-secondary-gateway"
secondaryRouteHostname = "secondary.example.com"
)

routeForPrimaryGWNN := types.NamespacedName{Name: routeForPrimaryGWName, Namespace: appBackendNamespace}
routeForSecondaryGWNN := types.NamespacedName{Name: routeForSecondaryGWName, Namespace: appBackendNamespace}
primaryPoolNN := types.NamespacedName{Name: primaryPoolName, Namespace: appBackendNamespace}
secondaryPoolNN := types.NamespacedName{Name: secondaryPoolName, Namespace: appBackendNamespace}
primaryGatewayNN := types.NamespacedName{Name: primaryGatewayName, Namespace: infraNamespace}
secondaryGatewayNN := types.NamespacedName{Name: secondaryGatewayName, Namespace: infraNamespace}

t.Run("Primary HTTPRoute, InferencePool, and Gateway path: verify status and traffic", func(t *testing.T) {
k8sutils.HTTPRouteAndInferencePoolMustBeAcceptedAndRouteAccepted(
t,
s.Client,
routeForPrimaryGWNN,
primaryGatewayNN,
primaryPoolNN,
)

primaryGwAddr := k8sutils.GetGatewayEndpoint(t, s.Client, s.TimeoutConfig, primaryGatewayNN)
primarySelector := labels.SelectorFromSet(labels.Set{backendAppLabelKey: primaryBackendLabel})
primaryPod := k8sutils.GetPod(t, s.Client, appBackendNamespace, primarySelector, s.TimeoutConfig.RequestTimeout)

trafficutils.MakeRequestAndExpectResponseFromPod(t, s.RoundTripper, s.TimeoutConfig, primaryGwAddr, primaryRoutePath, primaryPod)
})

t.Run("Secondary HTTPRoute, InferencePool, and Gateway path: verify status and traffic", func(t *testing.T) {
k8sutils.HTTPRouteAndInferencePoolMustBeAcceptedAndRouteAccepted(
t,
s.Client,
routeForSecondaryGWNN,
secondaryGatewayNN,
secondaryPoolNN,
)

secondaryGwAddr := k8sutils.GetGatewayEndpoint(t, s.Client, s.TimeoutConfig, secondaryGatewayNN)
secondarySelector := labels.SelectorFromSet(labels.Set{backendAppLabelKey: secondaryBackendLabel})
secondaryPod := k8sutils.GetPod(t, s.Client, appBackendNamespace, secondarySelector, s.TimeoutConfig.RequestTimeout)

trafficutils.MakeRequestAndExpectResponseFromPodWithHostname(t, s.RoundTripper, s.TimeoutConfig, secondaryGwAddr, secondaryRoutePath, secondaryRouteHostname, secondaryPod)
})
},
}
Loading