Skip to content

feat(conformance): Add HTTPRouteMultipleGatewaysDifferentPools test #838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c859896
copy from existing test to start with
SinaChavoshi May 14, 2025
fe540a7
update test and add gateway to main manifest
SinaChavoshi May 15, 2025
63fedab
fix spelling error
SinaChavoshi May 19, 2025
68812d7
check RouteConditionResolvedRefs instead of Reconciled
SinaChavoshi May 20, 2025
5efaeb2
add new line at the end of yaml files.
SinaChavoshi May 27, 2025
2cd8dab
delete blank lines in yaml files.
SinaChavoshi May 27, 2025
26704a1
change variable names to use primary / secondary
SinaChavoshi May 27, 2025
bb62da7
add to do for name change across the project.
SinaChavoshi May 27, 2025
796b584
fix comment
SinaChavoshi May 27, 2025
403b588
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 27, 2025
6506d44
change all variable names to use primary/secondary consistantly
SinaChavoshi May 27, 2025
dafd21e
add to do for http route testing
SinaChavoshi May 27, 2025
ecf11e4
HTTPRouteMustBeAcceptedAndResolved
SinaChavoshi May 27, 2025
f1ce162
add helper method.
SinaChavoshi May 27, 2025
f3d8e8c
using helper method for pool route accepted.
SinaChavoshi May 28, 2025
1cd8790
combline pool and route accpetnace
SinaChavoshi May 28, 2025
44b1427
add http route validation
SinaChavoshi May 28, 2025
d15c25e
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 28, 2025
1451a90
remove extra comments and yaml fields.
SinaChavoshi May 28, 2025
adcc104
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 28, 2025
1535ccc
remove duplicate method
SinaChavoshi May 28, 2025
ff25051
fix timeout
SinaChavoshi May 28, 2025
cf587a6
remove modifiedTimeoutConfig.HTTPRouteMustHaveCondition per review co…
SinaChavoshi May 28, 2025
afc8bc3
remove excessive logging from conformance.go
SinaChavoshi May 28, 2025
8588183
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
8b6a873
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
efe3990
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
b63459d
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
b903d8f
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
2d52812
Remove unecessary logs for accpeted reason.
SinaChavoshi May 28, 2025
939ae57
Remove all comments from yaml and go files and fix yaml
SinaChavoshi May 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions conformance/conformance.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,9 @@ import (

// Constants for the shared Gateway
const (
SharedGatewayName = "conformance-gateway" // Name of the Gateway in manifests.yaml
SharedGatewayNamespace = "gateway-conformance-infra" // Namespace of the Gateway
SharedGatewayName = "conformance-gateway" // Name of the primary Gateway in manifests.yaml
SharedGatewayNamespace = "gateway-conformance-infra" // Namespace of the shared Gateways
SecondaryGatewayName = "conformance-secondary-gateway" // Name of the secondary Gateway in manifests.yaml
)

// GatewayLayerProfileName defines the name for the conformance profile that tests
Expand Down Expand Up @@ -214,9 +215,15 @@ func RunConformanceWithOptions(t *testing.T, opts confsuite.ConformanceOptions)
cSuite.Setup(t, tests.ConformanceTests)

sharedGwNN := types.NamespacedName{Name: SharedGatewayName, Namespace: SharedGatewayNamespace}
secondaryGwNN := types.NamespacedName{Name: SecondaryGatewayName, Namespace: SharedGatewayNamespace}

// Validate Gateway setup.
// Validate Gateway setup for both Gateways.
t.Logf("Validating primary Gateway setup: %s/%s", sharedGwNN.Namespace, sharedGwNN.Name)
ensureGatewayAvailableAndReady(t, cSuite.Client, opts, sharedGwNN)

t.Logf("Validating secondary Gateway setup: %s/%s", secondaryGwNN.Namespace, secondaryGwNN.Name)
ensureGatewayAvailableAndReady(t, cSuite.Client, opts, secondaryGwNN)

t.Log("Running Inference Extension conformance tests against all registered tests")
err = cSuite.Run(t, tests.ConformanceTests)
require.NoError(t, err, "error running conformance tests")
Expand Down
26 changes: 24 additions & 2 deletions conformance/resources/manifests/manifests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ metadata:
gateway-conformance: backend

---
# Namespace for simple web server backends. This is expected by
# Namespace for simple web server backends. This is expected by
# the upstream conformance suite's Setup method.
apiVersion: v1
kind: Namespace
Expand All @@ -50,8 +50,30 @@ spec:
protocol: HTTP
allowedRoutes:
namespaces:
from: All
from: All
kinds:
# Allows HTTPRoutes to attach, which can then reference InferencePools.
- group: gateway.networking.k8s.io
kind: HTTPRoute

---
# --- Conformance Secondary Gateway Definition ---
# A second generic Gateway resource for tests requiring multiple Gateways.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: conformance-secondary-gateway
namespace: gateway-conformance-infra
spec:
gatewayClassName: "{GATEWAY_CLASS_NAME}"
listeners:
- name: http
port: 80
protocol: HTTP
hostname: "secondary.example.com" # Distinct hostname to differentiate from conformance-gateway
allowedRoutes:
namespaces:
from: All
kinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
119 changes: 119 additions & 0 deletions conformance/tests/basic/httproute_multiple_gateways_different_pools.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
/*
Copyright 2025 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package basic

import (
"testing"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
"sigs.k8s.io/gateway-api/conformance/utils/suite"

// Import the tests package to append to ConformanceTests
"sigs.k8s.io/gateway-api-inference-extension/conformance/tests"
"sigs.k8s.io/gateway-api-inference-extension/conformance/utils/config"
infrakubernetes "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/kubernetes"
gatewaykubernetes "sigs.k8s.io/gateway-api/conformance/utils/kubernetes"
)

func init() {
tests.ConformanceTests = append(tests.ConformanceTests, HTTPRouteMultipleGatewaysDifferentPools)
}

var HTTPRouteMultipleGatewaysDifferentPools = suite.ConformanceTest{
ShortName: "HTTPRouteMultipleGatewaysDifferentPools",
Description: "Validates two HTTPRoutes on different Gateways successfully referencing different InferencePools.",
Manifests: []string{"tests/basic/httproute_multiple_gateways_different_pools.yaml"},
Test: func(t *testing.T, s *suite.ConformanceTestSuite) {
const (
appBackendNamespace = "gateway-conformance-app-backend"
infraNamespace = "gateway-conformance-infra"
gateway1Name = "conformance-gateway"
gateway2Name = "conformance-secondary-gateway"
routeForGW1Name = "route-for-gw1"
routeForGW2Name = "route-for-gw2"
poolAName = "pool-a"
poolBName = "pool-b"
)

routeForGW1NN := types.NamespacedName{Name: routeForGW1Name, Namespace: appBackendNamespace}
routeForGW2NN := types.NamespacedName{Name: routeForGW2Name, Namespace: appBackendNamespace}
poolANN := types.NamespacedName{Name: poolAName, Namespace: appBackendNamespace}
poolBNN := types.NamespacedName{Name: poolBName, Namespace: appBackendNamespace}
gateway1NN := types.NamespacedName{Name: gateway1Name, Namespace: infraNamespace}
gateway2NN := types.NamespacedName{Name: gateway2Name, Namespace: infraNamespace}

var timeoutConfig config.InferenceExtensionTimeoutConfig = config.DefaultInferenceExtensionTimeoutConfig()

t.Run("HTTPRoute for Gateway 1 should be Accepted and Reconciled", func(t *testing.T) {
acceptedCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionAccepted),
Status: metav1.ConditionTrue,
Reason: string(gatewayv1.RouteReasonAccepted),
}
gatewaykubernetes.HTTPRouteMustHaveCondition(t, s.Client, timeoutConfig.TimeoutConfig, routeForGW1NN, gateway1NN, acceptedCondition)
t.Logf("HTTPRoute %s is Accepted by Gateway %s", routeForGW1NN.String(), gateway1NN.String())

reconciledCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionType("Reconciled")),
Status: metav1.ConditionTrue,
Reason: "ReconciliationSucceeded",
}
gatewaykubernetes.HTTPRouteMustHaveCondition(t, s.Client, timeoutConfig.TimeoutConfig, routeForGW1NN, gateway1NN, reconciledCondition)
t.Logf("HTTPRoute %s is Reconciled by Gateway %s", routeForGW1NN.String(), gateway1NN.String())
})

t.Run("InferencePool A (pool-a) should be Accepted", func(t *testing.T) {
acceptedCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionAccepted),
Status: metav1.ConditionTrue,
Reason: string(gatewayv1.RouteReasonAccepted),
}
infrakubernetes.InferencePoolMustHaveCondition(t, s.Client, poolANN, acceptedCondition)
t.Logf("InferencePool %s parent status shows Accepted by Gateway %s (via HTTPRoute %s)", poolANN.String(), gateway1NN.String(), routeForGW1NN.String())
})

t.Run("HTTPRoute for Gateway 2 should be Accepted and Reconciled", func(t *testing.T) {
acceptedCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionAccepted),
Status: metav1.ConditionTrue,
Reason: string(gatewayv1.RouteReasonAccepted),
}
gatewaykubernetes.HTTPRouteMustHaveCondition(t, s.Client, timeoutConfig.TimeoutConfig, routeForGW2NN, gateway2NN, acceptedCondition)
t.Logf("HTTPRoute %s is Accepted by Gateway %s", routeForGW2NN.String(), gateway2NN.String())

reconciledCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionType("Reconciled")),
Status: metav1.ConditionTrue,
Reason: "ReconciliationSucceeded",
}
gatewaykubernetes.HTTPRouteMustHaveCondition(t, s.Client, timeoutConfig.TimeoutConfig, routeForGW2NN, gateway2NN, reconciledCondition)
t.Logf("HTTPRoute %s is Reconciled by Gateway %s", routeForGW2NN.String(), gateway2NN.String())
})

t.Run("InferencePool B (pool-b) should be Accepted", func(t *testing.T) {
acceptedCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionAccepted),
Status: metav1.ConditionTrue,
Reason: string(gatewayv1.RouteReasonAccepted),
}
infrakubernetes.InferencePoolMustHaveCondition(t, s.Client, poolBNN, acceptedCondition)
t.Logf("InferencePool %s parent status shows Accepted by Gateway %s (via HTTPRoute %s)", poolBNN.String(), gateway2NN.String(), routeForGW2NN.String())
})
},
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
# httproute_multiple_gateways_different_pools.yaml
# This manifest defines resources for the httproute_multiple_gateways_different_pools conformance test.
# It includes two distinct backend deployments, their EPP services, two InferencePools,
# and two HTTPRoutes, each linking to one of the two shared Gateways from the base manifests.

# --- Backend Deployment A ---
apiVersion: apps/v1
kind: Deployment
metadata:
name: multi-gw-backend-a-deployment
namespace: gateway-conformance-app-backend
labels:
app: multi-gw-backend-a
spec:
replicas: 1
selector:
matchLabels:
app: multi-gw-backend-a
template:
metadata:
labels:
app: multi-gw-backend-a
spec:
containers:
- name: agnhost-echo-a
image: k8s.gcr.io/e2e-test-images/agnhost:2.39
args:
- serve-hostname
- --port=8080
ports:
- name: http
containerPort: 8080
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2

---
# --- Service for Backend Deployment A (EPP for pool-a) ---
apiVersion: v1
kind: Service
metadata:
name: pool-a-epp
namespace: gateway-conformance-app-backend
spec:
selector:
app: multi-gw-backend-a
ports:
- name: http
protocol: TCP
port: 8080
targetPort: 8080
- name: epp
protocol: TCP
port: 9002
targetPort: 9002

---
# --- Backend Deployment B ---
apiVersion: apps/v1
kind: Deployment
metadata:
name: multi-gw-backend-b-deployment
namespace: gateway-conformance-app-backend
labels:
app: multi-gw-backend-b
spec:
replicas: 1
selector:
matchLabels:
app: multi-gw-backend-b
template:
metadata:
labels:
app: multi-gw-backend-b
spec:
containers:
- name: agnhost-echo-b
image: k8s.gcr.io/e2e-test-images/agnhost:2.39
args:
- serve-hostname
- --port=8080
ports:
- name: http
containerPort: 8080
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2

---
# --- Service for Backend Deployment B (EPP for pool-b) ---
apiVersion: v1
kind: Service
metadata:
name: pool-b-epp
namespace: gateway-conformance-app-backend
spec:
selector:
app: multi-gw-backend-b
ports:
- name: http
protocol: TCP
port: 8080
targetPort: 8080
- name: epp
protocol: TCP
port: 9002
targetPort: 9002

---
# --- InferencePool A Definition ---
apiVersion: inference.networking.x-k8s.io/v1alpha2
kind: InferencePool
metadata:
name: pool-a
namespace: gateway-conformance-app-backend
spec:
selector:
app: "multi-gw-backend-a"
targetPortNumber: 8080
extensionRef:
name: pool-a-epp

---
# --- InferencePool B Definition ---
apiVersion: inference.networking.x-k8s.io/v1alpha2
kind: InferencePool
metadata:
name: pool-b
namespace: gateway-conformance-app-backend
spec:
selector:
app: "multi-gw-backend-b"
targetPortNumber: 8080
extensionRef:
name: pool-b-epp

---
# --- HTTPRoute for Gateway 1 (conformance-gateway) ---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: route-for-gw1
namespace: gateway-conformance-app-backend
spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: conformance-gateway # Referencing shared conformance-gateway
namespace: gateway-conformance-infra # Gateway is in infra namespace
sectionName: http
hostnames:
- "gw1.example.com" # Specific hostname for this route
rules:
- backendRefs:
- group: inference.networking.x-k8s.io
kind: InferencePool
name: pool-a
port: 8080
weight: 1
matches:
- path:
type: PathPrefix
value: /test-gw1

---
# --- HTTPRoute for Gateway 2 (conformance-secondary-gateway) ---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: route-for-gw2
namespace: gateway-conformance-app-backend
spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: conformance-secondary-gateway # Referencing shared conformance-secondary-gateway
namespace: gateway-conformance-infra # Gateway is in infra namespace
sectionName: http
hostnames:
- "secondary.example.com" # Matching hostname of conformance-secondary-gateway
rules:
- backendRefs:
- group: inference.networking.x-k8s.io
kind: InferencePool
name: pool-b
port: 8080
weight: 1
matches:
- path:
type: PathPrefix
value: /test-gw2
Loading