Skip to content

⚠️ Drop rest client request latecy metric #1586

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

⚠️ Drop rest client request latecy metric #1586

wants to merge 1 commit into from

Conversation

2uasimojo
Copy link
Contributor

Remove the rest_client_request_latency_seconds metric.

This metric has been deprecated since kube 1.14 and turned off in kube 1.17.

fixes #1423

Co-Authored-By: @suhanime

@k8s-ci-robot k8s-ci-robot added do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jul 9, 2021
@k8s-ci-robot k8s-ci-robot requested review from alenkacz and gerred July 9, 2021 15:02
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: 2uasimojo
To complete the pull request process, please assign vincepri after the PR has been reviewed.
You can assign the PR to them by writing /assign @vincepri in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
Copy link
Contributor

Hi @2uasimojo. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 9, 2021
@2uasimojo 2uasimojo mentioned this pull request Jul 9, 2021
Closed
logicalhan
logicalhan reviewed Jul 9, 2021
View reviewed changes
Copy link

@logicalhan logicalhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to break a bunch of people who alert off of this metric, is that going to be okay?

@2uasimojo
Copy link
Contributor Author

2uasimojo commented Jul 9, 2021 •
edited
Loading

This is going to break a bunch of people who alert off of this metric, is that going to be okay?

This is where it would help me to understand what the actual effect of kubernetes/kubernetes#83836 was. Wouldn't that have already broken such consumers?

Also: They could restore by registering the metric themselves, like this, right?

alvaroaleman
alvaroaleman reviewed Jul 9, 2021
View reviewed changes
Copy link
Member

@alvaroaleman alvaroaleman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 9, 2021
@logicalhan
Copy link

This is where it would help me to understand what the actual effect of kubernetes/kubernetes#83836 was. Wouldn't that have already broken such consumers?

We actually have a deprecation policy, we announce a metric deprecation, we use our prometheus wrapper to annotate our metrics, our framework, on a version bump will auto-hide the metric (which yes breaks consumers). However, they can re-enable it for a release in order to properly migrate off of it. After that, the metric is deleted.

We offer a staged way to softly push people from using a metric that we intend to delete, in this PR, it'll just abruptly break people...

@alvaroaleman
Copy link
Member

This is going to break a bunch of people who alert off of this metric, is that going to be okay?

@logicalhan is that an assumption or do you personally know of a case where someone uses this? The metric includes the whole url as a label, which is obviously a cardinality problem

@logicalhan
Copy link

@logicalhan is that an assumption or do you personally know of a case where someone uses this? The metric includes the whole url as a label, which is obviously a cardinality problem

I'm assuming people use recording rules to aggregate the metric into something more useable.

@logicalhan
Copy link

FYI (We also have a way to enforce cardinality bounds): kubernetes/enhancements#1692

@logicalhan
Copy link

We try really, really hard not to break people upstream...

@alvaroaleman
Copy link
Member

alvaroaleman commented Jul 9, 2021 •
edited
Loading

We try really, really hard not to break people upstream...

Sure but deactivating a metric and forcing ppl to do sth to get it back is always going to break them. I am also fine with adding an explicit opt-in for this and removing it in a future release, but the only value add I see in that is for cases where ppl use this metric and don't know how to properly copy the code that is currently being removed.

@2uasimojo
⚠️ Drop rest client request latecy metric
4cbef7a 
Remove the `rest_client_request_latency_seconds` metric.

This metric has been deprecated since kube 1.14 and [turned off in kube
1.17](kubernetes/kubernetes#83836).

Co-Authored-By: suhanime
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Jul 9, 2021
@2uasimojo
Copy link
Contributor Author

Okay; if either

recording rules

or

a way to enforce cardinality bounds

allows us to effectively nix this metric from taking up space, we're happy to do that. What does the former look like? /me starts reading the latter...

@2uasimojo
Copy link
Contributor Author

adding an explicit opt-in for this

like this? #1587

@logicalhan
Copy link

Sure but deactivating a metric and forcing ppl to do sth to get it back is always going to break them.

Yes but in order to unbreak in this case is to roll-back a version. In our case, you just need to enable a flag. This means you can bump the version and use it in an unbroken way, but you must now explicitly acknowledge that this metric is not one you can depend on in a future release. It is a forcing function which is softer than, you can't use this release if you can't tolerate not having this metric.

@logicalhan
Copy link

adding an explicit opt-in for this

like this? #1587

Yeah I probably prefer the optional approach, it's softer and friendly to users.

@2uasimojo
Copy link
Contributor Author

Okay, let's pursue this via #1587 (or something like it). Thanks!

@2uasimojo 2uasimojo closed this Jul 9, 2021
@2uasimojo 2uasimojo deleted the issues/1423/nix branch July 9, 2021 15:36
@2uasimojo 2uasimojo mentioned this pull request Jul 9, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@logicalhan logicalhan logicalhan left review comments

@alvaroaleman alvaroaleman alvaroaleman left review comments

@alenkacz alenkacz Awaiting requested review from alenkacz

@gerred gerred Awaiting requested review from gerred

Assignees
No one assigned
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Disable rest client latency metric
4 participants
@2uasimojo @k8s-ci-robot @logicalhan @alvaroaleman