Skip to content

Implement credentials tracking #834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

Implement credentials tracking #834

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e91398a
Implement credentials tracking
Mar 22, 2024
55ea43a
Track credential usage within BitbucketCredentials#lookupCredentials
Mar 28, 2024
cc028d4
Rename BitbucketCredentials#lookupCredentials to lookupCredentialsAnd…
Mar 28, 2024
d1a1fe1
Replace deprecated usage of CredentialsProvider#lookupCredentials
Mar 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketApiUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
}

serverUrl = StringUtils.defaultIfBlank(serverUrl, serverUrlFallback);
StandardCredentials credentials = BitbucketCredentials.lookupCredentials(
StandardCredentials credentials = BitbucketCredentials.lookupCredentialsAndTrackUsage(

Check warning on line 53 in src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketApiUtils.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Not covered line 

Line 53 is not covered by tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why tracks credentials when populate the select of mirrors?

serverUrl,
context,
credentialsId,
Expand Down
9 changes: 2 additions & 7 deletions src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketBuildStatusNotifications.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,11 +165,6 @@ private static void createStatus(@NonNull Run<?, ?> build, @NonNull TaskListener
}
}

private static @CheckForNull BitbucketSCMSource findBitbucketSCMSource(Run<?, ?> build) {
SCMSource s = SCMSource.SourceByItem.findSource(build.getParent());
return s instanceof BitbucketSCMSource ? (BitbucketSCMSource) s : null;
}

private static void sendNotifications(BitbucketSCMSource source, Run<?, ?> build, TaskListener listener)
throws IOException, InterruptedException {
BitbucketSCMSourceContext sourceContext = new BitbucketSCMSourceContext(null,
Expand Down Expand Up @@ -244,7 +239,7 @@ public static class JobCheckOutListener extends SCMListener {
@Override
public void onCheckout(Run<?, ?> build, SCM scm, FilePath workspace, TaskListener listener, File changelogFile,
SCMRevisionState pollingBaseline) throws Exception {
BitbucketSCMSource source = findBitbucketSCMSource(build);
BitbucketSCMSource source = BitbucketSCMSource.findForRun(build);
if (source == null) {
return;
}
Expand Down Expand Up @@ -277,7 +272,7 @@ public static class JobCompletedListener extends RunListener<Run<?, ?>> {

@Override
public void onCompleted(Run<?, ?> build, TaskListener listener) {
BitbucketSCMSource source = findBitbucketSCMSource(build);
BitbucketSCMSource source = BitbucketSCMSource.findForRun(build);
if (source == null) {
return;
}
Expand Down
80 changes: 46 additions & 34 deletions src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketCredentials.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
package com.cloudbees.jenkins.plugins.bitbucket;

import com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketAuthenticator;
import com.cloudbees.plugins.credentials.Credentials;
import com.cloudbees.plugins.credentials.CredentialsMatchers;
import com.cloudbees.plugins.credentials.CredentialsProvider;
import com.cloudbees.plugins.credentials.common.StandardCertificateCredentials;
Expand All @@ -32,8 +33,8 @@
import com.cloudbees.plugins.credentials.domains.URIRequirementBuilder;
import edu.umd.cs.findbugs.annotations.CheckForNull;
import edu.umd.cs.findbugs.annotations.NonNull;
import hudson.model.Item;
import hudson.model.Queue;
import hudson.model.queue.Tasks;
import hudson.security.ACL;
import hudson.security.AccessControlled;
import hudson.util.FormValidation;
Expand All @@ -44,6 +45,7 @@
import org.apache.commons.lang.StringUtils;
import org.kohsuke.stapler.AncestorInPath;
import org.kohsuke.stapler.QueryParameter;
import org.springframework.security.core.Authentication;

/**
* Utility class for common code accessing credentials
Expand All @@ -53,26 +55,32 @@
throw new IllegalAccessError("Utility class");
}

/**
* Performs a lookup of credentials for the given context. Additionally, usage of the credentials is tracked for the
* given {@link SCMSourceOwner} via {@link CredentialsProvider#track(Item, Credentials)}
*/
@CheckForNull
static <T extends StandardCredentials> T lookupCredentials(@CheckForNull String serverUrl,
@CheckForNull SCMSourceOwner context,
@CheckForNull String id,
@NonNull Class<T> type) {
static <T extends StandardCredentials> T lookupCredentialsAndTrackUsage(@CheckForNull String serverUrl,
@CheckForNull SCMSourceOwner context,
@CheckForNull String id,
@NonNull Class<T> type) {
if (StringUtils.isNotBlank(id) && context != null) {
return CredentialsMatchers.firstOrNull(
CredentialsProvider.lookupCredentials(
type,
context,
context instanceof Queue.Task
? Tasks.getDefaultAuthenticationOf((Queue.Task) context)
: ACL.SYSTEM,
URIRequirementBuilder.fromUri(serverUrl).build()
),
CredentialsMatchers.allOf(
CredentialsMatchers.withId(id),
CredentialsMatchers.anyOf(CredentialsMatchers.instanceOf(type))
)
final T credentials = CredentialsMatchers.firstOrNull(
CredentialsProvider.lookupCredentialsInItem(
type,
context,
getAuthenticationForContext(context),
URIRequirementBuilder.fromUri(serverUrl).build()
),
CredentialsMatchers.allOf(
CredentialsMatchers.withId(id),
CredentialsMatchers.anyOf(CredentialsMatchers.instanceOf(type))
)
);

CredentialsProvider.track(context, credentials);

return credentials;
}
return null;
}
Expand All @@ -87,34 +95,32 @@
return result;
}
result.includeMatchingAs(
context instanceof Queue.Task
? Tasks.getDefaultAuthenticationOf((Queue.Task) context)
: ACL.SYSTEM,
context,
StandardCredentials.class,
URIRequirementBuilder.fromUri(serverUrl).build(),
AuthenticationTokens.matcher(BitbucketAuthenticator.authenticationContext(serverUrl))
getAuthenticationForContext(context),
context,
StandardCredentials.class,
URIRequirementBuilder.fromUri(serverUrl).build(),
AuthenticationTokens.matcher(BitbucketAuthenticator.authenticationContext(serverUrl))
);
return result;
}

static FormValidation checkCredentialsId(
@AncestorInPath @CheckForNull SCMSourceOwner context,
@QueryParameter String value,
@QueryParameter String serverUrl) {
if (!value.isEmpty()) {
AccessControlled contextToCheck = context == null ? Jenkins.get() : context;
contextToCheck.checkPermission(CredentialsProvider.VIEW);
if (CredentialsMatchers.firstOrNull(
CredentialsProvider.lookupCredentials(
StandardCertificateCredentials.class,
context,
context instanceof Queue.Task ? Tasks.getDefaultAuthenticationOf((Queue.Task) context) : ACL.SYSTEM,
URIRequirementBuilder.fromUri(serverUrl).build()),
CredentialsMatchers.allOf(
CredentialsMatchers.withId(value),
AuthenticationTokens.matcher(BitbucketAuthenticator.authenticationContext(serverUrl))
)
CredentialsProvider.lookupCredentialsInItem(
StandardCertificateCredentials.class,
context,
getAuthenticationForContext(context),

Check warning on line 118 in src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketCredentials.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Not covered lines 

Lines 98-118 are not covered by tests
URIRequirementBuilder.fromUri(serverUrl).build()),
CredentialsMatchers.allOf(
CredentialsMatchers.withId(value),
AuthenticationTokens.matcher(BitbucketAuthenticator.authenticationContext(serverUrl))
)
) != null) {
return FormValidation.warning("A certificate was selected. You will likely need to configure Checkout over SSH.");
}
Expand All @@ -123,4 +129,10 @@
return FormValidation.warning("Credentials are required for build notifications");
}
}

private static Authentication getAuthenticationForContext(SCMSourceOwner context) {
return context instanceof Queue.Task

Check warning on line 134 in src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketCredentials.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Partially covered line 

Line 134 is only partially covered, one branch is missing
? ((Queue.Task) context).getDefaultAuthentication2()
: ACL.SYSTEM2;

Check warning on line 136 in src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketCredentials.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Not covered line 

Line 136 is not covered by tests
}
}
2 changes: 1 addition & 1 deletion src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketGitSCMBuilder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ public BitbucketSCMSource scmSource() {
@NonNull
public BitbucketGitSCMBuilder withCredentials(String credentialsId, BitbucketRepositoryProtocol protocol) {
if (StringUtils.isNotBlank(credentialsId)) {
StandardCredentials credentials = BitbucketCredentials.lookupCredentials(
StandardCredentials credentials = BitbucketCredentials.lookupCredentialsAndTrackUsage(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is already tracked by the git plugin on checkout. If your pipeline does not have a checkout scm you never use this credentials.

scmSource.getServerUrl(),
scmSource.getOwner(),
DescriptorImpl.SAME.equals(scmSource.getCheckoutCredentialsId()) ? credentialsId : scmSource.getCheckoutCredentialsId(),
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketSCMNavigator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -494,7 +494,7 @@ public void visitSources(SCMSourceObserver observer) throws IOException, Interru
listener.getLogger().format("Must specify a repository owner%n");
return;
}
StandardCredentials credentials = BitbucketCredentials.lookupCredentials(
StandardCredentials credentials = BitbucketCredentials.lookupCredentialsAndTrackUsage(
serverUrl,
observer.getContext(),
credentialsId,
Expand Down Expand Up @@ -545,7 +545,7 @@ public List<Action> retrieveActions(@NonNull SCMNavigatorOwner owner,
// TODO when we have support for trusted events, use the details from event if event was from trusted source
listener.getLogger().printf("Looking up team details of %s...%n", getRepoOwner());
List<Action> result = new ArrayList<>();
StandardCredentials credentials = BitbucketCredentials.lookupCredentials(
StandardCredentials credentials = BitbucketCredentials.lookupCredentialsAndTrackUsage(
serverUrl,
owner,
credentialsId,
Expand Down
8 changes: 7 additions & 1 deletion src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketSCMSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@
import hudson.model.Action;
import hudson.model.Actionable;
import hudson.model.Item;
import hudson.model.Run;
import hudson.model.TaskListener;
import hudson.plugins.git.GitSCM;
import hudson.scm.SCM;
Expand Down Expand Up @@ -1072,7 +1073,7 @@ public DescriptorImpl getDescriptor() {

@CheckForNull
/* package */ StandardCredentials credentials() {
return BitbucketCredentials.lookupCredentials(
return BitbucketCredentials.lookupCredentialsAndTrackUsage(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's too earlier, in this way you track same credentials multiple time on webhook event. It should be tracked only at the beginning of retrieve method

getServerUrl(),
getOwner(),
getCredentialsId(),
Expand Down Expand Up @@ -1225,6 +1226,11 @@ public static void setEventDelaySeconds(int eventDelaySeconds) {
BitbucketSCMSource.eventDelaySeconds = Math.min(300, Math.max(0, eventDelaySeconds));
}

public static BitbucketSCMSource findForRun(Run<?, ?> run) {
SCMSource s = SCMSource.SourceByItem.findSource(run.getParent());
return s instanceof BitbucketSCMSource ? (BitbucketSCMSource) s : null;
}

private void initCloneLinks() {
if (primaryCloneLinks == null) {
BitbucketApi bitbucket = buildBitbucketClient();
Expand Down
27 changes: 27 additions & 0 deletions src/main/java/com/cloudbees/jenkins/plugins/bitbucket/CredentialTrackingRunListener.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package com.cloudbees.jenkins.plugins.bitbucket;

import com.cloudbees.plugins.credentials.CredentialsProvider;
import hudson.Extension;
import hudson.model.Run;
import hudson.model.listeners.RunListener;

/**
* Tracks the usage of credentials
*/
@Extension
public class CredentialTrackingRunListener extends RunListener<Run<?, ?>> {
Comment on lines +11 to +12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised that this is needed. Is this how other plugins track the credentials of runs?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest I have no idea.
This is my first time working on the jenkins / a jenksin plugin codebase and all of it is rather overwhelming. I just noticed that we didn't see any usages of our bitbucket credentials so I started to look into it. But even to find CredentialsProvider.track I had to look through the code of the credentials plugin as I couldn't find any documentation about it - the only hint was the UI showing Note: usage tracking requires the cooperation of plugins and consequently may not track every use. which set me down that path.

With my limited knowledge though it kind of makes sense as at the time of the actual job invocation any generic code wouldn't know about the configured credentials anymore as they would have been transfored already into whatever format is actually required to perform the task (eg an environment variable, or a request header, etc.), and even if you would hook exactly into that transformation you most likely wouldn't have access to the actual Run anymore. Of course I might be completely wrong here - so I'm happy about any insight. I am also fine with dropping this specific thing from the PR and opening up a separat ticket / PR if for the actual credentials tracking of job runs.

Please just let me know on how you want to proceed here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other plugins may be using CredentialsProvider.findCredentialById, which usually calls CredentialsProvider.track. However, a search for CredentialsProvider in github-branch-source-plugin shows only CredentialsProvider.lookupCredentials, CredentialsProvider.USE_ITEM, and CredentialsProvider.lookupStores.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So how should we procceed here?
Should I drop the listener for now? I don't think I can dig into this in more detail before end of next week.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KalleOlaviNiemitalo any feedback on this?

@Override
public void onInitialize(Run<?, ?> run) {
final BitbucketSCMSource source = BitbucketSCMSource.findForRun(run);

if (source == null) {
return;
}

final boolean usesSshCheckout = source.getTraits().stream().anyMatch(scmSourceTrait -> scmSourceTrait instanceof SSHCheckoutTrait);

if (!usesSshCheckout) {

Check warning on line 23 in src/main/java/com/cloudbees/jenkins/plugins/bitbucket/CredentialTrackingRunListener.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Partially covered line 

Line 23 is only partially covered, one branch is missing
CredentialsProvider.track(run, source.credentials());
}
}
}
4 changes: 2 additions & 2 deletions src/test/java/com/cloudbees/jenkins/plugins/bitbucket/BranchScanningIntegrationTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ public void uriResolverByCredentialsTest() throws Exception {
CredentialsProvider.lookupStores(j.jenkins).iterator().next()
.addCredentials(Domain.global(), c);

StandardCredentials creds = BitbucketCredentials.lookupCredentials(
StandardCredentials creds = BitbucketCredentials.lookupCredentialsAndTrackUsage(
null ,
source.getOwner(),
c.getId(),
Expand All @@ -127,7 +127,7 @@ public void uriResolverByCredentialsTest() throws Exception {
CredentialsProvider.lookupStores(j.jenkins).iterator().next()
.addCredentials(Domain.global(), c);

creds = BitbucketCredentials.lookupCredentials(
creds = BitbucketCredentials.lookupCredentialsAndTrackUsage(
null,
source.getOwner(),
c.getId(),
Expand Down
Loading