Skip to content

remove update-release-activity background job and calculate the release-activity on the fly #1291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Mar 1, 2021

Conversation

syphar
Copy link
Member

@syphar syphar commented Feb 27, 2021

the previous solution in update_release_activity used 60 separate queries.

We can easily replace this with a materialized view while also extending the range to all of history (refreshing the view takes ~200ms currently). This would enable us also to extend the time being shown, if we wanted to.

Another alternative approach could be to drop the background job completely (simplifying is always nice), and use a simple query with GROUP.
This would runs in ~15ms (compared to the <1ms for the query based on the materialized view).

When there is a decision on which way to go I would also add some more tests.

@syphar syphar force-pushed the better-release-activity branch from 4f0303a to 046037e Compare February 27, 2021 10:02
@syphar
Copy link
Member Author

syphar commented Feb 27, 2021

Just saw there is one small bug, will fix it

@jyn514
Copy link
Member

jyn514 commented Feb 28, 2021

Another alternative approach could be to drop the background job completely (simplifying is always nice), and use a simple query with GROUP.

Can you expand on this more? It sounds nice, I'd prefer the simpler solution where possible.

@syphar
Copy link
Member Author

syphar commented Feb 28, 2021

Another alternative approach could be to drop the background job completely (simplifying is always nice), and use a simple query with GROUP.

Can you expand on this more? It sounds nice, I'd prefer the simpler solution where possible.

@jyn514

  • remove the cronjob
  • remove the release_activity_updater
  • let the view directly run a single query (with date-filters, and GROUP BY).

Which definitely simplifies the code, but it adds ~15-20ms to the view (which could be fine for a view that isn't used that much)

@syphar
Copy link
Member Author

syphar commented Feb 28, 2021

@jyn514 I just did the simplifying change as I would see it.

If that's not what we want, I can just revert the commit.

Copy link
Member

@jyn514 jyn514 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good. I like calculating the info on demand instead of having the cronjob.

Co-authored-by: Joshua Nelson <[email protected]>
@syphar syphar changed the title store release activity in materialized view and use it for the view @syphar remove update-release-activity background job and calculate the release-activity on the fly Mar 1, 2021
@syphar syphar changed the title @syphar remove update-release-activity background job and calculate the release-activity on the fly remove update-release-activity background job and calculate the release-activity on the fly Mar 1, 2021
@syphar syphar requested a review from jyn514 March 1, 2021 17:51
@syphar
Copy link
Member Author

syphar commented Mar 1, 2021

@jyn514 I changed it to using the int-cast, also I extended the test a little.

Copy link
Member

@jyn514 jyn514 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Thank you!

@jyn514 jyn514 merged commit 5389b57 into rust-lang:master Mar 1, 2021
Nemo157 added a commit to Nemo157/docs.rs that referenced this pull request Mar 6, 2021
rust-lang#1291 removed this command, so starting the docker image fails
@syphar syphar deleted the better-release-activity branch March 7, 2021 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants