Skip to content

run cargo update #1587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 19, 2022
Merged

run cargo update #1587

merged 1 commit into from
Jan 19, 2022

Conversation

syphar
Copy link
Member

@syphar syphar commented Jan 8, 2022

Creating this also as a note to myself.

Were there specific reasons why we don't just regularly run cargo update? I would assume when everything compiles and tests pass we're fine.

If updates broke prod in the past there is probably a gap in the tests, which we could fill.

Looking back, I saw some commits that specifically updated some packages (3bfdc65), but others that look like they just update everything compatible (94daab5)

@syphar syphar added the S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed label Jan 8, 2022
@syphar syphar self-assigned this Jan 8, 2022
@syphar
Copy link
Member Author

syphar commented Jan 8, 2022

IMHO we should regularly do this for semver-compatible releases, or are there reasons not to?

An alternative could also be to enable dependabot for this and check its PRs release-by-release (including their changelogs).

cc @Nemo157 @jyn514

@jyn514
Copy link
Member

jyn514 commented Jan 9, 2022

If updates broke prod in the past there is probably a gap in the tests, which we could fill.

The last time we did a mass update, there was a memory leak in a (very) indirect dependency, which only became noticeable after running in prod under load for over a day: #1278. I don't think we have a good way to catch issues of that sort; I vaguely remember that the standard library has leaks when run under valgrind because it doesn't recognize rust-alloc, or something like that.

That said, I agree 6 months is probably too infrequent - I don't mind doing it more often, we should just be especially careful with deploys that update dependencies.

@jyn514
Copy link
Member

jyn514 commented Jan 9, 2022

I ran cargo update locally and got this output:

    Updating addr2line v0.16.0 -> v0.17.0
    Updating adler32 v1.0.4 -> v1.2.0
    Updating aho-corasick v0.7.10 -> v0.7.18
    Updating ansi_term v0.11.0 -> v0.12.1
    Updating anyhow v1.0.42 -> v1.0.52
    Removing arrayvec v0.5.1
    Updating async-trait v0.1.36 -> v0.1.52
    Updating autocfg v1.0.0 -> v1.0.1
    Updating backtrace v0.3.61 -> v0.3.63
    Updating base-x v0.2.6 -> v0.2.8
    Removing bitvec v0.19.5
      Adding block-buffer v0.10.0
    Updating bstr v0.2.13 -> v0.2.17
    Updating bumpalo v3.3.0 -> v3.9.1
    Removing bytes v0.5.4
    Removing bytes v1.0.1
      Adding bytes v1.1.0
    Updating bytesize v1.0.0 -> v1.1.0
    Updating bzip2 v0.4.2 -> v0.4.3
    Updating bzip2-sys v0.1.10+1.0.8 -> v0.1.11+1.0.8
    Updating cast v0.2.3 -> v0.2.7
    Updating cc v1.0.69 -> v1.0.72
    Updating chrono-tz v0.5.1 -> v0.6.1
      Adding chrono-tz-build v0.0.2
    Updating clap v2.33.1 -> v2.34.0
    Removing cloudabi v0.1.0
    Updating const-random v0.1.8 -> v0.1.13
    Updating const-random-macro v0.1.8 -> v0.1.13
    Updating const_fn v0.4.5 -> v0.4.9
      Adding convert_case v0.4.0
    Updating core-foundation v0.9.1 -> v0.9.2
    Updating core-foundation-sys v0.8.2 -> v0.8.3
      Adding cpufeatures v0.2.1
    Removing cpuid-bool v0.1.2
    Updating crates-index v0.15.1 -> v0.15.5
    Updating crc32fast v1.2.0 -> v1.3.0
    Updating criterion v0.3.3 -> v0.3.5
    Updating criterion-plot v0.4.3 -> v0.4.4
    Updating crossbeam-channel v0.5.1 -> v0.5.2
    Updating crossbeam-deque v0.8.0 -> v0.8.1
    Updating crossbeam-epoch v0.9.3 -> v0.9.6
    Removing crossbeam-utils v0.7.2
    Removing crossbeam-utils v0.8.3
      Adding crossbeam-utils v0.8.6
      Adding crunchy v0.2.2
      Adding crypto-common v0.1.1
    Updating crypto-mac v0.10.0 -> v0.10.1
    Updating csv v1.1.3 -> v1.1.6
    Updating derive_more v0.99.7 -> v0.99.17
      Adding digest v0.10.1
      Adding dirs v3.0.2
      Adding dirs-sys v0.3.6
    Updating dtoa v0.4.5 -> v0.4.8
    Updating dtoa-short v0.3.2 -> v0.3.3
    Updating either v1.5.3 -> v1.6.1
    Updating encoding_rs v0.8.23 -> v0.8.30
      Adding fastrand v1.6.0
    Updating filetime v0.2.10 -> v0.2.15
    Updating flate2 v1.0.14 -> v1.0.22
    Removing funty v1.1.0
    Updating futures v0.3.5 -> v0.3.19
    Updating futures-channel v0.3.13 -> v0.3.19
    Updating futures-core v0.3.13 -> v0.3.19
    Updating futures-executor v0.3.5 -> v0.3.19
    Updating futures-io v0.3.13 -> v0.3.19
    Updating futures-macro v0.3.13 -> v0.3.19
    Updating futures-sink v0.3.13 -> v0.3.19
    Updating futures-task v0.3.13 -> v0.3.19
    Updating futures-util v0.3.13 -> v0.3.19
    Updating generic-array v0.14.4 -> v0.14.5
    Removing getrandom v0.1.14
    Removing getrandom v0.2.2
      Adding getrandom v0.1.16
      Adding getrandom v0.2.3
    Updating gimli v0.25.0 -> v0.26.1
    Updating git2 v0.13.12 -> v0.13.25
    Updating globset v0.4.5 -> v0.4.8
    Updating globwalk v0.8.0 -> v0.8.1
    Updating h2 v0.3.3 -> v0.3.10
    Updating half v1.6.0 -> v1.8.2
    Removing hashbrown v0.9.0
      Adding hashbrown v0.9.1
      Adding hashbrown v0.11.2
    Updating heck v0.3.1 -> v0.3.3
    Updating hermit-abi v0.1.13 -> v0.1.19
    Updating hex v0.4.2 -> v0.4.3
      Adding hmac v0.12.0
    Updating http v0.2.1 -> v0.2.6
    Updating http-body v0.4.1 -> v0.4.4
    Updating httparse v1.4.1 -> v1.5.1
    Updating httpdate v1.0.1 -> v1.0.2
    Updating humansize v1.1.0 -> v1.1.1
    Updating hyper v0.14.11 -> v0.14.16
    Updating idna v0.2.0 -> v0.2.3
    Updating ignore v0.4.16 -> v0.4.18
    Updating indexmap v1.6.0 -> v1.8.0
    Updating inotify v0.7.0 -> v0.7.1
    Updating inotify-sys v0.1.3 -> v0.1.5
    Updating instant v0.1.6 -> v0.1.12
    Updating ipnet v2.3.0 -> v2.3.1
      Adding itertools v0.10.3
    Removing itoa v0.4.5
      Adding itoa v0.4.8
      Adding itoa v1.0.1
    Updating jobserver v0.1.21 -> v0.1.24
    Updating js-sys v0.3.49 -> v0.3.55
    Updating kuchiki v0.8.0 -> v0.8.1
    Updating lazycell v1.2.1 -> v1.3.0
    Removing lexical-core v0.7.5
    Updating libc v0.2.98 -> v0.2.112
    Updating libflate v1.0.0 -> v1.1.1
    Updating libflate_lz77 v1.0.0 -> v1.1.0
    Updating libgit2-sys v0.12.14+1.1.0 -> v0.12.26+1.3.0
    Updating libssh2-sys v0.2.19 -> v0.2.23
    Updating libz-sys v1.1.2 -> v1.1.3
    Removing lock_api v0.3.4
    Removing lock_api v0.4.1
      Adding lock_api v0.4.5
    Updating markup5ever v0.10.0 -> v0.10.1
    Updating matches v0.1.8 -> v0.1.9
      Adding md-5 v0.10.0
    Updating memchr v2.4.0 -> v2.4.1
    Updating memoffset v0.6.3 -> v0.6.5
      Adding minimal-lexical v0.2.1
    Removing miniz_oxide v0.3.6
    Removing mio v0.6.22
    Removing mio v0.7.11
      Adding mio v0.6.23
      Adding mio v0.7.14
    Updating miow v0.2.1 -> v0.2.2
    Updating native-tls v0.2.7 -> v0.2.8
    Updating net2 v0.2.34 -> v0.2.37
    Updating nix v0.20.0 -> v0.20.2
    Updating nom v6.1.2 -> v7.1.0
    Updating notify v4.0.15 -> v4.0.17
    Updating ntapi v0.3.4 -> v0.3.6
    Updating num-integer v0.1.42 -> v0.1.44
    Updating num-traits v0.2.11 -> v0.2.14
    Updating num_cpus v1.13.0 -> v1.13.1
    Updating object v0.26.0 -> v0.27.1
    Updating once_cell v1.7.2 -> v1.9.0
    Updating oorandom v11.1.1 -> v11.1.3
    Updating openssl v0.10.29 -> v0.10.38
    Updating openssl-probe v0.1.2 -> v0.1.4
    Updating openssl-sys v0.9.56 -> v0.9.72
    Removing parking_lot v0.10.2
    Removing parking_lot v0.11.0
      Adding parking_lot v0.11.2
    Removing parking_lot_core v0.7.2
    Removing parking_lot_core v0.8.0
      Adding parking_lot_core v0.8.5
    Updating parse-zoneinfo v0.2.1 -> v0.3.0
    Updating path-slash v0.1.3 -> v0.1.4
      Adding phf v0.10.1
      Adding phf_codegen v0.10.0
      Adding phf_generator v0.10.0
      Adding phf_shared v0.10.0
    Updating pin-project-lite v0.2.4 -> v0.2.8
    Updating pkg-config v0.3.17 -> v0.3.24
    Updating plotters v0.2.14 -> v0.3.1
      Adding plotters-backend v0.3.2
      Adding plotters-svg v0.3.1
    Updating postgres v0.19.0 -> v0.19.2
    Updating postgres-derive v0.4.0 -> v0.4.1
    Updating postgres-protocol v0.6.0 -> v0.6.3
    Updating postgres-types v0.2.0 -> v0.2.2
    Updating ppv-lite86 v0.2.8 -> v0.2.16
    Updating proc-macro-error v1.0.2 -> v1.0.4
    Updating proc-macro-error-attr v1.0.2 -> v1.0.4
    Removing proc-macro-nested v0.1.4
    Updating proc-macro2 v1.0.24 -> v1.0.36
    Updating quote v1.0.9 -> v1.0.14
    Updating r2d2 v0.8.8 -> v0.8.9
    Updating r2d2_postgres v0.18.0 -> v0.18.1
    Removing radium v0.5.3
    Updating rand v0.8.3 -> v0.8.4
    Updating rand_chacha v0.3.0 -> v0.3.1
    Updating rand_core v0.6.2 -> v0.6.3
    Updating rand_hc v0.3.0 -> v0.3.1
    Updating rayon v1.5.0 -> v1.5.1
    Updating rayon-core v1.9.0 -> v1.9.1
    Removing redox_syscall v0.1.56
    Removing redox_syscall v0.2.5
      Adding redox_syscall v0.2.10
    Updating regex v1.3.7 -> v1.5.4
    Updating regex-automata v0.1.9 -> v0.1.10
    Updating regex-syntax v0.6.17 -> v0.6.25
    Updating reqwest v0.11.2 -> v0.11.8
    Updating rustc-demangle v0.1.16 -> v0.1.21
    Updating rustwide v0.13.0 -> v0.13.1
    Updating ryu v1.0.4 -> v1.0.9
    Updating sass-sys v0.4.19 -> v0.4.22
    Updating scheduled-thread-pool v0.2.4 -> v0.2.5
    Updating security-framework v2.1.2 -> v2.3.1
    Updating security-framework-sys v2.1.1 -> v2.4.2
      Adding semver v0.10.0
    Updating serde v1.0.110 -> v1.0.133
    Updating serde_cbor v0.11.1 -> v0.11.2
    Updating serde_derive v1.0.110 -> v1.0.133
    Updating serde_json v1.0.53 -> v1.0.74
    Removing sha2 v0.9.1
      Adding sha2 v0.9.9
      Adding sha2 v0.10.1
    Updating signal-hook-registry v1.2.0 -> v1.4.0
    Updating siphasher v0.3.3 -> v0.3.7
    Updating slab v0.4.2 -> v0.4.5
    Updating smallvec v1.6.1 -> v1.7.0
      Adding smartstring v0.2.9
    Removing smol_str v0.1.16
    Removing socket2 v0.3.19
    Removing socket2 v0.4.1
      Adding socket2 v0.4.2
    Updating stable_deref_trait v1.1.1 -> v1.2.0
    Updating standback v0.2.8 -> v0.2.17
    Updating string_cache v0.8.0 -> v0.8.2
    Updating structopt v0.3.14 -> v0.3.25
    Updating structopt-derive v0.4.7 -> v0.4.18
    Updating subtle v2.2.3 -> v2.4.1
    Updating syn v1.0.60 -> v1.0.85
    Removing syn-mid v0.5.0
    Updating synstructure v0.12.3 -> v0.12.6
    Updating systemstat v0.1.7 -> v0.1.10
    Removing tap v1.0.1
    Updating tar v0.4.37 -> v0.4.38
    Updating tempfile v3.1.0 -> v3.3.0
    Updating tendril v0.4.1 -> v0.4.2
    Updating tera v1.5.0 -> v1.15.0
    Updating termcolor v1.1.0 -> v1.1.2
    Updating test-case v1.2.0 -> v1.2.1
    Updating thiserror v1.0.26 -> v1.0.30
    Updating thiserror-impl v1.0.26 -> v1.0.30
    Updating time v0.2.25 -> v0.2.27
    Updating time-macros v0.1.0 -> v0.1.1
    Updating time-macros-impl v0.1.1 -> v0.1.2
      Adding tiny-keccak v2.0.2
    Updating tinytemplate v1.1.0 -> v1.2.1
      Adding tinyvec v1.5.1
      Adding tinyvec_macros v0.1.0
    Updating tokio v1.10.0 -> v1.15.0
    Updating tokio-macros v1.1.0 -> v1.7.0
    Updating tokio-postgres v0.7.0 -> v0.7.5
    Updating tokio-stream v0.1.5 -> v0.1.8
    Updating tokio-util v0.6.5 -> v0.6.9
    Updating toml v0.5.6 -> v0.5.8
    Updating tower-service v0.3.0 -> v0.3.1
    Updating tracing v0.1.25 -> v0.1.29
    Updating tracing-core v0.1.17 -> v0.1.21
    Updating try-lock v0.2.2 -> v0.2.3
    Updating twoway v0.2.1 -> v0.2.2
    Updating typenum v1.12.0 -> v1.15.0
      Adding uncased v0.9.6
    Updating unicode-bidi v0.3.4 -> v0.3.7
    Updating unicode-normalization v0.1.12 -> v0.1.19
    Updating unicode-segmentation v1.6.0 -> v1.8.0
    Updating unicode-width v0.1.7 -> v0.1.9
    Updating unicode-xid v0.2.0 -> v0.2.2
    Updating url v2.2.1 -> v2.2.2
    Updating utf-8 v0.7.5 -> v0.7.6
    Updating vcpkg v0.2.8 -> v0.2.15
    Updating version_check v0.9.2 -> v0.9.4
    Updating walkdir v2.3.1 -> v2.3.2
    Updating wasm-bindgen v0.2.72 -> v0.2.78
    Updating wasm-bindgen-backend v0.2.72 -> v0.2.78
    Updating wasm-bindgen-futures v0.4.22 -> v0.4.28
    Updating wasm-bindgen-macro v0.2.72 -> v0.2.78
    Updating wasm-bindgen-macro-support v0.2.72 -> v0.2.78
    Updating wasm-bindgen-shared v0.2.72 -> v0.2.78
    Updating web-sys v0.3.39 -> v0.3.55
    Removing wyz v0.2.0
    Updating xdg v2.2.0 -> v2.4.0
    Updating xml-rs v0.8.3 -> v0.8.4
    Updating zeroize v1.1.0 -> v1.4.3
    Updating zstd v0.5.2+zstd.1.4.5 -> v0.5.4+zstd.1.4.7
    Updating zstd-safe v2.0.4+zstd.1.4.5 -> v2.0.6+zstd.1.4.7
    Updating zstd-sys v1.4.16+zstd.1.4.5 -> v1.4.18+zstd.1.4.7

(note that this generates a slightly different Cargo.lock than the PR since crossbeam already had a release in the meantime:)

diff --git a/Cargo.lock b/Cargo.lock
index b6cd36f..92c806f 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -531,9 +531,9 @@ dependencies = [
 
 [[package]]
 name = "crossbeam-channel"
-version = "0.5.1"
+version = "0.5.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "06ed27e177f16d65f0f0c22a213e17c696ace5dd64b14258b52f9417ccb52db4"
+checksum = "e54ea8bc3fb1ee042f5aace6e3c6e025d3874866da222930f70ce62aceba0bfa"
 dependencies = [
  "cfg-if 1.0.0",
  "crossbeam-utils",
@@ -552,9 +552,9 @@ dependencies = [
 
 [[package]]
 name = "crossbeam-epoch"
-version = "0.9.5"
+version = "0.9.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4ec02e091aa634e2c3ada4a392989e7c3116673ef0ac5b72232439094d73b7fd"
+checksum = "97242a70df9b89a65d0b6df3c4bf5b9ce03c5b7309019777fbde37e7537f8762"
 dependencies = [
  "cfg-if 1.0.0",
  "crossbeam-utils",
@@ -565,9 +565,9 @@ dependencies = [
 
 [[package]]
 name = "crossbeam-utils"
-version = "0.8.5"
+version = "0.8.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d82cfc11ce7f2c3faef78d8a684447b40d503d9681acebed6cb728d45940c4db"
+checksum = "cfcae03edb34f947e64acdb1c33ec169824e20657e9ecb61cef6c8c74dcb8120"
 dependencies = [
  "cfg-if 1.0.0",
  "lazy_static",
@@ -984,6 +984,15 @@ version = "0.2.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7"
 
+[[package]]
+name = "fastrand"
+version = "1.6.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "779d043b6a0b90cc4c0ed7ee380a6504394cee7efd7db050e3774eee387324b2"
+dependencies = [
+ "instant",
+]
+
 [[package]]
 name = "filetime"
 version = "0.2.15"
@@ -3880,13 +3889,13 @@ dependencies = [
 
 [[package]]
 name = "tempfile"
-version = "3.2.0"
+version = "3.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dac1c663cfc93810f88aed9b8941d48cabf856a1b111c29a40439018d870eb22"
+checksum = "5cdb1ef4eaeeaddc8fbd371e5017057064af0911902ef36b39801f67cc6d79e4"
 dependencies = [
  "cfg-if 1.0.0",
+ "fastrand",
  "libc",
- "rand 0.8.4",
  "redox_syscall",
  "remove_dir_all 0.5.3",
  "winapi 0.3.9",

That seems like too many crates to reasonably have dependabot open PRs for all of them, and I'm hesitant to remove Cargo.lock altogether since it makes the build very hard to reproduce. So I think periodic PRs with a mass update are the best alternative.

@syphar
Copy link
Member Author

syphar commented Jan 9, 2022

If updates broke prod in the past there is probably a gap in the tests, which we could fill.

The last time we did a mass update, there was a memory leak in a (very) indirect dependency, which only became noticeable after running in prod under load for over a day: #1278. I don't think we have a good way to catch issues of that sort; I vaguely remember that the standard library has leaks when run under valgrind because it doesn't recognize rust-alloc, or something like that.

But that's good to know. Now as you're saying it I also vaguely remember something about this.
Memory leaks are hard to cover in tests.
I was planning on merging and deploying updates only when I can watch prod on the side for a day or so.

That said, I agree 6 months is probably too infrequent - I don't mind doing it more often, we should just be especially careful with deploys that update dependencies.

Then I'll do it more regularly.

That seems like too many crates to reasonably have dependabot open PRs for all of them, and I'm hesitant to remove Cargo.lock altogether since it makes the build very hard to reproduce. So I think periodic PRs with a mass update are the best alternative.

Yeah, true. I see crates.io is using renovate for this, which seems to do a merged lockfile maintenance PR, but until we get that I can just do it manually.

While some features of dependabot are really nice (for example pulling the changelog into the PR description), the amount of dependencies ( and so dependabot PRs and commits) would be quite much noise in this repo.

@syphar
Copy link
Member Author

syphar commented Jan 9, 2022

I hope subsequent cargo upgrade will also help with the duplications. In the lockfile we have ~50 crates with more than one version.

@syphar
Copy link
Member Author

syphar commented Jan 13, 2022

@Nemo157 what can we do to get this forward?

Split into smaller pieces? How to split?

@Nemo157
Copy link
Member

Nemo157 commented Jan 13, 2022

I think despite this being a pretty massive update, we can probably deploy it as is, just plan not to deploy anything else for a few days to keep an eye out for issues.

@syphar syphar added S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it and removed S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed labels Jan 15, 2022
@syphar
Copy link
Member Author

syphar commented Jan 16, 2022

I'll merge & deploy this tomorrow morning, so I can watch the server while working.

@syphar syphar merged commit b60665a into rust-lang:master Jan 19, 2022
@syphar syphar deleted the cargo-update branch January 19, 2022 06:49
@syphar syphar removed the S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it label Jan 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants