Skip to content

Upgrade rusoto (and tokio) #1264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 4, 2021
Merged

Conversation

coolreader18
Copy link
Contributor

@coolreader18 coolreader18 commented Jan 24, 2021

Fixes #1102

The new rusoto version also updates tokio, so this is a WIP of upgrading that.

Currently waiting on rust-lang/rustwide#51

Copy link
Member

@jyn514 jyn514 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has compile errors. Also tokio 0.2 is still in the dependency tree, I'd prefer not to update until that's fixed.

> cargo tree -i tokio:0.2.22
tokio v0.2.22
├── h2 v0.2.5
│   └── hyper v0.13.6
│       ├── hyper-tls v0.4.1
│       │   └── reqwest v0.10.10
│       │       └── rustwide v0.11.0
│       │           └── docs-rs v0.6.0 (/home/joshua/src/rust/docs.rs)
│       └── reqwest v0.10.10 (*)
├── hyper v0.13.6 (*)
├── hyper-tls v0.4.1 (*)
├── postgres v0.17.5
│   └── schemamama_postgres v0.3.0
│       └── docs-rs v0.6.0 (/home/joshua/src/rust/docs.rs)
├── reqwest v0.10.10 (*)
├── rustwide v0.11.0 (*)
├── tokio-postgres v0.5.5
│   └── postgres v0.17.5 (*)
├── tokio-tls v0.3.1
│   ├── hyper-tls v0.4.1 (*)
│   └── reqwest v0.10.10 (*)
└── tokio-util v0.3.1
    ├── h2 v0.2.5 (*)
    └── tokio-postgres v0.5.5 (*)

@coolreader18
Copy link
Contributor Author

@jyn514 are you sure you're on the right branch? I got rid of all the instances of tokio 0.2 in the dep tree I'm pretty sure, cargo tree -i tokio:0.2.22 gives error: package ID specification tokio:0.2.22 matched no packages

@jyn514
Copy link
Member

jyn514 commented Feb 4, 2021

Oh hmm something is fishy then - cherry-picking your changes to Cargo.toml and running cargo update doesn't replicate the same behavior. Can you say how you generated Cargo.lock?

@coolreader18
Copy link
Contributor Author

Huh, that's really strange. I just ran cargo update, I didn't do anything fancy. Do you want to try with the latest commit? I bumped rustwide in Cargo.toml, maybe that would have something to do with it

@coolreader18
Copy link
Contributor Author

docs.rs on  upgrade-rusoto is 📦 v0.6.0 via 🦀 v1.49.0 
❯ git checkout master Cargo.lock
Updated 1 path from 0b2f861

docs.rs on  upgrade-rusoto [+] is 📦 v0.6.0 via 🦀 v1.49.0 
❯ cargo update
# a lot

docs.rs on  upgrade-rusoto [!+] is 📦 v0.6.0 via 🦀 v1.49.0 
❯ git add --all

docs.rs on  upgrade-rusoto is 📦 v0.6.0 via 🦀 v1.49.0 
❯ git status
On branch upgrade-rusoto
Your branch is up to date with 'origin/upgrade-rusoto'.

nothing to commit, working tree clean

@jyn514
Copy link
Member

jyn514 commented Feb 4, 2021

Ok, I see - cargo update does something different than cargo update -p rustwide. The first updates everything that's semver compatible, the second only updates things that need to be updated to stay in sync with Cargo.toml.

Can you separate the cargo update commit from the semver changes? Preferably even a separate PR but I'm ok with keeping it in this one if it's easier.

@jyn514
Copy link
Member

jyn514 commented Feb 4, 2021

If you can post the output from cargo update, that would also make me feel a lot better that dependencies aren't being added without others removed. The lockfile isn't really human readable and I haven't found a good diff tool for it.

@coolreader18
Copy link
Contributor Author

I've separated the commits, I think the main one that was breaking things was schemamama_postgres, which updated to tokio 1.0 without a new semver minor version like rustwide originally did.

@jyn514
Copy link
Member

jyn514 commented Feb 4, 2021

Thanks. These were the versions changed:

> git checkout 38a0f57123fe3db9a1528f8941000124fb67d0d4
> git checkout HEAD~ Cargo.lock
> cargo update -p rustwide
    Updating crates.io index
    Removing arrayref v0.3.6
    Removing arrayvec v0.5.1
      Adding attohttpc v0.16.1
    Updating base64 v0.11.0 -> v0.13.0
    Removing blake2b_simd v0.5.10
      Adding bytes v1.0.1
      Adding cfg-if v1.0.0
    Removing constant_time_eq v0.1.5
      Adding crypto-mac v0.10.0
    Removing dirs v2.0.2
      Adding dirs-next v2.0.0
    Removing dirs-sys v0.3.5
      Adding dirs-sys-next v0.1.2
      Adding form_urlencoded v1.0.0
      Adding getrandom v0.2.2
    Updating h2 v0.2.5 -> v0.3.0
      Adding hmac v0.10.1
    Updating http-body v0.3.1 -> v0.4.0
      Adding httpdate v0.3.2
    Updating hyper v0.13.6 -> v0.14.2
    Updating hyper-tls v0.4.1 -> v0.5.0
      Adding ipnet v2.3.0
    Updating js-sys v0.3.39 -> v0.3.47
      Adding mio v0.7.7
    Removing mio-named-pipes v0.1.6
    Updating miow v0.3.3 -> v0.3.6
      Adding ntapi v0.3.4
    Updating once_cell v1.4.0 -> v1.5.2
      Adding pin-project v1.0.5
      Adding pin-project-internal v1.0.5
      Adding pin-project-lite v0.2.4
      Adding postgres v0.19.0
      Adding postgres-protocol v0.6.0
      Adding postgres-types v0.2.0
    Updating proc-macro2 v1.0.13 -> v1.0.24
    Updating r2d2_postgres v0.16.0 -> v0.18.0
      Adding rand v0.8.3
      Adding rand_chacha v0.3.0
      Adding rand_core v0.6.1
      Adding rand_hc v0.3.0
      Adding redox_syscall v0.2.4
    Updating redox_users v0.3.4 -> v0.4.0
    Updating reqwest v0.10.6 -> v0.11.0
    Updating rusoto_core v0.45.0 -> v0.46.0
    Updating rusoto_credential v0.45.0 -> v0.46.0
    Updating rusoto_s3 v0.45.0 -> v0.46.0
    Updating rusoto_signature v0.45.0 -> v0.46.0
    Removing rust-argon2 v0.7.0
    Updating rustwide v0.11.0 -> v0.12.0
    Updating serde_urlencoded v0.6.1 -> v0.7.0
    Updating socket2 v0.3.12 -> v0.3.19
    Updating syn v1.0.22 -> v1.0.60
      Adding tokio v1.1.1
    Updating tokio-macros v0.2.5 -> v1.0.0
      Adding tokio-native-tls v0.3.0
      Adding tokio-postgres v0.7.0
      Adding tokio-stream v0.1.2
    Removing tokio-tls v0.3.1
      Adding tokio-util v0.6.3
      Adding tracing v0.1.22
      Adding tracing-core v0.1.17
      Adding tracing-futures v0.2.4
    Updating url v2.1.1 -> v2.2.0
      Adding wasi v0.10.2+wasi-snapshot-preview1
    Updating wasm-bindgen v0.2.62 -> v0.2.70
    Updating wasm-bindgen-backend v0.2.62 -> v0.2.70
    Updating wasm-bindgen-futures v0.4.12 -> v0.4.20
    Updating wasm-bindgen-macro v0.2.62 -> v0.2.70
    Updating wasm-bindgen-macro-support v0.2.62 -> v0.2.70
    Updating wasm-bindgen-shared v0.2.62 -> v0.2.70
      Adding wildmatch v1.0.13
> git checkout 94daab526fd6b5e1f58479278d3b918df24c912c
> cargo update
    Updating crates.io index
    Updating ab_glyph_rasterizer v0.1.2 -> v0.1.4
    Updating addr2line v0.12.1 -> v0.14.1
      Adding adler v0.2.3
    Updating adler32 v1.0.4 -> v1.2.0
    Updating aho-corasick v0.7.10 -> v0.7.15
    Updating arc-swap v0.4.6 -> v0.4.8
      Adding arrayvec v0.5.2
    Updating async-trait v0.1.36 -> v0.1.42
    Updating autocfg v1.0.0 -> v1.0.1
    Updating backtrace v0.3.48 -> v0.3.56
    Updating base-x v0.2.6 -> v0.2.8
    Updating base64 v0.12.1 -> v0.12.3
      Adding bitvec v0.19.4
    Updating bstr v0.2.13 -> v0.2.14
    Updating bumpalo v3.3.0 -> v3.6.0
    Updating byteorder v1.3.4 -> v1.4.2
    Removing bytes v0.5.4
    Updating bytesize v1.0.0 -> v1.0.1
    Updating cc v1.0.54 -> v1.0.66
    Updating chrono-tz v0.5.1 -> v0.5.3
    Updating clap v2.33.1 -> v2.33.3
    Removing cloudabi v0.1.0
    Updating comrak v0.8.0 -> v0.8.2
    Updating const-random v0.1.8 -> v0.1.13
    Updating const-random-macro v0.1.8 -> v0.1.13
      Adding const_fn v0.4.5
    Updating core-foundation v0.7.0 -> v0.9.1
    Updating core-foundation-sys v0.7.0 -> v0.8.2
    Updating crates-index v0.15.1 -> v0.15.5
    Updating crc32fast v1.2.0 -> v1.2.1
    Updating criterion v0.3.3 -> v0.3.4
      Adding crossbeam-channel v0.5.0
    Updating crossbeam-deque v0.7.3 -> v0.8.0
    Updating crossbeam-epoch v0.8.2 -> v0.9.1
    Removing crossbeam-queue v0.2.1
    Updating crossbeam-utils v0.7.2 -> v0.8.1
      Adding crunchy v0.2.2
    Updating csv v1.1.3 -> v1.1.5
    Updating derive_more v0.99.7 -> v0.99.11
    Updating dtoa v0.4.5 -> v0.4.7
    Updating dtoa-short v0.3.2 -> v0.3.3
    Updating either v1.5.3 -> v1.6.1
    Updating encoding_rs v0.8.23 -> v0.8.28
    Updating filetime v0.2.10 -> v0.2.14
    Updating flate2 v1.0.14 -> v1.0.20
      Adding funty v1.1.0
    Updating futures v0.3.5 -> v0.3.12
    Updating futures-channel v0.3.5 -> v0.3.12
    Updating futures-core v0.3.5 -> v0.3.12
    Updating futures-executor v0.3.5 -> v0.3.12
    Updating futures-io v0.3.5 -> v0.3.12
    Updating futures-macro v0.3.5 -> v0.3.12
    Updating futures-sink v0.3.5 -> v0.3.12
    Updating futures-task v0.3.5 -> v0.3.12
    Updating futures-util v0.3.5 -> v0.3.12
    Updating generic-array v0.14.3 -> v0.14.4
    Updating getrandom v0.1.14 -> v0.1.16
    Updating gimli v0.21.0 -> v0.23.0
    Updating git2 v0.13.12 -> v0.13.17
    Updating globset v0.4.5 -> v0.4.6
    Updating globwalk v0.8.0 -> v0.8.1
    Updating half v1.6.0 -> v1.7.1
    Updating hashbrown v0.9.0 -> v0.9.1
    Updating heck v0.3.1 -> v0.3.2
    Updating hermit-abi v0.1.13 -> v0.1.18
    Updating http v0.2.1 -> v0.2.3
    Updating httparse v1.3.4 -> v1.3.5
    Updating ignore v0.4.16 -> v0.4.17
    Updating indexmap v1.6.0 -> v1.6.1
    Updating inotify v0.7.0 -> v0.7.1
    Updating inotify-sys v0.1.3 -> v0.1.5
    Updating instant v0.1.6 -> v0.1.9
      Adding itertools v0.10.0
    Updating itoa v0.4.5 -> v0.4.7
    Updating kuchiki v0.8.0 -> v0.8.1
    Updating lazycell v1.2.1 -> v1.3.0
      Adding lexical-core v0.7.4
    Updating libc v0.2.74 -> v0.2.85
    Updating libflate v1.0.0 -> v1.0.3
    Updating libgit2-sys v0.12.14+1.1.0 -> v0.12.18+1.1.0
    Updating libssh2-sys v0.2.19 -> v0.2.21
    Removing lock_api v0.3.4
    Removing lock_api v0.4.1
      Adding lock_api v0.4.2
    Updating log v0.4.8 -> v0.4.14
    Removing memchr v1.0.2
    Removing memchr v2.3.3
      Adding memchr v2.3.4
    Updating memoffset v0.5.4 -> v0.6.1
    Updating miniz_oxide v0.3.6 -> v0.4.3
    Updating mio v0.6.22 -> v0.6.23
    Updating miow v0.2.1 -> v0.2.2
    Updating native-tls v0.2.4 -> v0.2.7
    Updating net2 v0.2.34 -> v0.2.37
    Updating nom v3.2.1 -> v6.1.0
    Updating ntapi v0.3.4 -> v0.3.6
    Updating num-integer v0.1.42 -> v0.1.44
    Updating num-traits v0.2.11 -> v0.2.14
    Updating object v0.19.0 -> v0.23.0
    Updating oorandom v11.1.1 -> v11.1.3
    Updating openssl v0.10.29 -> v0.10.32
    Updating openssl-sys v0.9.56 -> v0.9.60
    Removing parking_lot v0.10.2
    Removing parking_lot v0.11.0
      Adding parking_lot v0.11.1
    Removing parking_lot_core v0.7.2
    Removing parking_lot_core v0.8.0
      Adding parking_lot_core v0.8.2
    Updating parse-zoneinfo v0.2.1 -> v0.3.0
    Updating path-slash v0.1.3 -> v0.1.4
    Updating pin-project v0.4.17 -> v0.4.27
    Updating pin-project-internal v0.4.17 -> v0.4.27
    Updating pkg-config v0.3.17 -> v0.3.19
    Updating plotters v0.2.14 -> v0.3.0
      Adding plotters-backend v0.3.0
      Adding plotters-svg v0.3.0
    Updating ppv-lite86 v0.2.8 -> v0.2.10
    Updating proc-macro-error v1.0.2 -> v1.0.4
    Updating proc-macro-error-attr v1.0.2 -> v1.0.4
    Updating proc-macro-hack v0.5.15 -> v0.5.19
    Updating proc-macro-nested v0.1.4 -> v0.1.7
    Updating quote v1.0.6 -> v1.0.8
    Updating r2d2 v0.8.8 -> v0.8.9
      Adding radium v0.5.3
    Updating rayon v1.3.0 -> v1.5.0
    Updating rayon-core v1.7.0 -> v1.9.0
    Updating redox_syscall v0.1.56 -> v0.1.57
    Updating regex v1.3.7 -> v1.4.3
    Updating regex-syntax v0.6.17 -> v0.6.22
    Updating rustc-demangle v0.1.16 -> v0.1.18
    Updating ryu v1.0.4 -> v1.0.5
    Updating sass-sys v0.4.19 -> v0.4.21
    Updating scheduled-thread-pool v0.2.4 -> v0.2.5
    Updating security-framework v0.4.4 -> v2.0.0
    Updating security-framework-sys v0.4.3 -> v2.0.0
      Adding semver v0.10.0
    Updating serde v1.0.110 -> v1.0.123
    Updating serde_derive v1.0.110 -> v1.0.123
    Updating serde_json v1.0.53 -> v1.0.61
    Updating sha2 v0.9.1 -> v0.9.3
      Adding shell-words v1.0.0
    Updating signal-hook-registry v1.2.0 -> v1.3.0
    Removing smallvec v0.6.13
    Removing smallvec v1.4.0
      Adding smallvec v0.6.14
      Adding smallvec v1.6.1
      Adding smartstring v0.2.6
    Removing smol_str v0.1.16
    Updating stable_deref_trait v1.1.1 -> v1.2.0
    Updating standback v0.2.8 -> v0.2.14
      Adding static_assertions v1.1.0
    Updating string_cache v0.8.0 -> v0.8.1
    Updating structopt v0.3.14 -> v0.3.21
    Updating structopt-derive v0.4.7 -> v0.4.14
    Updating subtle v2.2.3 -> v2.4.0
    Removing syn-mid v0.5.0
    Updating synstructure v0.12.3 -> v0.12.4
    Updating systemstat v0.1.5 -> v0.1.6
      Adding tap v1.0.0
    Updating tar v0.4.26 -> v0.4.32
    Updating tempfile v3.1.0 -> v3.2.0
    Updating tendril v0.4.1 -> v0.4.2
    Updating tera v1.5.0 -> v1.6.1
    Updating termcolor v1.1.0 -> v1.1.2
    Updating thiserror v1.0.20 -> v1.0.23
    Updating thiserror-impl v1.0.20 -> v1.0.23
    Updating thread_local v1.0.1 -> v1.1.3
    Updating time v0.2.16 -> v0.2.25
    Updating time-macros v0.1.0 -> v0.1.1
      Adding tiny-keccak v2.0.2
    Updating tinytemplate v1.1.0 -> v1.2.0
      Adding tinyvec v1.1.1
      Adding tinyvec_macros v0.1.0
    Updating toml v0.5.6 -> v0.5.8
    Updating tower-service v0.3.0 -> v0.3.1
    Updating try-lock v0.2.2 -> v0.2.3
    Updating ttf-parser v0.6.1 -> v0.6.2
    Updating unicode-normalization v0.1.12 -> v0.1.16
    Updating unicode-segmentation v1.6.0 -> v1.7.1
    Updating unicode-width v0.1.7 -> v0.1.8
    Updating unicode-xid v0.2.0 -> v0.2.1
    Updating vcpkg v0.2.8 -> v0.2.11
    Updating version_check v0.9.1 -> v0.9.2
    Updating web-sys v0.3.39 -> v0.3.47
    Updating winapi v0.3.8 -> v0.3.9
      Adding wyz v0.2.0
      Adding xdg v2.2.0
    Updating zeroize v1.1.0 -> v1.2.0
    Updating zstd v0.5.2+zstd.1.4.5 -> v0.5.4+zstd.1.4.7
    Updating zstd-safe v2.0.4+zstd.1.4.5 -> v2.0.6+zstd.1.4.7
    Updating zstd-sys v1.4.16+zstd.1.4.5 -> v1.4.18+zstd.1.4.7

and the number of duplicates went down from 23 to 16 which is definitely a step in the right direction :)

$ git checkout master
$ cargo tree -d |  grep -v '[├─│]' | grep -v '\*' | sort -u > duplicates-before
$ git checkout FETCH_HEAD
$ cargo tree -d |  grep -v '[├─│]' | grep -v '\*' | sort -u > duplicates-after
$ diff duplicates-before duplicates-after | grep '^<' | wc -l
23
$ diff duplicates-before duplicates-after | grep '^<' | wc -l
16

schemamama_postgres, which updated to tokio 1.0 without a new semver minor version like rustwide originally did.

By the way, 0.13 -> 0.14 is not a minor version bump because it's before 1.0, it's a major bump (which is why you had to change Cargo.toml).

@jyn514 jyn514 merged commit 66fa7c8 into rust-lang:master Feb 4, 2021
@jyn514 jyn514 added the S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it label Feb 4, 2021
@coolreader18
Copy link
Contributor Author

Right, that's what I meant, semver incompatible I guess. ¯\_(ツ)_/¯

@syphar syphar removed the S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it label Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RUSTSEC-2020-0053: dirs is unmaintained, use dirs-next instead
3 participants