Skip to content

valtree's to_le() actually produces different result on big-endian from little-endian. #103183

Closed
@oli-obk

Description

@oli-obk
   For commit https://github.com/rust-lang/rust/pull/96591/commits/e14b34c386ad2809e937e0e6e0379c5cc5474954 , the two `to_le()` actually produces different result on big-endian from little-endian.
// Note: Don't use `StableHashResult` impl of `u64` here directly, since that
// would lead to endianness problems.
let hash: u128 = hasher.finish();
let hash_short = (hash.to_le() as u64).to_le();

Assume hash: u128=0x5e0f03940fda80bb6348c650c7b26618, then on LE, hash_short: u64 = 0x6348c650c7b26618, while BE hash_short: u64 = 0x5e0f03940fda80bb, which fails some unit tests on big endian targets because of hash mismatch.

If it's expected behavior to make hash value the same on BE/LE, code here should remove two to_le calls. We don't need to adjust endianness unless converting between multi-byte integer and byte sequences.

Originally posted by @ecnelises in #96591 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCategory: This is a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions