Failed to mmap

[miedzinski]

I can't install anything with cargo.

$ cargo install --verbose rustfmt
    Updating registry `https://github.com/rust-lang/crates.io-index`
error: failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [2/-1] Failed to mmap. Could not write data: Invalid argument

or by putting dependency in Cargo.toml

$ cat Cargo.toml
[package]
name = "test"
version = "0.1.0"
authors = ["test"]

[dependencies]
rustfmt = "0.5"
$ cargo build --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
error: Unable to update registry https://github.com/rust-lang/crates.io-index

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [2/-1] Failed to mmap. Could not write data: Invalid argument

$ rustc --version
rustc 1.9.0 (e4e8b6668 2016-05-18)
$ cargo --version
cargo 0.10.0-nightly (10ddd7d 2016-04-08)

I am running up-to-date Ubuntu 16.04 LTS.

[alexcrichton]

Whoa, haven't seen that one before! Are you perhaps running on a system that's running low on memory? It may be worth running strace as well to take a look at the parameters to the mmap call that's failing.

[miedzinski]

mmap(NULL, 11, PROT_READ|PROT_WRITE, MAP_SHARED, 5, 0) = -1 EINVAL (Invalid argument)

There is also one thing I forgot to mention and it may be the cause. That Ubuntu is inside VirtualBox 5.0.22 on Windows host. I'm using this setup for years and never had any problems, so I haven't thought about mentioning this in first place.

Definitely RAM usage isn't an issue (host or guest).

---

This may be related to #1578. My whole home dir is on vbox shared folder.

$ df -T
Filesystem     Type     1K-blocks      Used Available Use% Mounted on
udev           devtmpfs    490492         0    490492   0% /dev
tmpfs          tmpfs       101592      3272     98320   4% /run
/dev/sda1      ext4      32894832   8020564  23180268  26% /
tmpfs          tmpfs       507956         8    507948   1% /dev/shm
tmpfs          tmpfs         5120         0      5120   0% /run/lock
tmpfs          tmpfs       507956         0    507956   0% /sys/fs/cgroup
/home          vboxsf   524287996 220802040 303485956  43% /home/dominik
tmpfs          tmpfs       101592         0    101592   0% /run/user/1000

I'm not sure whether it's important, but i'm using git from guest without any issues.

---

libgit2/git2go#194 (comment) looks familiar.

[alexcrichton]

Interesting! Could you gist the full syscall log as well? It looks like it's attempting to map whatever file descriptor 5 is, and it may be good to see how that's in play

[miedzinski]

https://paste.ubuntu.com/17997204/

I guess it's /home/dominik/.cargo/registry/index/github.com-88ac128001ac3a9a/.git/objects/pack/pack_git2_iAED2h/

[alexcrichton]

cc @carlosmn, does this sound familiar in libgit2?

It looks like the relevant syscalls leading up to the failure here are:

open("/home/dominik/.cargo/registry/index/github.com-88ac128001ac3a9a/.git/objects/pack/pack_git2_iAED2h", O_RDWR|O_CREAT|O_EXCL, 0600) = 5
[..]
lseek(5, 10, SEEK_SET)                  = 10
write(5, "P", 1)                        = 1
mmap(NULL, 11, PROT_READ|PROT_WRITE, MAP_SHARED, 5, 0) = -1 EINVAL (Invalid argument)

I think that cargo is using libgit2 0.2.23 right now FWIW

[carlosmn]

It looks to be the same issue as the one in the git2go tracker. The VirtualBox filesystem does not like something we're doing, but it's not clear what.

It's possible that it doesn't support files with holes even though it doesn't actually complain about them, or maybe it's just with mmap. Someone reported the tests run fine, which is really odd as it's the same code. It probably doesn't help that it seems to be translating unix calls into Windows filesystem calls, which breaks the assumptions we're making in the library.

@miedzinski can you clone fine if you're on VBox but without a shared home dir?

[miedzinski]

@carlosmn I can use all git commands (including clone) on vboxsf. If you are asking about libgit2 (which git binary doesn't use, right?), then you have to wait few hours until I come back home and try it with cargo.

[miedzinski]

@carlosmn, @alexcrichton, it works flawlessly.

$ cargo install rustfmt
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading rustfmt v0.5.0
# downloading and compiling lots of crates
  Installing /home/libgit/.cargo/bin/rustfmt
  Installing /home/libgit/.cargo/bin/cargo-fmt
$ ~/.cargo/bin/rustfmt -V
0.5.0

This is the same virtual machine, but this time cargo directory is on ext4.

[alexcrichton]

@miedzinski could you try compiling this and see if it runs into the same error when cloning onto a shared directory?

#include <git2.h>
#include <assert.h>

int main() {
  git_repository *repo;
  assert(git_libgit2_init() == 1);
  assert(git_clone(&repo,
                   "https://github.com/rust-lang/crates.io-index",
                   "crates.io-index",
                   NULL) == 0);
}

[miedzinski]

a.out: test.c:10: main: Assertion git_clone(&repo, "https://github.com/rust-lang/crates.io-index", "crates.io-index", NULL) == 0' failed.`

Built against 0.24.1-2 from Ubuntu repository.

[alexcrichton]

@miedzinski could you try using strace as well to ensure that it's failing with the same mmap error as before?

[miedzinski]

https://paste.ubuntu.com/18054099/

[alexcrichton]

Hm it's not clear from that gist that it's the same mmap error you're hitting with Cargo. It's also doing a different operation with a clone rather than a fetch, so that may be factoring in as well. It may be worth poking around various incantations to see if it can be reproduced with just libgit2 so we can report the bug upstream.

[carlosmn]

That error looks like libgit2 was not built with HTTPS support. Trying with git:// might provide more reliable results.

I've opened libgit2/libgit2#3845 to track this upstream, but I'm not sure that there's a fix that doesn't involve a lot of yak shaving so we don't need to read the data back from disk.