v0.18.0 postinstall missing dependencies

[daltones]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
I think some dependencies are missing at the time when postinstall scripts are ran.

I'm having this error:

yarn install v0.18.0
info No lockfile found.
warning test: No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
warning fsevents@1.0.15: The platform "linux" is incompatible with this module.
info "fsevents@1.0.15" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
[1/3] ⠐ gifsicle
[2/3] ⠐ jpegtran-bin
[3/3] ⠐ optipng-bin
[-/3] ⠐ waiting...
error [my-project]/node_modules/jpegtran-bin: Command failed.
Exit code: 1
Command: sh
Arguments: -c node lib/install.js
Directory: [my-project]/node_modules/jpegtran-bin
Output:
module.js:472
    throw err;
    ^

Error: Cannot find module 'ini'
    at Function.Module._resolveFilename (module.js:470:15)
    at Function.Module._load (module.js:418:25)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> ([my-project]/node_modules/rc/lib/utils.js:3:12)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

In another project, after a yarn upgrade I also encountered this problem with other packages (gifsicle's postinstall missing deep-extend). So it seems like a general problem.

If the current behavior is a bug, please provide the steps to reproduce.
Given this package.json:

{
  "name": "test",
  "devDependencies": {
    "babel-cli": "^6.18.0",
    "gulp-imagemin": "^3.1.1"
  }
}

Run yarn install.

I noticed that if I try to install only one of these two, it will work. I guess that's an issue with shared dependencies between them.

What is the expected behavior?
Installation completes successful.

Please mention your node.js, yarn and operating system version.
node.js: v7.2.0
yarn: v0.18.0
OS: Arch Linux

[olingern]

This appears to be a dependency problem with fsevents. Your OS, Arch Linux, doesn't support it. If you head over to Strongloop's fsevents Github page, you'll see this in their README

Native access to OS X FSEvents in Node.js

The FSEvents API in OS X allows applications to register for notifications of changes to a given directory tree. It is a very fast and lightweight alternative to kqueue.

So, I would say this isn't a Yarn issue. The other errors are probably a result of the previous failing.

[daltones]

@olingern No, it's just an expected warning. It was always there (even with npm or older versions of yarn).

I pointed a problem of missing dependencies that must have been installed before postinstall.

Just to be clear about the problem with common dependencies (somewhere in the tree of babel-cli and gulp-imagemin):

These two will complete successful:

{
  "name": "test",
  "devDependencies": {
    "babel-cli": "^6.18.0"
  }
}

{
  "name": "test",
  "devDependencies": {
    "gulp-imagemin": "^3.1.1"
  }
}

But this will lead to the error:

{
  "name": "test",
  "devDependencies": {
    "babel-cli": "^6.18.0",
    "gulp-imagemin": "^3.1.1"
  }
}

[mvestergaard]

I was able to replicate this on windows. I think it may be related to #2116. I'll try to investigate.

[olingern]

@daltones This isn't a warning, it's an error, but install continues.

info "fsevents@1.0.15" is an optional dependency and failed compatibility check. *Excluding it from installation.*

That dependency isn't being met because of your OS. This would hold true on Windows, too

Looking a little closer, it is optional, so I'll attempt on OSX as well.

[mvestergaard]

@olingern You're misunderstanding what the problem is.
deep-extend is a transient dependency of fsevents, it is also a dependency of something else. Because fsevents is being skipped, so is deep-extend and therefore something else reliant on it, is failing.

This is a bug.

[daltones]

@olingern It's just a warning that an optional dependency won't be installed. But the problem is that it messes up with other dependencies required somewhere else.

[olingern]

The package.json configuration you listed above installs without issue on OSX. I still believe this is an OS compatibility issue.

[mvestergaard]

@olingern fsvents isn's skipped on OSX so you can't replicate it.

I'm attempting to write a test that catches this problem. I know this worked with #1997 but it's broken with #2116

[olingern]

@daltones @mvestergaard I see. You guys are right. It looks like dependencies are being ignored when an optional doesn't resolve.

[mvestergaard]

It's proving difficult to replicate in a test though, it seems like it has to go multiple levels deep. I tried this contrived example:

dep-a - optional (incompatible with any os)
  - dep-c
dep-b
  - dep-c

But that works fine. So I'm going to put the shared dependency deeper and see if that's causing it.

[olingern]

@mvestergaard Since I was able to install on OSX, looking through the lockfile made figuring the dependency tree easier for this. Interestingly enough, it seems to be rooted in multi-leveled optional dependencies

- babel-cli
  - chokidar 
    - fsevents   
      - node-pre-gyp
        - rc
          - ini

Important to note above, chokidar and fsevents are optional dependencies

[mvestergaard]

I've also found that:

- a (optional, incompatible)
  - b
    - c
      - g
- d
  - e
    - f
      - g

Results in: a, c, d, e, f, g being installed, only b is skipped. It should be d, e, f, g.
I'm trying to write some tests that points me in the right direction, but I find it a bit difficult as I can't seem to figure out how to setup tooling to be able to debug yarn.

The above was my bad, that works.

I've sort of given up on writing a test that replicates this across all platforms. Seems like some sort of edge case.
@kittens @bestander maybe you guys can look into this?

[fernandofleury]

In case someone is looking for a temporarily solution, executing a "vanilla" npm install will do the trick.

[bestander]

Anyone eager to send a PR or a PR with a breaking unit test for starters?