Setting `viaIR` to true in solc results in zero coverage

[Fraccaman]

Following the docs here, and running npx hardhat coverage gives this result:

Version
=======
> solidity-coverage: v0.7.21

Instrumenting for coverage...
=============================

> contract/Bridge.sol
> contract/Governance.sol
> contract/Hub.sol
> interface/IBridge.sol
> interface/ICommon.sol
> interface/IGovernance.sol
> interface/IHub.sol

Compilation:
============

Nothing to compile

Network Info
============
> HardhatEVM: v2.9.3
> network:    hardhat



  Bridge
    ✔ Initialize contract testing (84ms)
    ✔ Update validate set testing (749ms)
    ✔ Authorize testing

  Governance
    ✔ Initialize contract testing (44ms)
    ✔ Update validator set testing (429ms)
    ✔ Upgrade contract testing (67ms)
    ✔ Add contract testing (69ms)

  Hub
    ✔ Initialize contract testing (157ms)


  8 passing (5s)

------------------|----------|----------|----------|----------|----------------|
File              |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
------------------|----------|----------|----------|----------|----------------|
 contract/        |        0 |        0 |        0 |        0 |                |
  Bridge.sol      |        0 |        0 |        0 |        0 |... 335,345,355 |
  Governance.sol  |        0 |        0 |        0 |        0 |... 229,230,233 |
  Hub.sol         |        0 |        0 |        0 |        0 |... 84,88,92,96 |
 interface/       |      100 |      100 |      100 |      100 |                |
  IBridge.sol     |      100 |      100 |      100 |      100 |                |
  ICommon.sol     |      100 |      100 |      100 |      100 |                |
  IGovernance.sol |      100 |      100 |      100 |      100 |                |
  IHub.sol        |      100 |      100 |      100 |      100 |                |
------------------|----------|----------|----------|----------|----------------|
All files         |        0 |        0 |        0 |        0 |                |
------------------|----------|----------|----------|----------|----------------|

> Istanbul reports written to ./coverage/ and ./coverage.json

which is weird. My .solcover.js content is:

module.exports = {
    configureYulOptimizer: true
};

Not sure what I'm missing.

[cgewecke]

Could you try running npx hardhat clean && npx hardhat coverage?

Also if you could share your config or a reproducible example that would be helpful.

[Fraccaman]

The output didn't change :(

npx hardhat clean && npx hardhat coverage                                                                                                                    ✔  12.22.1   18:51:27

Version
=======
> solidity-coverage: v0.7.21

Instrumenting for coverage...
=============================

> contract/Bridge.sol
> contract/Governance.sol
> contract/Hub.sol
> interface/IBridge.sol
> interface/ICommon.sol
> interface/IGovernance.sol
> interface/IHub.sol

Compilation:
============

Warning: Unused function parameter. Remove or comment out the variable name to silence this warning.
   --> contracts/contract/Bridge.sol:333:9:
    |
333 |         ValidatorSetArgs calldata validatorSetArgs,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


Warning: Unused function parameter. Remove or comment out the variable name to silence this warning.
   --> contracts/contract/Bridge.sol:334:9:
    |
334 |         Signature[] calldata signatures,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


Warning: Unused function parameter. Remove or comment out the variable name to silence this warning.
   --> contracts/contract/Bridge.sol:335:9:
    |
335 |         address to
    |         ^^^^^^^^^^


Warning: Function state mutability can be restricted to pure
   --> contracts/contract/Bridge.sol:332:5:
    |
332 |     function withdraw(
    |     ^ (Relevant source part starts here and spans across multiple lines).


Compiled 14 Solidity files successfully

Network Info
============
> HardhatEVM: v2.9.3
> network:    hardhat



  Bridge
    ✔ Initialize contract testing (92ms)
    ✔ Update validate set testing (761ms)
    ✔ Authorize testing

  Governance
    ✔ Initialize contract testing (39ms)
    ✔ Update validator set testing (444ms)
    ✔ Upgrade contract testing (65ms)
    ✔ Add contract testing (69ms)

  Hub
    ✔ Initialize contract testing (155ms)


  8 passing (5s)

------------------|----------|----------|----------|----------|----------------|
File              |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
------------------|----------|----------|----------|----------|----------------|
 contract/        |        0 |        0 |        0 |        0 |                |
  Bridge.sol      |        0 |        0 |        0 |        0 |... 335,345,355 |
  Governance.sol  |        0 |        0 |        0 |        0 |... 229,230,233 |
  Hub.sol         |        0 |        0 |        0 |        0 |... 84,88,92,96 |
 interface/       |      100 |      100 |      100 |      100 |                |
  IBridge.sol     |      100 |      100 |      100 |      100 |                |
  ICommon.sol     |      100 |      100 |      100 |      100 |                |
  IGovernance.sol |      100 |      100 |      100 |      100 |                |
  IHub.sol        |      100 |      100 |      100 |      100 |                |
------------------|----------|----------|----------|----------|----------------|
All files         |        0 |        0 |        0 |        0 |                |
------------------|----------|----------|----------|----------|----------------|

> Istanbul reports written to ./coverage/ and ./coverage.json

Ill setup a reproducible example and report back! Thanks for the quick response :)

[Fraccaman]

@cgewecke here you go! Thanks again for your help!

[cgewecke]

Thanks for the repro. If I comment out viaIR in your compiler settings as below I get the expected report. You might want to set this conditionally using an environment variable that's toggled on when coverage runs.

solidity: {
    version: "0.8.13",
    settings: {
      optimizer: {
        enabled: true,
        runs: 200,
      },
      //viaIR: true,
    },
  },

Report

[EDIT - For clarification, some optimizer settings remove the instrumentation soldiity-coverage injects into the contracts to track the execution trace. I've edited your issue title to reflect what I think the underlying cause is so other people can easily find it.]

TODO: set viaIR to false by default.

[Fraccaman]

ah, I see! thanks for the suggestion, works like a charm now :)

[aleph-v]

Hi @cgewecke, do y'all have plans to support coverage with viaIR == true? The default of turning off viaIR for our code means coverage cannot run because viaIR eliminates stack depth too deep errors. Example error from our code:

Compilation:
============
YulException: Variable var__143_mpos is 3 slot(s) too deep inside the stack.

[Xutp-F]

hello，do you need to write tests manually？

[jamesmorgan]

Any updates on this - would be great to get coverage back in action for viaIR projects