Merge #3280

3280: Update to cardano-node 1.35-rc2 r=Anviking a=Anviking

### Overview

- Updates dependencies to target `cardano-node 1.35-rc2`
- Adds support to make the wallet _mostly_ work in Babbage
- Rework of nix setup: we now derive nix files from `cabal.project` rather than the weird `stack.yaml`-based approach we had before.


### Detailed todo-list

- [x] Get nix happy with the new dependencies
- [x] Make `cabal build all` work with modulo warnings
- [x] Update to more recent revisions
    - [x] Fix compilation in tests again
- [x] Make `cabal build all` work with _no_ warnings
    - [x] Fix puzzling persistent TH name shadowing warnings -> fixed by sam?
- [x] Ensure existing tests pass
    - [x] unit
    - [x] Look closer at balanceTx golden unit failure after first seeing if balanceTx integration tests pass on both alonzo and Babbage
    - [x] Integration (on alonzo like on master)
        - [x] Was blocked by readTVar, fixed with input-output-hk/io-sim#6 (review)
        - [x] Fix failures relating to Babbage era -> fixed with a hack 9e2568e
        - [x] Fix cardano-wallet-core Aeson failures
- [x] More closely review code-changes
- [x] Clean up git commit history
    - [x] Flatten merge commits
    - [x] Squash back-and-forth nix and revision changes
    - [x] Rebase off master 
- [x] Ensure revisions in stack.yaml and cabal.project are updated consistently 
    - Alternatively remove `stack.yaml` and ensure ./nix/regenerate.sh still works and that we know how to update sha256map.nix — #3328 but without the sha256map.nix
- [x] Ensure CI is completely green: [Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-3280), ADP-1888
    - There may be CI specific problems with nix, windows and more (there probably are)
    - [x] fix hydra wine TH error e.g. `$(TH.lift =<< TH.runIO (readFile "app/foo.txt"))`
          - seems relevant input-output-hk/haskell.nix#1508
          - seems gone now
    - [x] `Ambiguous module name` https://hydra.iohk.io/build/15469467
        - https://input-output-rnd.slack.com/archives/C48HA39RS/p1654388106509579
        - Bumping Haskell.nix again should help?
        - Didn't seem like it worked...
        - https://input-output-rnd.slack.com/archives/CAP8NM7N0/p1654828014483969
     - [x] Remove-iserv problem https://hydra.iohk.io/build/15604435 
         - Update from Hamish https://input-output-rnd.slack.com/archives/CAP8NM7N0/p1655213059547499?thread_ts=1654828014.483969&cid=CAP8NM7N0
     - [x] Docs shell - we don't need explicit emanote input in nix config
     - [x] Profiled shell:  `recursionzmschemeszm5zi2zi2zi2zm6NC6xZZx2tI3826DrVhqI9v_DataziFunctorziFoldableziTH_zdfMakeBaseFunctorNamezuzdcmakeBaseFunctor_closure` https://hydra.iohk.io/build/15694577/nixlog/2/tail
     - [x] musl cli goldens https://hydra.iohk.io/build/15694618/nixlog/82
     - [x] `hie-bios: hie-bios: this operation requires -fexternal-interpreter
     - [x] `Ambiguous module name 'Options.Applicative':` in HLS - now we at least have an ugly `-XPackageImport` workaround.
- [x] Update node revision in readme.md

### Things we need eventually, but not strictly for this PR

> -  Update to final 1.35 tag once it is released - can be done in separate PR
> -  Run existing integration tests on both Alonzo and Babbage eras in CI
> - Depend on cardano-addresses with aeson2 support merged, tagged and released
> - Depend on a cardano-sl-x509 where aeson 2 support is merged to master
> - Consider appropriateness Tx deserialisation hack 9e2568e

Moved to jira tasks in https://input-output.atlassian.net/browse/ADP-1538

### Comments

Includes #3258, #3293

<!-- Additional comments, links, or screenshots to attach, if any. -->

### Issue Number

ADP-1744

<!-- Reference the Jira/GitHub issue that this PR relates to, and which requirements it tackles.
  Note: Jira issues of the form ADP- will be auto-linked. -->


Co-authored-by: Johannes Lund <[email protected]>
Co-authored-by: Pawel Jakubas <[email protected]>
Co-authored-by: Samuel Evans-Powell <[email protected]>
Co-authored-by: Jonathan Knowles <[email protected]>

Author: 5 people

.buildkite/pipeline.yml

@@ -32,7 +32,6 @@ steps:
     key: nix
     commands:
       - './nix/regenerate.sh'
-      - 'nix build .#checkStackProject -o check-stack-project.sh && ./check-stack-project.sh'
     agents:
       system: x86_64-linux
 

README.md

@@ -71,7 +71,7 @@ See **Installation Instructions** for each available [release](https://github.co
 >
 > | cardano-wallet | cardano-node (compatible versions) | SMASH (compatible versions)
 > | --- | --- | ---
-> | `master` branch | [1.34.1](https://github.com/input-output-hk/cardano-node/releases/tag/1.34.1) | [1.6.1](https://github.com/input-output-hk/smash/releases/tag/1.6.1)
+> | `master` branch | [1.35.0-rc2](https://github.com/input-output-hk/cardano-node/releases/tag/1.35.0-rc2) | [1.6.1](https://github.com/input-output-hk/smash/releases/tag/1.6.1)
 > | [v2022-05-27](https://github.com/input-output-hk/cardano-wallet/releases/tag/v2022-05-27) | [1.34.1](https://github.com/input-output-hk/cardano-node/releases/tag/1.34.1) | [1.6.1](https://github.com/input-output-hk/smash/releases/tag/1.6.1)
 > | [v2022-04-27](https://github.com/input-output-hk/cardano-wallet/releases/tag/v2022-04-27) | [1.34.1](https://github.com/input-output-hk/cardano-node/releases/tag/1.34.1) | [1.6.1](https://github.com/input-output-hk/smash/releases/tag/1.6.1)
 > | [v2022-01-18](https://github.com/input-output-hk/cardano-wallet/releases/tag/v2022-01-18) | [1.33.0](https://github.com/input-output-hk/cardano-node/releases/tag/1.33.0) | [1.6.1](https://github.com/input-output-hk/smash/releases/tag/1.6.1)

cabal.project

@@ -38,7 +38,7 @@
 --
 --------------------------------------------------------------------------------
 
-index-state: 2022-03-29T23:10:49Z
+index-state: 2022-05-18T00:00:00Z
 with-compiler: ghc-8.10.7
 
 packages:
@@ -53,48 +53,60 @@ packages:
   , lib/shelley/
   , lib/strict-non-empty-containers/
 
+-- Using RDRAND instead of /dev/urandom as an entropy source for key
+-- generation is dubious. Set the flag so we use /dev/urandom by default.
+package cryptonite
+  flags: -support_rdrand
+
 -- Using a fork until our patches can be merged upstream
 
 -- TODO: ADP-1713
 source-repository-package
   type: git
   location: https://github.com/biocad/servant-openapi3
   tag: 4165b837d3a71debd1059c3735460075840000b5
+  --sha256: 1dngrr353kjhmwhn0b289jzqz5rf32llwcv79zcyq15ldpqpbib9
 
 -- TODO: ADP-1713
 source-repository-package
   type: git
   location: https://github.com/paolino/openapi3
   tag: c30d0de6875d75edd64d1aac2272886528bc492d
+  --sha256: 0b0fzj5vrnfrc8qikabxhsnp4p8lrjpssblbh2rb7aji5hzzfli9
 
 source-repository-package
   type: git
   location: https://github.com/input-output-hk/optparse-applicative
   tag: 7497a29cb998721a9068d5725d49461f2bba0e7a
+  --sha256: 1gvsrg925vynwgqwplgjmp53vj953qyh3wbdf34pw21c8r47w35r
 
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/Win32-network
     tag: 3825d3abf75f83f406c1f7161883c438dac7277d
+    --sha256: 19wahfv726fa3mqajpqdqhnl9ica3xmf68i254q45iyjcpj1psqx
 
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/cardano-addresses
-    tag: ba8390504b6613957b73bf8d0d2aab1a989f104d
+    tag: b9f424cc64459a95a2f190a1839ec9bc94cc778c
+    --sha256: 0iqzppqqb1xqnp55yd0xil9qfy8fz8474w834nmfg2w692bynyb9
     subdir: command-line
             core
 
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/bech32
     tag: ab61914443e5f53624d3b2995767761b3f68e576
+    --sha256: 0isqh5s6rdhmqa3jhvc32zb3kvzy149hmzddx1ld9f9jhls4f3wg
     subdir: bech32
             bech32-th
 
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/cardano-base
-    tag: 41545ba3ac6b3095966316a99883d678b5ab8da8
+    tag: 631cb6cf1fa01ab346233b610a38b3b4cba6e6ab
+    --sha256: 0944wg2nqazmhlmsynwgdwxxj6ay0hb9qig9l128isb2cjia0hlp
     subdir:
             base-deriving-via
             binary
@@ -111,11 +123,13 @@ source-repository-package
     type: git
     location: https://github.com/input-output-hk/cardano-crypto
     tag: f73079303f663e028288f9f4a9e08bcca39a923e
+    --sha256: 1n87i15x54s0cjkh3nsxs4r1x016cdw1fypwmr68936n3xxsjn6q
 
 source-repository-package
   type: git
   location: https://github.com/input-output-hk/plutus
-  tag: fc5fe94d3d9525d032bcbc79ff0e1ebcfd8ef143
+  tag: d24a7540e4659b57ce2ab25dadb968991e232191
+  --sha256: 15glaghkaa5kh1ayfkw2jahkqq3955kmfzplmb210gvrlnlghlaa
   subdir:
     plutus-core
     plutus-ledger-api
@@ -129,19 +143,24 @@ source-repository-package
   type: git
   location: https://github.com/input-output-hk/ekg-forward
   tag: 297cd9db5074339a2fb2e5ae7d0780debb670c63
+  --sha256: 1zcwry3y5rmd9lgxy89wsb3k4kpffqji35dc7ghzbz603y1gy24g
 
 source-repository-package
   type: git
   location: https://github.com/input-output-hk/cardano-config
   tag: 1646e9167fab36c0bff82317743b96efa2d3adaa
+  --sha256: 11kf65x38laqhwspsl28j2x5a4rky8mfr6356w0li5g53sfykmjc
 
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/cardano-ledger
-    tag: 1a9ec4ae9e0b09d54e49b2a40c4ead37edadcce5
+    tag: e290bf8d0ea272a51e9acd10adc96b4e12e00d37
+    --sha256: 1pmdg80a8irrqgdhbp46a9jx628mjbrj0k89xv5nb5dy37z5ig5f
     subdir:
       eras/alonzo/impl
       eras/alonzo/test-suite
+      eras/babbage/impl
+      eras/babbage/test-suite
       eras/byron/chain/executable-spec
       eras/byron/crypto
       eras/byron/crypto/test
@@ -156,7 +175,7 @@ source-repository-package
       libs/cardano-ledger-pretty
       libs/cardano-protocol-tpraos
       libs/cardano-data
-      libs/compact-map
+      libs/vector-map
       libs/set-algebra
       libs/small-steps
       libs/small-steps-test
@@ -165,7 +184,8 @@ source-repository-package
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/cardano-node
-    tag: 73f9a746362695dc2cb63ba757fbcabb81733d23
+    tag: 95c3692cfbd4cdb82071495d771b23e51840fb0e
+    --sha256: 1s4jjksz185dg4lp36ldha7vccxh0rk8zlvqms00zhg8kla21kr5
     subdir:
       cardano-api
       cardano-git-rev
@@ -180,38 +200,48 @@ source-repository-package
     type: git
     location: https://github.com/input-output-hk/cardano-prelude
     tag: bb4ed71ba8e587f672d06edf9d2e376f4b055555
+    --sha256: 00h10l5mmiza9819p9v5q5749nb9pzgi20vpzpy1d34zmh6gf1cj
     subdir: cardano-prelude
             cardano-prelude-test
 
+-- In particular we rely on the code from this PR:
+--  * <https://github.com/input-output-hk/cardano-sl-x509/pull/6>
+-- being merged.
 source-repository-package
     type: git
-    location: https://github.com/input-output-hk/cardano-sl-x509
-    tag: 12925934c533b3a6e009b61ede555f8f26bac037
+    location: https://github.com/MELD-labs/cardano-sl-x509
+    tag: 819997049b9cc675b28059d3d6f511d08f934d10
+    --sha256: 1ia8vlqghis92cla8qmqa6kh8f3jn29b01fshyk5hmgy5373s684
 
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/goblins
     tag: cde90a2b27f79187ca8310b6549331e59595e7ba
+    --sha256: 17c88rbva3iw82yg9srlxjv2ia5wjb9cyqw44hik565f5v9svnyg
 
+-- In particular we rely on the code from this PR:
+--  * <https://github.com/input-output-hk/iohk-monitoring-framework/pull/622>
+-- being merged.
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/iohk-monitoring-framework
-    tag: 808724ff8a19a33d0ed06f9ef59fbd900b08553c
+    tag: 066f7002aac5a0efc20e49643fea45454f226caa
+    --sha256: 0s6x4in11k5ba7nl7la896g28sznf9185xlqg9c604jqz58vj9nj
     subdir: contra-tracer
             iohk-monitoring
             plugins/backend-aggregation
             plugins/backend-ekg
             plugins/backend-monitoring
             plugins/backend-trace-forwarder
+            plugins/scribe-systemd
             tracer-transformers
 
 source-repository-package
     type: git
     location: https://github.com/input-output-hk/ouroboros-network
-    tag: 4fac197b6f0d2ff60dc3486c593b68dc00969fbf
+    tag: 04245dbd69387da98d3a37de9f400965e922bb0e
+    --sha256: 0bgfclc7h441dwq9z69697nqfir6shj4358zxmwjiaifp93zkc2p
     subdir:
-      io-sim
-      io-classes
       monoidal-synchronisation
       network-mux
       ouroboros-consensus
@@ -222,25 +252,48 @@ source-repository-package
       ouroboros-network
       ouroboros-network-framework
       ouroboros-network-testing
-      strict-stm
-      typed-protocols
-      typed-protocols-cborg
-      typed-protocols-examples
       ntp-client
 
 source-repository-package
-    type: git
-    location: https://github.com/snoyberg/http-client.git
-    tag: 1a75bdfca014723dd5d40760fad854b3f0f37156
-    subdir: http-client
+  type: git
+  location: https://github.com/input-output-hk/io-sim
+  tag: 57e888b1894829056cb00b7b5785fdf6a74c3271
+  --sha256: 1kv8lwmrw1c0g03jy3i3fgk3c8d47ihjcslg295djqj442y95y2f
+  subdir:
+    io-classes
+    io-sim
+    strict-stm
+
+source-repository-package
+  type: git
+  location: https://github.com/input-output-hk/typed-protocols
+  tag: 181601bc3d9e9d21a671ce01e0b481348b3ca104
+  --sha256: 1lr97b2z7l0rpsmmz92rsv27qzd5vavz10cf7n25svya4kkiysp5
+  subdir:
+    typed-protocols
+    typed-protocols-cborg
+    typed-protocols-examples
 
 -- Drops an instance breaking cardano-node.
 source-repository-package
     type: git
-    location: https://github.com/input-output-hk/flat.git
+    location: https://github.com/input-output-hk/flat
     tag: ee59880f47ab835dbd73bea0847dab7869fc20d8
     --sha256: 1lrzknw765pz2j97nvv9ip3l1mcpf2zr4n56hwlz0rk7wq7ls4cm
 
+-- Until https://github.com/tibbe/ekg-json/pull/12 gets merged with aeson2 support
+source-repository-package
+  type: git
+  location: https://github.com/vshabanov/ekg-json
+  tag: 00ebe7211c981686e65730b7144fbf5350462608
+  --sha256: 1zvjm3pb38w0ijig5wk5mdkzcszpmlp5d4zxvks2jk1rkypi8gsm
+
+source-repository-package
+  type: git
+  location: https://github.com/input-output-hk/hedgehog-extras
+  tag: 967d79533c21e33387d0227a5f6cc185203fe658
+  --sha256: 0rbqb7a64aya1qizlr3im06hdydg9zr6sl3i8bvqqlf7kpa647sd
+
 -- -------------------------------------------------------------------------
 -- Constraints tweaking
 
@@ -252,33 +305,45 @@ allow-newer:
     hjsonschema:*
   , hjsonpointer:*
   , cardano-sl-x509:ip
-  , cardano-addresses:aeson
+  , *:aeson
   , size-based:template-haskell
 
+-- Copied from (copied from cardano-node/cabal.project)
+allow-newer:
+  *:aeson,
+  monoidal-containers:aeson,
+  size-based:template-haskell
+
+allow-newer:
+  async-timer:unliftio-core
+
 constraints:
-    hedgehog >= 1.0.2
-  -- QUICK FIX: hedgehog >= 1.1 defines Rec constructor, PlutusIR also defines
-  -- Rec constructor, and imports Hedgehog unqualified, leading to ambiguous
-  -- reference to Rec.
-  -- FIXME: https://input-output.atlassian.net/browse/ADP-1545
-  , hedgehog < 1.1
-  , hashable < 1.4
+    hashable < 1.4
   , bimap >= 0.4.0
+  , openapi3 >= 3.2.0
   , libsystemd-journal >= 1.4.4
   , systemd >= 2.3.0
   -- dependency of systemd-2.3.0
   , network >= 3.1.1.1
   -- choose versions that work with base >= 4.12
   , hjsonpointer >= 1.5.0
   , hjsonschema >= 1.10.0
+  , Cabal >= 3.4.0.0
+  , async-timer >= 0.2.0.0
+  , unliftio-core >= 0.2.0.1
+
+  -- This pin is just here to make cabalProejct == stackProject.
+  -- Remove it if you are happy for a newer version to be picked when possible.
+  , semigroups == 0.19.2
+  , persistent < 2.13.3.4
 
 -- ----------------------------------------------------------------
 -- Flags for dependencies
 
 package cardano-config
-  flags: -systemd
+  flags: +systemd
 package cardano-node
-  flags: -systemd
+  flags: +systemd
 
 -- -------------------------------------------------------------------------
 -- Enable specific tests in this repo