Run Windows node CSI Drivers as HostProcess containers

[mauriciopoppe]

The HostProcess container feature became beta in 1.23, we'd like to leverage it in CSI Drivers which will run as privileged jobs in the Windows host, there'll be more details about the transition steps in the design doc.

By making the CSI Driver a HostProcess pod we no longer need the binary in the client/server model (although we will still support it). There are some items for the maintenance of the current client/server model of CSI Proxy.

Tasks

• Update the .github/workflows scripts to work with both v1.x and master #222
• Update the Prow jobs to work with the v1.x branch and the master branch #223
• Update the integration scripts to work with the v1.x and master branches #224
• Release CSI Proxy v1.1.2 #225
• Design doc: Using Windows HostProcess containers in CSI Drivers running in Windows nodes @mauriciopoppe created CSI Drivers in Windows as HostProcess Pods kubernetes/enhancements#3636
• Blogpost draft: https://docs.google.com/document/d/18WwuKSONG_qkD3vaoppL_ikeDL0Ms0Wwv7HsyN8Oel0/edit

Items for the refactor of CSI Proxy to become a go library:

Tasks

• Refactor the FS API group to become a library #226
• Update the integration tests to run in a HostProcess pod #227
• Document the go structs used by clients when they import CSI Proxy #232
• Refactor the Disk API group to become a library #234
• Refactor the Volume API group to become a library #235
• Refactor the System API group to become a library #236
• Refactor the SMB API group to become a library #237
• Refactor the iSCSI API group to become a library #238
• Document the pkg/<api group>/<api>.go functions used by clients when they import CSI Proxy #239
• Remove v1beta and v1alpha compatibility methods in the server side for the API groups that are GA #241
• Rename API to HostAPI for v2 #242
• Update the README, DEVELOPMENT docs for master & library development #243
• Rename Smb and smb to SMB throughout the codebase #246
• Rename Iscsi to iSCSI throughout the codebase  #248
• Remove v1 integration test code #249
• Remove v1 pkg code and client library #250
• Deprecate filesystem API group #253
• Rename Id to ID #265
• Rename Iqn to IQN in iSCSI #264

Items to use the new CSI Proxy library in a CSI Driver (PD CSI Driver):

Tasks

• Update the Windows node manifests adding the HostProcess Pod fields kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1061
• Refactor the Windows node code to use the CSI Proxy library pkg/api code instead of the client/ code kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1062
• Verify that the E2E jobs are passing with the Windows node component running as a HostProcess Pod kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1063
• Migrate CSI Proxy to v2 kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1070
• Try https://github.com/microsoft/windows-host-process-containers-base-image as a new base image after the POC is completed

/assign @mauriciopoppe

[msau42]

@mauriciopoppe if the work to migrate to hostprocess has not merged yet, can we do a release now with the existing workflow and then create a branch?

[mauriciopoppe]

@msau42 yeah we can do that, I wanted to try to release in the v1.x branch so that if we need to have a new release we would have everything set up (if that's needed although that might not happen). I'll proceed with the release in the master branch.