[MDEPLOY-206] Support parallel deployment at end (needs resolver changes as well)

[jira-importer]

Matt Nelson opened MDEPLOY-206 and commented

When deploying at end with a large reactor the deployment process can take some time. If the DeployMojo was updated to support a configuration value for the amount of parallelism the deployment could be speed up substantially.

The change in m-deploy-p is nearly trivial (create one DeployRequest with everything), but once MRESOLVER-319 in place (planned to be part of Maven 3.9.1), parallel deploy w/ deployAtEnd will be handled by resolver.

---

Issue Links:

• MDEPLOY-204 Must deploy maven-metadata.xml last
  ("is blocked by")
• MRESOLVER-32 Support parallel artifact/metadata uploads

Remote Links:

• GitHub Pull Request #35
  
• GitHub Pull Request #38
  
• GitHub Pull Request #5
  
• GH PR #38
  

3 votes, 5 watchers

[jira-importer]

Matt Nelson commented

Adding a blocker to MDEPLOY-204 since a parallel deployment would further exacerbate the potential for uploading the artifacts out of order.

[jira-importer]

Matt Nelson commented

How is the scope of this issue changed with the maven-3.5.0 conversion to maven-resolver?

I am using parallel builds with the deployAtEnd flag on a reactor with 40+ modules. Holding onto all the artifacts until the end there is a huge backlog of IO operations that are taking 33% of the build time. Some of that time could be attributed to the resources on the repository, but there is a huge potential for decreased build times by performing the uploads with any amount of parallelism. For each module there are 6 uploads and 1 download. The <version>/maven-metadata.xml appears to be uploaded twice

15:25:19 [INFO] Uploaded: http://repo/com/company/project/my-artifact/0.15-SNAPSHOT/my-artifact-0.15-20170516.202515-32.jar (30689 KB at 7579.2 KB/sec)
15:25:21 [INFO] Uploaded: http://repo/com/company/project/my-artifact/0.15-SNAPSHOT/my-artifact-0.15-20170516.202515-32.pom (10 KB at 5.7 KB/sec)
15:25:21 [INFO] Downloaded: http://repo/com/company/project/my-artifact/maven-metadata.xml (420 B at 3.5 KB/sec)
15:25:22 [INFO] Uploaded: http://repo/com/company/project/my-artifact/0.15-SNAPSHOT/maven-metadata.xml (988 B at 1.4 KB/sec)
15:25:24 [INFO] Uploaded: http://repo/com/company/project/my-artifact/maven-metadata.xml (420 B at 0.2 KB/sec)
15:25:24 [INFO] Uploaded: http://repo/com/company/project/my-artifact/0.15-SNAPSHOT/my-artifact-0.15-20170516.202515-32-sources.jar (85 KB at 187.9 KB/sec)
15:25:25 [INFO] Uploaded: http://repo/com/company/project/my-artifact/0.15-SNAPSHOT/maven-metadata.xml (988 B at 1.7 KB/sec)

[jira-importer]

Matt Nelson commented

Once I got into the implementation details of this plugin, I realized that MDEPLOY-204 is not a blocker. The parallelism is batched by project instead of by distinct upload request. As such the upload order per module is unchanged.

[jira-importer]

Tamas Cservenak commented

Added link to GH PR #38 as it adds the needed bits (and simplifies) deploy plugin, while resolver will receive needed stuff as part of MRESOLVER-32

[jira-importer]

Tamas Cservenak commented

Merged PR as 2ff9a2e