Improvement to build, install, and distribution mechanisms

[sdavtaker]

Describe the issue

Feedback from our customers show that AWS SDK for C++ complexity to build is high. Our customers come from very diverse disciplines and backgrounds, that requires AWS SDK for C++ to build under a vast set of constraints. These constraints usually come from using different operating systems, toolchains, memory management processes, and domain specific practices to mention a few.

Since AWS SDK for C++ was first released, we extended our cmake scripts to fulfill needs from new platforms, new uses cases we identified, and new tooling mostly as a collection of small individual changes with no significant impact on the build as a whole. However, as the collection of them grew, the build as a whole became a very complex piece of code to work with. This building complexity many times translates in time spent for developers who want to install the AWS SDK for C++ to work in their projects, for contributors and reviewers wanting to implement changes to fix issues with it, for package managers to release distributions, for release managers to distribute it as a dependency of their projects.

We are working in a redesign of our cmake scripts to be released at v1.10. We expect to provide a simpler and idiomatic modern cmake experience based in standard industry patterns.
Here is a list of some of the changes we will introduce:

• Implement superbuild pattern.
• Remove the need to pull dependencies via git submodules.
• Restructure project directories to easily distinct between generated-src and src.
• Provide CPack targets.
• Declaring dependencies closer to their targets so they are not required when not used. For example, no test tools will be required when tests are not expected to be run.
• Make configuration parameters names reflect better cmake community practices.
• A legacy mode that will mantain backward compatibility with current build scripts for allowing an easy transition to customers.
• Simplify toolchain customizations.
• Set cmake_minimum_required to 3.12 version.

We will develop these changes in branch build-dev-1.10 until ready for release. We are going to publish PRs in each step of the way and link them to this Issue. We encourage everyone interested to participate in this Issue discussion thread, and the PRs.

Steps to Reproduce

No response

Current behavior

No response

AWS CPP SDK version used

1.9

compiler and version used

any

Operating System and version

any

[sdavtaker]

First preparation PR: #1886

[kaihirota]

I'm trying to use DynamoDB Streams and Kinesis within Unreal Engine, and the process is extremely complex and painful. Of course, part of it is also due to UE's fault, but that's another story. Anyway, to use a third party library within Unreal Engine, I have to package the library as a Unreal Engine "plugin" that contains a "third party library" folder which includes compiled and source code for Windows, Mac, etc. Like many people, I could not figure out how to make this work. Fortunately, I was able to purchase a plugin for a fee from the UE marketplace, and the plugin's sole value proposition is enabling users to use the AWS SDK (but only dynamodb submodule) within UE.

While I'm sure this represents a niche and rare use case of AWS SDK, this is one area which would benefit largely from easier integration of AWS.