Merge installation logic into swiftly as an init subcommand

DESIGN.md

@@ -19,19 +19,32 @@ This document contains the high level design of swiftly. Not all features have b
   - [Implementation sketch - macOS](#implementation-sketch---macos)
   - [`config.json` schema](#configjson-schema)
 
-## Linux
+## Installation of swiftly
+
+The installation of swiftly is divided into two phases: delivery and initialization. Delivery of the swiftly binary can be accomplished using different methods:
+
+* Shell "one-liner" with a string of commands that can be copy/pasted into the user's shell to securely download from the trusted website and proceed to initialization
+* Direct download from a trusted website with guidance on the correct binary for the user's platform to download and then how move on to initialization
+* System-level package (e.g. homebrew, pkg, apt-get, rpm) that downloads and places the swiftly binary in a system location outside of the user's home directory, often runnable from the user's path
+* Manual compilation of the swiftly binary from this git repository (e.g. from a development environment)
+
+We'll need an initialization phase, which detects information about the OS and distribution in the case of Linux. The initialization mode is also responsible for setting up the directory structure for the toolchains (if necessary), setting up the shell environment for the user, and determining any operating system level dependencies that are required, but missing. Swiftly has its own configuration stored in the form of a `config.json` file, which will be created as part of initialization. Initialization creates a `env.sh` script that sets the `PATH`, swiftly environment variables `SWIFTLY_HOME_DIR` and `SWIFTLY_BIN_DIR`. The user's profile is modified to source this file and set up the environment for using swiftly.  None of the delivery methods can perform all of these steps on their own. System package managers don't normally update all users' profile or update the user's home directory structure directly.
 
-### Installation of swiftly
+Swiftly can perform these tasks itself with the capabilities provided by the Swift language and libraries, such as rich argument parsing, and launching system processes provided that the binary is delivered to the user. The trigger for the initialization is done via an `init` subcommand with some initialization detection for the other subcommands to help guide users who have gone off track.
+
+```
+swiftly init
+```
 
-We'll need a bootstrapping script which detects information about the OS and downloads the correct pre-built swiftly executable. We can use [rustup-init.sh](https://github.com/rust-lang/rustup/blob/master/rustup-init.sh) as a general guide on implementing such a script, though it is more complicated and supports far more systems than I think we need to. At least for the initial release, I think we'll only need to support the platforms listed on [Swift.org - Getting Started](http://swift.org/getting-started), namely:
-- Ubuntu 16.04
-- Ubuntu 18.04
-- Ubuntu 20.04
-- CentOS 7
-- CentOS 8
-- Amazon Linux 2
+The swiftly binary itself is moved (or copied as a fallback) into the SWIFTLY_BIN_DIR location (or platform default) if it is not run from a system location where it is managed by a system package manager. If the binary could not be moved then the user is notified that they can remove the original.
 
-Once it has detected which platform the user is running, the script will then create `$HOME/.local/share/swiftly` (or a different path, if the user provides one. For an initial MVP, I think we can always install there). It'll also create `$HOME/.local/bin` if needed, download the prebuilt swiftly executable appropriate for the platform, and drop it in there.
+## Updating swiftly
+
+As part of swiftly's regular operations it can detect that the current configuration is out of date and error out. The `config.json` file contains a version at the moment it was created or last upgraded. In the case of an older version it will direct the user to run init to perform the upgrade. If a downgrade situation is detected then swiflty will fail with an error.
+
+There is also a self-update mechanism that will automate the delivery of the new swiftly binary, verifies it and runs the init subcommand to initiate the upgrade procedure. Note that the self-update will error out without performing any operations if swiftly is installed in the system, outside of the SWIFTLY_BIN_DIR (or platform default) and the user's home directory. In any case the self-update will exit successfully right away if it determines that the current swiftly is the latest version and report to the user that it is up-to-date.
+
+## Linux
 
 ### Installation of a Swift toolchain
 
@@ -52,11 +65,6 @@ The `~/.local/bin` directory would include symlinks pointing to the `bin` direct
 This is all very similar to how rustup does things, but I figure there's no need to reinvent the wheel here.
 
 ## macOS
-### Installation of swiftly
-
-Similar to Linux, the bootstrapping script for macOS will create a `~/.local/bin` directory and drop the swiftly executable in it. A `~/.local/share/swiftly/env` file will be created and a message will be printed suggesting users add source `~/.local/share/swiftly/env` to their `.bash_profile` or `.zshrc`.
-
-The bootstrapping script will detect if xcode is installed and prompt the user to install it if it isn’t. We could also ask the user if they’d like us to install the xcode command line tools for them via `xcode-select --install`.
 
 ### Installation of a Swift toolchain
 

Documentation/SwiftlyDocs.docc/SwiftlyDocs.md

@@ -15,6 +15,7 @@ Install and manage your Swift programming language toolchains.
 - <doc:install-toolchains>
 - <doc:uninstall-toolchains>
 - <doc:update-toolchain>
+- <doc:automated-install>
 
 ### Reference
 

Documentation/SwiftlyDocs.docc/automated-install.md

@@ -0,0 +1,66 @@
+# Install Swiftly Automatically
+
+Swiftly can be installed automatically in places like build/CI systems.
+
+This guide will help you to script to the installation of swiftly and toolchains so that it can be unattended. We assume that you have working understanding of your build system. The examples are based on a typical Unix environment.
+
+First, download a swiftly binary from a trusted source, such as your artifact repository, or a well-known website for the operating system (e.g. Linux) and processor architecture (e.g. arm64, or x86_64). Here's an example using the popular curl command.
+
+```
+curl -L <trusted_location_of_swiftly> > swiftly
+```
+
+> Tip: If you are using Linux you will need the "ca-certificates" package for the root certificate authorities that will establish the trust that swiftly needs to make API requests that it needs. This package is frequently pre-installed on end-user environments, but may not be present in more minimal installations.
+
+Once swiftly is downloaded you can run the init subcommand to finish the installation. This command will use the default initialization options and proceed without prompting.
+
+```
+./swiftly init --assume-yes
+```
+
+Swiftly is installed, but the current shell may not yet be updated with the new environment variables, such as the PATH. The init command prints instructions on how to update the current shell environment without opening a new shell. This is an example of the output taken from Linux, but the details might be different for other OSes (and users):
+
+```
+To begin using installed swiftly from your current shell, first run the following command:
+
+. "/root/.local/share/swiftly/env.sh"
+```
+
+> Note: on macOS systems you may need to run 'hash -r' to recalcuate the zsh PATH cache when installing swiftly and toolchains.
+
+You can go ahead and add this command to the list of commands in your build script so that the build can proceed to call swiftly from the path. The usual next step is to install a specific swift toolchain like this:
+
+```
+swiftly install 5.10.1 --post-install-file=post-install.sh
+```
+
+It's possible that there will be some post-installation steps to prepare the build system for using the swift toolchain. The `post-install-file` option gives a file, post-install.sh, that is created if there are post installation steps for this toolchain. You can check if the file exists and run it to perform those final steps. If the build runs as the root user you can check it and run it like this in a typical Unix shell:
+
+```
+if [ -f post-install.sh ]; then
+    . post-install.sh
+fi
+```
+
+> Note: If the build system runs your script as a regular user then you will need to take this into account by either pre-installing the toolchain's system dependencies or running the `post-install.sh` script in a secure manner as the administrative user.
+
+If you want to install swiftly, or the binaries that it manages into different locations these can be customized using environment variables before running `swiftly init`.
+
+```
+SWIFTLY_HOME_DIR - The location of the swiftly configuration files, and environment scripts
+SWIFTLY_BIN_DIR - The location of the swiftly binary and toolchain symbolic links (e.g. swift, swiftc, etc.)
+```
+
+Sometimes the build system platform can't be automatically detected, or isn't supported by swift. You can provide the platform as an option to the swiftly init subcommand:
+
+```
+swiftly init --platform=<platform_name>
+```
+
+There are other customizable options, such as overwrite. For more details about the available options, check the help:
+
+```
+swiftly init --help
+```
+
+In summary, swiftly can be installed and install toolchains unattended on build and CI-style systems. This HOWTO guide has outlined the process to script the process covering some of the different options available to you.

Documentation/SwiftlyDocs.docc/getting-started.md

@@ -6,6 +6,12 @@ To download swiftly and install Swift, run the following in your terminal, then
 curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash
 ```
 
+Alternatively, you can download the swiftly binary and install itself like this:
+
+```
+swiftly init
+```
+
 Once swiftly is installed you can use it to install the latest available swift toolchain like this:
 
 ```

Documentation/SwiftlyDocs.docc/install-toolchains.md

@@ -10,6 +10,8 @@ The easiest way to install a swift toolchain is to select the latest stable rele
 $ swiftly install latest
 ```
 
+> Note: After you install a toolchain there may be certain system dependencies that are needed. Swiftly will provide instructions.
+
 If this is the only toolchain that is installed then swiftly will automatically "use" it so that when you run swift (or any other toolchain command) it will be this version.
 
 ```

Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md

@@ -23,7 +23,7 @@ swiftly [--version] [--help]
 Install a new toolchain.
 
 ```
-swiftly install <version> [--use] [--token=<token>] [--verify] [--version] [--help]
+swiftly install <version> [--use] [--token=<token>] [--verify] [--post-install-file=<post-install-file>] [--version] [--help]
 ```
 
 **version:**
@@ -73,6 +73,14 @@ fails with an "unauthorized" status code, it likely means the rate limit has bee
 *Verify the toolchain's PGP signature before proceeding with installation.*
 
 
+**--post-install-file=\<post-install-file\>:**
+
+*A file path to a location for a post installation script*
+
+If the toolchain that is installed has extra post installation steps they they will be
+written to this file as commands that can be run after the installation.
+
+
 **--version:**
 
 *Show the version.*
@@ -178,7 +186,7 @@ Finally, all installed toolchains can be uninstalled by specifying 'all':
 
 **--assume-yes:**
 
-*Uninstall all selected toolchains without prompting for confirmation.*
+*Disable confirmation prompts by assuming 'yes'*
 
 
 **--version:**
@@ -241,7 +249,7 @@ The installed snapshots for a given devlopment branch can be listed by specifyin
 Update an installed toolchain to a newer version.
 
 ```
-swiftly update [<toolchain>] [--assume-yes] [--verify] [--version] [--help]
+swiftly update [<toolchain>] [--assume-yes] [--verify] [--post-install-file=<post-install-file>] [--version] [--help]
 ```
 
 **toolchain:**
@@ -284,14 +292,62 @@ A specific snapshot toolchain can be updated by including the date:
 
 **--assume-yes:**
 
-*Update the selected toolchains without prompting for confirmation.*
+*Disable confirmation prompts by assuming 'yes'*
 
 
 **--verify:**
 
 *Verify the toolchain's PGP signature before proceeding with installation.*
 
 
+**--post-install-file=\<post-install-file\>:**
+
+*A file path to a location for a post installation script*
+
+If the toolchain that is installed has extra post installation steps they they will be
+written to this file as commands that can be run after the installation.
+
+
+**--version:**
+
+*Show the version.*
+
+
+**--help:**
+
+*Show help information.*
+
+
+
+
+## init
+
+Perform swiftly initialization into your user account.
+
+```
+swiftly init [--no-modify-profile] [--overwrite] [--platform=<platform>] [--assume-yes] [--version] [--help]
+```
+
+**--no-modify-profile:**
+
+*Do not attempt to modify the profile file to set environment variables (e.g. PATH) on login.*
+
+
+**--overwrite:**
+
+*Overwrite the existing swiftly installation found at the configured SWIFTLY_HOME, if any. If this option is unspecified and an existing installation is found, the swiftly executable will be updated, but the rest of the installation will not be modified.*
+
+
+**--platform=\<platform\>:**
+
+*Specify the current Linux platform for swiftly.*
+
+
+**--assume-yes:**
+
+*Disable confirmation prompts by assuming 'yes'*
+
+
 **--version:**
 
 *Show the version.*

Package.swift

@@ -3,6 +3,7 @@
 import PackageDescription
 
 let ghApiCacheResources = (1...16).map { Resource.embedInCode("gh-api-cache/swift-tags-page\($0).json") }
+let ghApiCacheExcludedResources = (17...27).map { "gh-api-cache/swift-tags-page\($0).json" }
 
 let package = Package(
     name: "swiftly",
@@ -90,6 +91,7 @@ let package = Package(
         .testTarget(
             name: "SwiftlyTests",
             dependencies: ["Swiftly"],
+            exclude: ghApiCacheExcludedResources,
             resources: ghApiCacheResources + [
                 .embedInCode("gh-api-cache/swift-releases-page1.json"),
                 .embedInCode("mock-signing-key-private.pgp"),

README.md

@@ -11,6 +11,11 @@ To download swiftly and install Swift, run the following in your terminal, then
 curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash
 ```
 
+Alternatively, you can download the swiftly binary and it can install itself:
+```
+swiftly init
+```
+
 ### Basic usage
 
 ```