Skip to content

Commit eba9bd9

Browse files
authored
Merge installation logic into swiftly as an init subcommand (#127)
Merge installation logic into swiftly as an init subcommand Instead of relying on a separate shell script with certain issues swiftly can perform its own user initialization with many of the features of the script, including prompts, flags, and options. It also prepares swiftly for possible delivery through system package managers. Provide a design for the new installation system in the DESIGN.md. Bump the Swiftly version to 0.4.0 with the dev suffix to signal a new development version of swiftly that can be compiled and used to install for early adoption and testing. Create a new init subcommand that performs the post-download init procedure for the user. Perform a validation whether swiftly is initialized at the beginning of each other subcommand and also check its version. The version is now encoded in the config.json file in order to detect upgrade or attempted downgrade. Update the install command so that it is capable of reporting missing system packages, providing the command that the user can run in order to install them. Add a special post installation file option that will provide the command in a file that can be run after installation to install the system packages. Add post installation script generation options to Platform protocol. Add platform (aka. distribution) detection capabilities to the Linux platform, and create the package lists that are needed for each distribution. Use the distribution-specific list and detection commands to decide if the packages are installed in the user's system so that a tailored post-installation script is generated, or none at all if the user's system is ready for the toolchain. Create a HOWTO guide to demonstrate how to install swiftly and toolchains unattended in places like CI systems. Highlight the various concerns that should be considered in the process of automating it. Since the platform definition auto detection is added to the init subcommand update the test harness so that it is capable of auto detecting the platform instead of relying on the environment variables. The environment variables remain as an override for testing purposes.
1 parent 9185e5a commit eba9bd9

30 files changed

+1330
-215
lines changed

DESIGN.md

Lines changed: 23 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -19,19 +19,32 @@ This document contains the high level design of swiftly. Not all features have b
1919
- [Implementation sketch - macOS](#implementation-sketch---macos)
2020
- [`config.json` schema](#configjson-schema)
2121

22-
## Linux
22+
## Installation of swiftly
23+
24+
The installation of swiftly is divided into two phases: delivery and initialization. Delivery of the swiftly binary can be accomplished using different methods:
25+
26+
* Shell "one-liner" with a string of commands that can be copy/pasted into the user's shell to securely download from the trusted website and proceed to initialization
27+
* Direct download from a trusted website with guidance on the correct binary for the user's platform to download and then how move on to initialization
28+
* System-level package (e.g. homebrew, pkg, apt-get, rpm) that downloads and places the swiftly binary in a system location outside of the user's home directory, often runnable from the user's path
29+
* Manual compilation of the swiftly binary from this git repository (e.g. from a development environment)
30+
31+
We'll need an initialization phase, which detects information about the OS and distribution in the case of Linux. The initialization mode is also responsible for setting up the directory structure for the toolchains (if necessary), setting up the shell environment for the user, and determining any operating system level dependencies that are required, but missing. Swiftly has its own configuration stored in the form of a `config.json` file, which will be created as part of initialization. Initialization creates a `env.sh` script that sets the `PATH`, swiftly environment variables `SWIFTLY_HOME_DIR` and `SWIFTLY_BIN_DIR`. The user's profile is modified to source this file and set up the environment for using swiftly. None of the delivery methods can perform all of these steps on their own. System package managers don't normally update all users' profile or update the user's home directory structure directly.
2332

24-
### Installation of swiftly
33+
Swiftly can perform these tasks itself with the capabilities provided by the Swift language and libraries, such as rich argument parsing, and launching system processes provided that the binary is delivered to the user. The trigger for the initialization is done via an `init` subcommand with some initialization detection for the other subcommands to help guide users who have gone off track.
34+
35+
```
36+
swiftly init
37+
```
2538

26-
We'll need a bootstrapping script which detects information about the OS and downloads the correct pre-built swiftly executable. We can use [rustup-init.sh](https://github.com/rust-lang/rustup/blob/master/rustup-init.sh) as a general guide on implementing such a script, though it is more complicated and supports far more systems than I think we need to. At least for the initial release, I think we'll only need to support the platforms listed on [Swift.org - Getting Started](http://swift.org/getting-started), namely:
27-
- Ubuntu 16.04
28-
- Ubuntu 18.04
29-
- Ubuntu 20.04
30-
- CentOS 7
31-
- CentOS 8
32-
- Amazon Linux 2
39+
The swiftly binary itself is moved (or copied as a fallback) into the SWIFTLY_BIN_DIR location (or platform default) if it is not run from a system location where it is managed by a system package manager. If the binary could not be moved then the user is notified that they can remove the original.
3340

34-
Once it has detected which platform the user is running, the script will then create `$HOME/.local/share/swiftly` (or a different path, if the user provides one. For an initial MVP, I think we can always install there). It'll also create `$HOME/.local/bin` if needed, download the prebuilt swiftly executable appropriate for the platform, and drop it in there.
41+
## Updating swiftly
42+
43+
As part of swiftly's regular operations it can detect that the current configuration is out of date and error out. The `config.json` file contains a version at the moment it was created or last upgraded. In the case of an older version it will direct the user to run init to perform the upgrade. If a downgrade situation is detected then swiflty will fail with an error.
44+
45+
There is also a self-update mechanism that will automate the delivery of the new swiftly binary, verifies it and runs the init subcommand to initiate the upgrade procedure. Note that the self-update will error out without performing any operations if swiftly is installed in the system, outside of the SWIFTLY_BIN_DIR (or platform default) and the user's home directory. In any case the self-update will exit successfully right away if it determines that the current swiftly is the latest version and report to the user that it is up-to-date.
46+
47+
## Linux
3548

3649
### Installation of a Swift toolchain
3750

@@ -52,11 +65,6 @@ The `~/.local/bin` directory would include symlinks pointing to the `bin` direct
5265
This is all very similar to how rustup does things, but I figure there's no need to reinvent the wheel here.
5366

5467
## macOS
55-
### Installation of swiftly
56-
57-
Similar to Linux, the bootstrapping script for macOS will create a `~/.local/bin` directory and drop the swiftly executable in it. A `~/.local/share/swiftly/env` file will be created and a message will be printed suggesting users add source `~/.local/share/swiftly/env` to their `.bash_profile` or `.zshrc`.
58-
59-
The bootstrapping script will detect if xcode is installed and prompt the user to install it if it isn’t. We could also ask the user if they’d like us to install the xcode command line tools for them via `xcode-select --install`.
6068

6169
### Installation of a Swift toolchain
6270

Documentation/SwiftlyDocs.docc/SwiftlyDocs.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Install and manage your Swift programming language toolchains.
1515
- <doc:install-toolchains>
1616
- <doc:uninstall-toolchains>
1717
- <doc:update-toolchain>
18+
- <doc:automated-install>
1819

1920
### Reference
2021

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Install Swiftly Automatically
2+
3+
Swiftly can be installed automatically in places like build/CI systems.
4+
5+
This guide will help you to script to the installation of swiftly and toolchains so that it can be unattended. We assume that you have working understanding of your build system. The examples are based on a typical Unix environment.
6+
7+
First, download a swiftly binary from a trusted source, such as your artifact repository, or a well-known website for the operating system (e.g. Linux) and processor architecture (e.g. arm64, or x86_64). Here's an example using the popular curl command.
8+
9+
```
10+
curl -L <trusted_location_of_swiftly> > swiftly
11+
```
12+
13+
> Tip: If you are using Linux you will need the "ca-certificates" package for the root certificate authorities that will establish the trust that swiftly needs to make API requests that it needs. This package is frequently pre-installed on end-user environments, but may not be present in more minimal installations.
14+
15+
Once swiftly is downloaded you can run the init subcommand to finish the installation. This command will use the default initialization options and proceed without prompting.
16+
17+
```
18+
./swiftly init --assume-yes
19+
```
20+
21+
Swiftly is installed, but the current shell may not yet be updated with the new environment variables, such as the PATH. The init command prints instructions on how to update the current shell environment without opening a new shell. This is an example of the output taken from Linux, but the details might be different for other OSes, username, or shell.
22+
23+
```
24+
To begin using installed swiftly from your current shell, first run the following command:
25+
26+
. "/root/.local/share/swiftly/env.sh"
27+
```
28+
29+
> Note: on macOS systems you may need to run 'hash -r' to recalcuate the zsh PATH cache when installing swiftly and toolchains.
30+
31+
You can go ahead and add this command to the list of commands in your build script so that the build can proceed to call swiftly from the path. The usual next step is to install a specific swift toolchain like this:
32+
33+
```
34+
swiftly install 5.10.1 --post-install-file=post-install.sh
35+
```
36+
37+
It's possible that there will be some post-installation steps to prepare the build system for using the swift toolchain. The `post-install-file` option gives a file, post-install.sh, that is created if there are post installation steps for this toolchain. You can check if the file exists and run it to perform those final steps. If the build runs as the root user you can check it and run it like this in a typical Unix shell:
38+
39+
```
40+
if [ -f post-install.sh ]; then
41+
. post-install.sh
42+
fi
43+
```
44+
45+
> Note: If the build system runs your script as a regular user then you will need to take this into account by either pre-installing the toolchain's system dependencies or running the `post-install.sh` script in a secure manner as the administrative user.
46+
47+
If you want to install swiftly, or the binaries that it manages into different locations these can be customized using environment variables before running `swiftly init`.
48+
49+
```
50+
SWIFTLY_HOME_DIR - The location of the swiftly configuration files, and environment scripts
51+
SWIFTLY_BIN_DIR - The location of the swiftly binary and toolchain symbolic links (e.g. swift, swiftc, etc.)
52+
```
53+
54+
Sometimes the build system platform can't be automatically detected, or isn't supported by swift. You can provide the platform as an option to the swiftly init subcommand:
55+
56+
```
57+
swiftly init --platform=<platform_name>
58+
```
59+
60+
There are other customizable options, such as overwrite. For more details about the available options, check the help:
61+
62+
```
63+
swiftly init --help
64+
```
65+
66+
In summary, swiftly can be installed and install toolchains unattended on build and CI-style systems. This HOWTO guide has outlined the process to script the process covering some of the different options available to you.

Documentation/SwiftlyDocs.docc/getting-started.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,12 @@ To download swiftly and install Swift, run the following in your terminal, then
66
curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash
77
```
88

9+
Alternatively, you can download the swiftly binary and install itself like this:
10+
11+
```
12+
swiftly init
13+
```
14+
915
Once swiftly is installed you can use it to install the latest available swift toolchain like this:
1016

1117
```

Documentation/SwiftlyDocs.docc/install-toolchains.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ The easiest way to install a swift toolchain is to select the latest stable rele
1010
$ swiftly install latest
1111
```
1212

13+
> Note: After you install a toolchain there may be certain system dependencies that are needed. Swiftly will provide instructions.
14+
1315
If this is the only toolchain that is installed then swiftly will automatically "use" it so that when you run swift (or any other toolchain command) it will be this version.
1416

1517
```

Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md

Lines changed: 60 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ swiftly [--version] [--help]
2323
Install a new toolchain.
2424

2525
```
26-
swiftly install <version> [--use] [--token=<token>] [--verify] [--version] [--help]
26+
swiftly install <version> [--use] [--token=<token>] [--verify] [--post-install-file=<post-install-file>] [--version] [--help]
2727
```
2828

2929
**version:**
@@ -73,6 +73,14 @@ fails with an "unauthorized" status code, it likely means the rate limit has bee
7373
*Verify the toolchain's PGP signature before proceeding with installation.*
7474

7575

76+
**--post-install-file=\<post-install-file\>:**
77+
78+
*A file path to a location for a post installation script*
79+
80+
If the toolchain that is installed has extra post installation steps they they will be
81+
written to this file as commands that can be run after the installation.
82+
83+
7684
**--version:**
7785

7886
*Show the version.*
@@ -178,7 +186,7 @@ Finally, all installed toolchains can be uninstalled by specifying 'all':
178186

179187
**--assume-yes:**
180188

181-
*Uninstall all selected toolchains without prompting for confirmation.*
189+
*Disable confirmation prompts by assuming 'yes'*
182190

183191

184192
**--version:**
@@ -241,7 +249,7 @@ The installed snapshots for a given devlopment branch can be listed by specifyin
241249
Update an installed toolchain to a newer version.
242250

243251
```
244-
swiftly update [<toolchain>] [--assume-yes] [--verify] [--version] [--help]
252+
swiftly update [<toolchain>] [--assume-yes] [--verify] [--post-install-file=<post-install-file>] [--version] [--help]
245253
```
246254

247255
**toolchain:**
@@ -284,14 +292,62 @@ A specific snapshot toolchain can be updated by including the date:
284292

285293
**--assume-yes:**
286294

287-
*Update the selected toolchains without prompting for confirmation.*
295+
*Disable confirmation prompts by assuming 'yes'*
288296

289297

290298
**--verify:**
291299

292300
*Verify the toolchain's PGP signature before proceeding with installation.*
293301

294302

303+
**--post-install-file=\<post-install-file\>:**
304+
305+
*A file path to a location for a post installation script*
306+
307+
If the toolchain that is installed has extra post installation steps they they will be
308+
written to this file as commands that can be run after the installation.
309+
310+
311+
**--version:**
312+
313+
*Show the version.*
314+
315+
316+
**--help:**
317+
318+
*Show help information.*
319+
320+
321+
322+
323+
## init
324+
325+
Perform swiftly initialization into your user account.
326+
327+
```
328+
swiftly init [--no-modify-profile] [--overwrite] [--platform=<platform>] [--assume-yes] [--version] [--help]
329+
```
330+
331+
**--no-modify-profile:**
332+
333+
*Do not attempt to modify the profile file to set environment variables (e.g. PATH) on login.*
334+
335+
336+
**--overwrite:**
337+
338+
*Overwrite the existing swiftly installation found at the configured SWIFTLY_HOME, if any. If this option is unspecified and an existing installation is found, the swiftly executable will be updated, but the rest of the installation will not be modified.*
339+
340+
341+
**--platform=\<platform\>:**
342+
343+
*Specify the current Linux platform for swiftly.*
344+
345+
346+
**--assume-yes:**
347+
348+
*Disable confirmation prompts by assuming 'yes'*
349+
350+
295351
**--version:**
296352

297353
*Show the version.*

Package.swift

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
import PackageDescription
44

55
let ghApiCacheResources = (1...16).map { Resource.embedInCode("gh-api-cache/swift-tags-page\($0).json") }
6+
let ghApiCacheExcludedResources = (17...27).map { "gh-api-cache/swift-tags-page\($0).json" }
67

78
let package = Package(
89
name: "swiftly",
@@ -90,6 +91,7 @@ let package = Package(
9091
.testTarget(
9192
name: "SwiftlyTests",
9293
dependencies: ["Swiftly"],
94+
exclude: ghApiCacheExcludedResources,
9395
resources: ghApiCacheResources + [
9496
.embedInCode("gh-api-cache/swift-releases-page1.json"),
9597
.embedInCode("mock-signing-key-private.pgp"),

README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,11 @@ To download swiftly and install Swift, run the following in your terminal, then
1111
curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash
1212
```
1313

14+
Alternatively, you can download the swiftly binary and it can install itself:
15+
```
16+
swiftly init
17+
```
18+
1419
### Basic usage
1520

1621
```

0 commit comments

Comments
 (0)