Merge #385

385: First bits of the second edition r=phil-opp a=phil-opp

This PR adds the first two posts for the second edition, “A Freestanding Rust Binary” and “A Minimal Rust Kernel”. The largest changes in comparison to the first edition are:

- Instead of GRUB, we use our own [bootloader](https://github.com/rust-osdev/bootloader) (written in Rust) and our [bootimage](https://github.com/rust-osdev/bootimage) tool. This removes the dependencies on GRUB and `nasm`. Note that both tools are still experimental and might contain bugs.
- Support for Windows and Mac: Without GRUB, there's nothing preventing us from building on Windows or Mac OS anymore! We added additional CI jobs at travis and appveyor to ensure that the project always builds on all three platforms. (At the moment, users still need to install LLD, but with the [LLVM 6 update of rustc](rust-lang/rust#47828) we should have a builtin LLD soon.)
- No assembly in the main posts. Instead, we're creating a `no_std` _executable_ and relying on our custom bootloader to perform the initial page mapping and the switch to long mode.
- No linker script: Our bootloader loads the kernel in a clean address space, so we can just use the default executable layout and don't need a linker script. (This also means that users that want a higher half kernel just need to update the mapping in their linker script. However, I'm not sure if it's a good idea to add this to the blog. Maybe later, when we begin to run user programs.)

These changes only land in “beta mode” with this PR, which means that they're not linked from the front page yet. We will do that in a follow up PR.

Author: bors[bot]

.appveyor.yml

@@ -0,0 +1,82 @@
+branches:
+  except:
+    # Used by bors
+    - staging.tmp
+
+# Appveyor configuration template for Rust using rustup for Rust installation
+# https://github.com/starkat99/appveyor-rust
+
+## Operating System (VM environment) ##
+
+# Rust needs at least Visual Studio 2013 Appveyor OS for MSVC targets.
+os: Visual Studio 2015
+
+## Build Matrix ##
+environment:
+  matrix:
+
+### MSVC Toolchains ###
+  # Nightly 64-bit MSVC
+    - channel: nightly
+      target: x86_64-pc-windows-msvc
+      MSYS_BITS: 64
+  # Nightly 32-bit MSVC
+    - channel: nightly
+      target: i686-pc-windows-msvc
+      MSYS_BITS: 32
+
+### GNU Toolchains ###
+  # Nightly 64-bit GNU
+    - channel: nightly
+      target: x86_64-pc-windows-gnu
+      MSYS_BITS: 64
+  # Nightly 32-bit GNU
+    - channel: nightly
+      target: i686-pc-windows-gnu
+      MSYS_BITS: 32
+
+cache:
+  - '%USERPROFILE%\.cargo\bin'
+  - '%USERPROFILE%\.cargo\config'
+  - '%USERPROFILE%\.cargo\env'
+  - '%USERPROFILE%\.cargo\.crates.toml'
+  - '%USERPROFILE%\.xargo'
+  - target
+
+## Install Script ##
+
+# This is the most important part of the Appveyor configuration. This installs the version of Rust
+# specified by the 'channel' and 'target' environment variables from the build matrix. This uses
+# rustup to install Rust.
+#
+# For simple configurations, instead of using the build matrix, you can simply set the
+# default-toolchain and default-host manually here.
+install:
+  - appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
+  - rustup-init -yv --default-toolchain %channel% --default-host %target%
+  - set PATH=%PATH%;%USERPROFILE%\.cargo\bin
+  - rustc -vV
+  - cargo -vV
+
+## Build Script ##
+
+# 'cargo test' takes care of building for us, so disable Appveyor's build stage. This prevents
+# the "directory does not contain a project or solution file" error.
+build: false
+
+before_test:
+  - mkdir bin
+  - mklink "bin\ld.exe" "C:\Program Files\LLVM\bin\lld.exe"
+  - set PATH=%CD%\bin;C:\msys64\mingw%MSYS_BITS%\bin;C:\msys64\usr\bin;%PATH%
+  - rustup component add rust-src
+  - set RUST_BACKTRACE=1
+  - if not exist %USERPROFILE%\.cargo\bin\cargo-install-update.exe cargo install cargo-update
+  - if not exist %USERPROFILE%\.cargo\bin\xargo.exe cargo install xargo
+  - if not exist %USERPROFILE%\.cargo\bin\bootimage.exe cargo install bootimage
+  - cargo install-update -a
+
+# Uses 'cargo test' to run tests and build. Alternatively, the project may call compiled programs
+# directly or perform other testing commands. Rust will automatically be placed in the PATH
+# environment variable.
+test_script:
+  - bootimage --target x86_64-blog_os

.travis.yml

@@ -1,31 +1,53 @@
 language: rust
 
+sudo: false
+
+notifications:
+    email:
+        on_success: never
+        on_failure: change
+
+branches:
+  except:
+    # Used by bors
+    - staging.tmp
+
 rust:
     - nightly
 
+os:
+  - linux
+  - osx
+
+addons:
+  apt:
+    sources:
+      - llvm-toolchain-trusty-5.0
+    packages:
+      - lld-5.0
+
 cache:
   directories:
-     - $HOME/.cargo
-     - $HOME/.xargo
-     - $TRAVIS_BUILD_DIR/target
+    - $HOME/.cargo
+    - $HOME/.xargo
+    - $TRAVIS_BUILD_DIR/target
+
+before_install:
+  - |
+    if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
+      set -e
+      LLVM_URL="https://releases.llvm.org/5.0.1/clang+llvm-5.0.1-x86_64-apple-darwin.tar.xz"
+      travis_retry wget -O llvm.tar.xz -nv ${LLVM_URL}
+      tar -xJ -f llvm.tar.xz
+      export PATH="`pwd`/clang+llvm-5.0.1-final-x86_64-apple-darwin/bin:$PATH"
+    fi
 
 before_script:
   - rustup component add rust-src
   - (test -x $HOME/.cargo/bin/cargo-install-update || cargo install cargo-update)
   - (test -x $HOME/.cargo/bin/xargo || cargo install xargo)
+  - (test -x $HOME/.cargo/bin/bootimage || cargo install bootimage)
   - cargo install-update -a
 
-sudo: false
-
-notifications:
-    email:
-        on_success: never
-        on_failure: change
-
-addons:
-  apt:
-    packages:
-      - nasm
-
 script:
-  - make
+  - bootimage --target x86_64-blog_os

Cargo.toml

@@ -1,22 +1,14 @@
 [package]
 authors = ["Philipp Oppermann <[email protected]>"]
 name = "blog_os"
-version = "0.1.0"
+version = "0.2.0"
 
 [dependencies]
-bit_field = "0.7.0"
-bitflags = "0.9.1"
-multiboot2 = "0.1.0"
-once = "0.3.2"
-rlibc = "1.0"
-spin = "0.4.5"
-volatile = "0.1.0"
-x86_64 = "0.1.2"
-linked_list_allocator = "0.4.2"
 
-[dependencies.lazy_static]
-features = ["spin_no_std"]
-version = "0.2.1"
+# the profile used for `cargo build`
+[profile.dev]
+panic = "abort" # disable stack unwinding on panic
 
-[lib]
-crate-type = ["staticlib"]
+# the profile used for `cargo build --release`
+[profile.release]
+panic = "abort" # disable stack unwinding on panic

Makefile

@@ -2,7 +2,7 @@
 title = "Catching Exceptions"
 order = 1
 path = "catching-exceptions"
-date = "2016-05-28"
+date  = 2016-05-28
 [extra]
 updated = "2016-06-25"
 +++

Xargo.toml

@@ -2,7 +2,7 @@
 title = "Better Exception Messages"
 order = 2
 path = "better-exception-messages"
-date = "2016-08-03"
+date  = 2016-08-03
 [extra]
 updated = "2016-11-01"
 +++

blog/content/extra/naked-exceptions/01-catching-exceptions/index.md

@@ -2,7 +2,7 @@
 title = "Returning from Exceptions"
 order = 3
 path = "returning-from-exceptions"
-date = "2016-09-21"
+date  = 2016-09-21
 [extra]
 updated = "2016-11-01"
 +++

blog/content/extra/naked-exceptions/02-better-exception-messages/index.md

@@ -0,0 +1,4 @@
++++
+title = "First Edition"
+template = "index.html"
++++

blog/content/extra/naked-exceptions/03-returning-from-exceptions/index.md

@@ -2,7 +2,7 @@
 title = "A minimal x86 kernel"
 order = 1
 path = "multiboot-kernel/"
-date = "2015-08-18"
+date  = 2015-08-18
 +++
 
 This post explains how to create a minimal x86 operating system kernel. In fact, it will just boot and print `OK` to the screen. In subsequent blog posts we will extend it using the [Rust] programming language.

blog/content/first-edition/_index.md

@@ -2,7 +2,7 @@
 title = "Entering Long Mode"
 order = 2
 path = "entering-longmode"
-date = "2015-08-25"
+date  = 2015-08-25
 [extra]
 updated = "2015-10-29"
 +++

blog/content/posts/01-multiboot-kernel/index.md

@@ -2,7 +2,7 @@
 title = "Set Up Rust"
 order = 3
 path = "set-up-rust"
-date = "2015-09-02"
+date  = 2015-09-02
 [extra]
 updated = "2017-04-12"
 +++

blog/content/posts/02-entering-longmode/index.md

@@ -2,7 +2,7 @@
 title = "Printing to Screen"
 order = 4
 path = "printing-to-screen"
-date = "2015-10-23"
+date  = 2015-10-23
 [extra]
 updated = "2016-10-31"
 +++

blog/content/posts/03-set-up-rust/index.md

@@ -2,7 +2,7 @@
 title = "Allocating Frames"
 order = 5
 path = "allocating-frames"
-date = "2015-11-15"
+date  = 2015-11-15
 +++
 
 In this post we create an allocator that provides free physical frames for a future paging module. To get the required information about available and used memory we use the Multiboot information structure. Additionally, we improve the `panic` handler to print the corresponding message and source line.

blog/content/posts/04-printing-to-screen/index.md

@@ -2,7 +2,7 @@
 title = "Page Tables"
 order = 6
 path = "page-tables"
-date = "2015-12-09"
+date  = 2015-12-09
 +++
 
 In this post we will create a paging module, which allows us to access and modify the 4-level page table. We will explore recursive page table mapping and use some Rust features to make it safe. Finally we will create functions to translate virtual addresses and to map and unmap pages.

blog/content/posts/05-allocating-frames/index.md

@@ -2,7 +2,7 @@
 title = "Remap the Kernel"
 order = 7
 path = "remap-the-kernel"
-date = "2016-01-01"
+date  = 2016-01-01
 [extra]
 updated = "2016-03-06"
 +++

blog/content/posts/06-page-tables/index.md

@@ -2,7 +2,7 @@
 title = "Kernel Heap"
 order = 8
 path = "kernel-heap"
-date = "2016-04-11"
+date  = 2016-04-11
 updated = "2017-11-19"
 +++
 

blog/content/posts/07-remap-the-kernel/index.md

@@ -2,7 +2,7 @@
 title = "Handling Exceptions"
 order = 9
 path = "handling-exceptions"
-date = "2017-03-26"
+date  = 2017-03-26
 +++
 
 In this post, we start exploring CPU exceptions. Exceptions occur in various erroneous situations, for example when accessing an invalid memory address or when dividing by zero. To catch them, we have to set up an _interrupt descriptor table_ that provides handler functions. At the end of this post, our kernel will be able to catch [breakpoint exceptions] and to resume normal execution afterwards.

blog/content/posts/08-kernel-heap/index.md

@@ -2,7 +2,7 @@
 title = "Double Faults"
 order = 10
 path = "double-faults"
-date = "2017-01-02"
+date  = 2017-01-02
 +++
 
 In this post we explore double faults in detail. We also set up an _Interrupt Stack Table_ to catch double faults on a separate kernel stack. This way, we can completely prevent triple faults, even on kernel stack overflow.

blog/content/posts/09-handling-exceptions/index.md

@@ -0,0 +1,4 @@
++++
+title = "Second Edition"
+template = "second-edition/index.html"
++++