Skip to content

[libc] update host build docs #120147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Dec 19, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions libc/docs/dev/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Developer Guides
Navigate to the links below for information on the respective topics:

.. toctree::
:maxdepth: 1

code_style
source_tree_layout
Expand Down
90 changes: 46 additions & 44 deletions libc/docs/full_host_build.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Full Host Build
:depth: 1
:local:

.. note::
.. note::
Fullbuild requires running headergen, which is a python program that depends on
pyyaml. The minimum versions are listed on the :ref:`header_generation`
page, as well as additional information.
Expand Down Expand Up @@ -99,11 +99,16 @@ a C++ standard library (like libc++). Hence, we do not include
`libc++ <https://libcxx.llvm.org/>`_, libcxx-abi and libunwind in the
LLVM only toolchain and use them to build and link C++ applications.

Below is the list of commands for a simple recipe to build and install the
libc components along with other components of an LLVM only toolchain. In this
we've set the Ninja generator, enabled a full compiler suite, set the build
type to "Debug", and enabled the Scudo allocator. The build also tells clang
to use the freshly built lld and compiler-rt.
Below is the cmake command for a bootstrapping build of LLVM. This will build
clang and lld with the current system's toolchain, then build compiler-rt and
LLVM-libc with that freshly built clang. This ensures that LLVM-libc can take
advantage of the latest clang features and optimizations.

This build also uses Ninja as cmake's generator, and sets lld and compiler-rt as
the default for the fresh clang. Those settings are recommended, but the build
should still work without them. The compiler-rt options are required for
building `Scudo <https://llvm.org/docs/ScudoHardenedAllocator.html>`_ as the
allocator for LLVM-libc.

.. note::
if your build fails with an error saying the compiler can't find
Expand All @@ -113,44 +118,27 @@ to use the freshly built lld and compiler-rt.
this command:
``sudo ln -s /usr/include/<TARGET TRIPLE>/asm /usr/include/asm``

.. TODO: Move from projects to runtimes for libc, compiler-rt
.. code-block:: sh

$> cd llvm-project # The llvm-project checkout
$> mkdir build
$> cd build
$> SYSROOT=/path/to/sysroot # Remember to set this!
$> cmake ../llvm \
-G Ninja \
-DLLVM_ENABLE_PROJECTS="clang;lld;libc;compiler-rt" \
-DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DLLVM_LIBC_FULL_BUILD=ON \
-DLLVM_LIBC_INCLUDE_SCUDO=ON \
-DCOMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC=ON \
-DCOMPILER_RT_BUILD_GWP_ASAN=OFF \
-DCOMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED=OFF \
-DCLANG_DEFAULT_LINKER=lld \
-DCLANG_DEFAULT_RTLIB=compiler-rt \
-DDEFAULT_SYSROOT=$SYSROOT \
-DCMAKE_INSTALL_PREFIX=$SYSROOT

We will go over some of the special options passed to the ``cmake`` command
above.

* **Enabled Projects** - Since we want to build and install clang, lld
and compiler-rt along with the libc, we specify
``clang;libc;lld;compiler-rt`` as the list of enabled projects.
* **The full build option** - Since we want to do build the full libc, we pass
``-DLLVM_LIBC_FULL_BUILD=ON``.
* **Scudo related options** - LLVM's libc uses
`Scudo <https://llvm.org/docs/ScudoHardenedAllocator.html>`_ as its allocator.
So, when building the full libc, we should specify that we want to include
Scudo in the libc. Since the libc currently only supports static linking, we
also specify that we do not want to build the Scudo shared library.
* **Default sysroot and install prefix** - This is the path to the tool chain
install directory. This is the directory where you intend to set up the sysroot.
-G Ninja \
-DLLVM_ENABLE_PROJECTS="clang;lld" \
-DLLVM_ENABLE_RUNTIMES="libc;compiler-rt" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DLLVM_LIBC_FULL_BUILD=ON \
-DLLVM_LIBC_INCLUDE_SCUDO=ON \
-DCOMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC=ON \
-DCOMPILER_RT_BUILD_GWP_ASAN=OFF \
-DCOMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED=OFF \
-DCLANG_DEFAULT_LINKER=lld \
-DCLANG_DEFAULT_RTLIB=compiler-rt \
-DCMAKE_INSTALL_PREFIX=$SYSROOT

Build and install
=================
Expand All @@ -164,13 +152,12 @@ Build and install
you may need to delete them from ``/usr/local/include``.

After configuring the build with the above ``cmake`` command, one can build and
install the libc, clang (and its support libraries and builtins), lld and
compiler-rt, with the following command:
install the toolchain with

.. code-block:: sh

$> ninja install-clang install-builtins install-compiler-rt \
install-core-resource-headers install-libc install-lld
$> ninja all
$> ninja install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is kind of nice to have the explicit install steps listed. At least to me it gives more confidence in what is or is not being installed. At the least, can't the user just run ninja install without ninja all, and the dependencies are wired up to build first, then install?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it's nice to list the install steps, however whenever I hear from people who are trying it out they always ignore this step and just run ninja install. I'll leave it as it was and add ninja install as an alternative.


Once the above command completes successfully, the ``$SYSROOT`` directory you
have specified with the CMake configure step above will contain a full LLVM-only
Expand All @@ -190,9 +177,9 @@ These instructions should work on a Debian-based x86_64 system:

$> apt download linux-libc-dev
$> dpkg -x linux-libc-dev*deb .
$> mv usr/include/* /path/to/sysroot/include
$> rm -rf usr linux-libc-dev*deb
$> ln -s x86_64-linux-gnu/asm ~/Programming/sysroot/include/asm
$> cp -r usr/* /path/to/sysroot/
$> rm -r usr linux-libc-dev*deb
$> ln -s /path/to/sysroot/include/x86_64-linux-gnu/asm /path/to/sysroot/include/asm

Using your newly built libc
===========================
Expand All @@ -208,3 +195,18 @@ invocation:
Because the libc does not yet support dynamic linking, the -static parameter
must be added to all clang invocations.


You can make sure you're using the newly built toolchain by trying out features
that aren't yet supported by the system toolchain, such as fixed point. The
following is an example program that demonstrates the difference:

.. code-block:: C

#include <stdio.h>
int main() {
printf("Hello, World!\n%.9f\n%.9lK\n",
4294967295.000000001,
4294967295.000000001ulK);
return 0;
}

38 changes: 19 additions & 19 deletions libc/test/src/__support/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -252,29 +252,29 @@ add_libc_test(
# FIXME: We shouldn't have regular executables created because we could be
# cross-compiling the tests and running through an emulator.
if(NOT LIBC_TARGET_OS_IS_GPU)
add_executable(
libc_str_to_float_comparison_test
str_to_float_comparison_test.cpp
)
# add_executable(
# libc_str_to_float_comparison_test
# str_to_float_comparison_test.cpp
# )

target_link_libraries(libc_str_to_float_comparison_test
PRIVATE
"${LIBC_TARGET}"
)
# target_link_libraries(libc_str_to_float_comparison_test
# PRIVATE
# "${LIBC_TARGET}"
# )

add_executable(
libc_system_str_to_float_comparison_test
str_to_float_comparison_test.cpp
)
# add_executable(
# libc_system_str_to_float_comparison_test
# str_to_float_comparison_test.cpp
# )

set(float_test_file ${CMAKE_CURRENT_SOURCE_DIR}/str_to_float_comparison_data.txt)
# set(float_test_file ${CMAKE_CURRENT_SOURCE_DIR}/str_to_float_comparison_data.txt)

add_custom_command(TARGET libc_str_to_float_comparison_test
POST_BUILD
COMMAND ${CMAKE_CROSSCOMPILING_EMULATOR} $<TARGET_FILE:libc_str_to_float_comparison_test> ${float_test_file}
DEPENDS ${float_test_file}
COMMENT "Test the strtof and strtod implementations against precomputed results."
VERBATIM)
# add_custom_command(TARGET libc_str_to_float_comparison_test
# POST_BUILD
# COMMAND ${CMAKE_CROSSCOMPILING_EMULATOR} $<TARGET_FILE:libc_str_to_float_comparison_test> ${float_test_file}
# DEPENDS ${float_test_file}
# COMMENT "Test the strtof and strtod implementations against precomputed results."
# VERBATIM)
endif()

add_subdirectory(CPP)
Expand Down
Loading