Skip to content

[cmake] Hardcode some check_include_file checks #104706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jan 16, 2025

Conversation

Endilll
Copy link
Contributor

@Endilll Endilll commented Aug 18, 2024

This patch removes 11 check_include_file invocations from configuration phase of LLVM subproject on most of the platforms, hardcoding the results. Fallback is left for platforms that we don't document as supported or that are not detectable via CMAKE_SYSTEM_NAME, e.g. z/OS.

This patch reduces configuration time on Linux by 10%, going from 44.7 seconds down to 40.6 seconds on my Debian machine (ramdisk, cmake -DLLVM_ENABLE_PROJECTS="clang;lldb;clang-tools-extra" -DLLVM_ENABLE_RUNTIMES="libunwind;libcxx;libcxxabi" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_TARGETS_TO_BUILD="X86" -DLLVM_ENABLE_DOXYGEN=ON -DLLVM_ENABLE_LIBCXX=ON -DBUILD_SHARED_LIBS=ON -DLLDB_ENABLE_PYTHON=ON ~/endill/llvm-project/llvm).

In order to determine the values to hardcode, I prepared the following header:

#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <fenv.h>
#include <mach/mach.h>
#include <malloc/malloc.h>
#include <pthread.h>
#include <signal.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/param.h>
#include <sys/resource.h>
#include <sys/stat.h>
#include <sys/time.h>
#include <sys/types.h>
#include <sysexits.h>
#include <termios.h>
#include <unistd.h>

int main() {}

and tried to compile it on the oldest versions of platforms that are still supported (which was problematic to determine sometimes): macOS 12, Cygwin, DragonFly BSD 6.4.0, FreeBSD 13.3, Haiku R1 beta 4, RHEL 8.10 as a glibc-based Linux, Alpine 3.17 as musl-based Linux, NetBSD 9, OpenBSD 7.4, Solaris 11.4, Windows SDK 10.0.17763.0, which corresponds to Windows 10 1809 and is the oldest Windows 10 SDK in Visual Studio Installer.

For platforms I don't have access to, which are AIX 7.2 TL5 and z/OS 2.4.0, I had to rely on the official documentation. I suspect that AIX offers a better set of headers than what this PR claims, so I'm open to input from people who have access to a live system to test it.

Similarly to AIX, I have values for z/OS compiled from the official documentation that are not included in this patch, because apparently upstream CMake doesn't even support z/OS, so I don't even know how to make a place to hold those values. I see if (ZOS) in several places across our CMake files, but it's a mystery to me where this variable comes from. Input from people who have access to live z/OS instance is welcome.

@Endilll Endilll added the cmake Build system in general and CMake in particular label Aug 18, 2024
@hubert-reinterpretcast
Copy link
Collaborator

@perry-ca, can you comment regarding z/OS?

@perry-ca
Copy link
Contributor

The list of header not found for z/OS is:

  • <link.h>
  • <mach/mach.h>
  • <malloc/malloc.h>
  • <sys/param.h>
  • <sysexits.h>

@Endilll
Copy link
Contributor Author

Endilll commented Aug 19, 2024

The list of header not found for z/OS is:

* <link.h>

* <mach/mach.h>

* <malloc/malloc.h>

* <sys/param.h>

* <sysexits.h>

This is exactly the list I came up with after consulting z/OS documentation. Thank you for confirming, and kudos to z/OS documentation writers.

@perry-ca Can you comment on how CMake works on z/OS? In particular, how to guard z/OS-specific code in CMake code. if (ZOS)?

@perry-ca
Copy link
Contributor

Correct it is if (ZOS). Thanks for checking with us.

@perry-ca
Copy link
Contributor

For CMAKE_SYSTEM_NAME it is if (CMAKE_SYSTEM_NAME MATCHES "OS390")

# Confirmed in
# https://github.com/llvm/llvm-project/pull/104706#issuecomment-2297153534
if (CMAKE_SYSTEM_NAME MATCHES "OS390")
set(ZOS 1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The port of cmake to z/OS, in Modules/Platform/OS390.cmake, already has this line in it. Will this be a problem?

set(OS390 1)
set(ZOS 1)

set(HAVE_SYSEXITS_H 0)
set(HAVE_TERMIOS_H 1)
set(HAVE_UNISTD_H 1)
else()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, once you set the values, the check queries will early-out because they detect the cached value. Having it always call the check functions (even if the value is set) may make it easy to add platforms where some of these are known to exist, but others may not be.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was referring to the set(ZOS 1) line. Since the cmake platform files are already defining ZOS should we be defining it in the llvm cmake files?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was referring to the set(ZOS 1) line. Since the cmake platform files are already defining ZOS should we be defining it in the llvm cmake files?

If ZOS is always provided on z/OS CMake for one reason or another, we better skip setting it indeed. But it's news for me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, once you set the values, the check queries will early-out because they detect the cached value.

I've never felt I understood cache variables enough to feel comfortable relying on them. But I agree, this should be the case.

Having it always call the check functions (even if the value is set) may make it easy to add platforms where some of these are known to exist, but others may not be.

Are you sure this would serve a practical purpose? If one cares enough to set some of them beforehand to improve configuration times, I don't see how they would not check the rest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, once you set the values, the check queries will early-out because they detect the cached value.

I agree that this is the behavior (https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/CheckIncludeFile.cmake?ref_type=heads#L46), but I don't see it documented at https://cmake.org/cmake/help/latest/module/CheckIncludeFile.html. So I'm reluctant to rely on this.

@petrhosek
Copy link
Member

Do you know why are some of these needed in the first place? For example, do we actually supports platforms that don't have errno.h?

@Endilll
Copy link
Contributor Author

Endilll commented Aug 19, 2024

Do you know why are some of these needed in the first place? For example, do we actually supports platforms that don't have errno.h?

Unfortunately, just getting all those older platforms listed in the description running, and testing those headers on them took a lot of time. Yes, I see a potential clean-up after this PR for #ifdef HAVE_ERRNO_H, but I don't know for sure if that's still needed.

@MaskRay
Copy link
Member

MaskRay commented Aug 20, 2024

Thanks for improving the cmake configure time! I think some cleanups can be made first. For example, link.h is not needed and we can employ #if __has_include(<link.h>) (#104893) .

(I'm starting a 3-week vacation this Friday and will have limited availability.)

@Endilll
Copy link
Contributor Author

Endilll commented Jan 13, 2025

@petrhosek @Ericson2314 Do you think this is good enough to be merged?

Copy link
Collaborator

@AaronBallman AaronBallman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a minor cleanup. Any final concerns @petrhosek or @Ericson2314?

MaskRay added a commit that referenced this pull request Jan 15, 2025
These are unneeded even on AIX, PURE_WINDOWS, and ZOS (per #104706)

* HAVE_ERRNO_H: introduced by 1a93330 (2009) but unneeded.
  The guarded ABI is unconditionally used by lldb.
* HAVE_FCNTL_H
* HAVE_FENV_H
* HAVE_SYS_STAT_H

Pull Request: #123087
@Endilll Endilll merged commit ec9aa4a into llvm:main Jan 16, 2025
8 checks passed
@Endilll Endilll deleted the cmake-header-checks branch January 16, 2025 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmake Build system in general and CMake in particular
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants