fstat TSan interceptor not behaving correctly on POWER

[dustanddreams]

Consider the following simple program:

#include <errno.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>

int
main()
{
        int fd;
        struct stat st;
 
        fd = open("/etc/passwd", O_RDONLY);
        if (fd < 0) {
                printf("open failed: %s\n", strerror(errno));
                return 1;
        }       
        if (fstat(fd, &st) < 0) {
                printf("fstat failed: %s\n", strerror(errno));
                return 1;
        }       
        close(fd);
}

When compiled on a POWER system (Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-83-generic ppc64le)) with -fsanitize=thread, regardless of the optimization level used, always fails with fstat failed: Invalid argument. (i.e. the fstat interceptor has returned EINVAL)

This happens using either the Ubuntu-provided compiler (gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)) or a fresh build of llvm trunk. (with -DLLVM_ENABLE_PROJECTS='clang;compiler-rt;lld' to be sure to build a fresh TSan library as well)

Running the binary under strace shows no actual fstat system call occurring.

This behaviour seems to be specific to TSan on POWER systems; the same test program run on amd64 or arm64 completes without errors.

[llvmbot]

@llvm/issue-subscribers-backend-powerpc

Author: Miod Vallat (dustanddreams)

Consider the following simple program: ``` #include <errno.h> #include <fcntl.h> #include <unistd.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/stat.h>

int
main()
{
int fd;
struct stat st;

    fd = open("/etc/passwd", O_RDONLY);
    if (fd < 0) {
            printf("open failed: %s\n", strerror(errno));
            return 1;
    }       
    if (fstat(fd, &st) < 0) {
            printf("fstat failed: %s\n", strerror(errno));
            return 1;
    }       
    close(fd);

}

When compiled on a POWER system (`Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-83-generic ppc64le)`) with `-fsanitize=thread`, regardless of the optimization level used, always fails with `fstat failed: Invalid argument`. (i.e. the `fstat` interceptor has returned `EINVAL`)

This happens using either the Ubuntu-provided compiler (`gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)`) or a fresh build of llvm trunk. (with `-DLLVM_ENABLE_PROJECTS='clang;compiler-rt;lld'` to be sure to build a fresh TSan library as well)

Running the binary under `strace` shows no actual `fstat` system call occurring.

This behaviour seems to be specific to TSan on POWER systems; the same test program run on amd64 or arm64 completes without errors.
</details>

[dustanddreams]

Further analysis shows that this is caused by the interceptors, when configured for a glibc target, ending up invoking __fxstat with a hardcoded zero as the stat struct version number in compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:

TSAN_INTERCEPTOR(int, fstat, int fd, void *buf) {
#if SANITIZER_GLIBC
  SCOPED_TSAN_INTERCEPTOR(__fxstat, 0, fd, buf);
  if (fd > 0)
    FdAccess(thr, pc, fd);
  return REAL(__fxstat)(0, fd, buf);
#else
  SCOPED_TSAN_INTERCEPTOR(fstat, fd, buf);
  if (fd > 0)
    FdAccess(thr, pc, fd);
  return REAL(fstat)(fd, buf);
#endif
}

But if you look into the glibc sources, that zero should really be _STAT_VER_KERNEL or _STAT_VER_LINUX as defined in glibc in sysdeps/unix/sysv/linux/*/xstatver.h and checked against in sysdeps/unix/sysv/linux/fxstat64.c. On amd64, _STAT_VER_KERNEL is 0 and _STAT_VER_LINUX is 1, but on 64-bit power, _STAT_VER_KERNEL is 1 and _STAT_VER_LINUX is 3, which causes the __fxstat call in __interceptor_fstat to always fail and return EINVAL.

[dustanddreams]

Removing the glibc-specific code from this interceptor and shockingly making the fstat and fstat64 interceptors invoke fstat and fstat64 rather than __fxstat64 fixes the issue on power.

[dustanddreams]

Fixed by #86625.