[libc] add basic arena allocator #121173
Open
[libc] add basic arena allocator #121173
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@llvm/pr-subscribers-github-workflow @llvm/pr-subscribers-libc Author: Tristan Ross (RossComputerGuy) ChangesThis PR adds support for multiple allocator implementations. This will be beneficial for distros and embedded systems where different kinds of allocators can be better than others. To start out with, I just added a primitive arena allocator. This PR also adds Patch is 31.20 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/121173.diff 36 Files Affected:
diff --git a/libc/CMakeLists.txt b/libc/CMakeLists.txt
index 00a07ea3c8ac75..eebab925ba489d 100644
--- a/libc/CMakeLists.txt
+++ b/libc/CMakeLists.txt
@@ -386,6 +386,12 @@ else()
set(libc_opt_high_flag "-O3")
endif()
+if(${LIBC_CONF_ALLOC_TYPE} MATCHES "LIBC_ALLOC_TYPE_SCUDO")
+ set(LLVM_LIBC_INCLUDE_SCUDO ON)
+elseif(LLVM_LIBC_INCLUDE_SCUDO)
+ message(FATAL_ERROR "Cannot include scudo and use a different allocator.")
+endif()
+
add_subdirectory(include)
add_subdirectory(config)
add_subdirectory(hdr)
diff --git a/libc/config/config.json b/libc/config/config.json
index 9a5d5c3c68da60..6017d2e004de3e 100644
--- a/libc/config/config.json
+++ b/libc/config/config.json
@@ -115,6 +115,16 @@
"LIBC_ADD_NULL_CHECKS": {
"value": true,
"doc": "Add nullptr checks in the library's implementations to some functions for which passing nullptr is undefined behavior."
+ },
+ "LIBC_CONF_ALLOC_TYPE": {
+ "value": "LIBC_ALLOC_TYPE_ARENA",
+ "doc": "The implementation used for allocations, acceptable values are LIBC_ALLOC_TYPE_EXTERN, LIBC_ALLOC_TYPE_SCUDO, LIBC_ALLOC_TYPE_ARENA."
+ }
+ },
+ "unistd": {
+ "LIBC_CONF_PAGE_SIZE": {
+ "value": "LIBC_PAGE_SIZE_SYSTEM",
+ "doc": "The value to use for the system page size, acceptable values are LIBC_CONF_PAGE_SIZE_SYSTEM, LIBC_CONF_PAGE_SIZE_4K, LIBC_CONF_PAGE_SIZE_16K, LIBC_CONF_PAGE_SIZE_64K."
}
}
}
diff --git a/libc/config/linux/aarch64/entrypoints.txt b/libc/config/linux/aarch64/entrypoints.txt
index 00f0c6a8bfb8e4..ce67118545c9a5 100644
--- a/libc/config/linux/aarch64/entrypoints.txt
+++ b/libc/config/linux/aarch64/entrypoints.txt
@@ -323,6 +323,7 @@ set(TARGET_LIBC_ENTRYPOINTS
libc.src.unistd.ftruncate
libc.src.unistd.getcwd
libc.src.unistd.geteuid
+ libc.src.unistd.getpagesize
libc.src.unistd.getpid
libc.src.unistd.getppid
libc.src.unistd.gettid
diff --git a/libc/config/linux/riscv/entrypoints.txt b/libc/config/linux/riscv/entrypoints.txt
index 49a8d61b938027..3b53396b5943b2 100644
--- a/libc/config/linux/riscv/entrypoints.txt
+++ b/libc/config/linux/riscv/entrypoints.txt
@@ -320,6 +320,7 @@ set(TARGET_LIBC_ENTRYPOINTS
libc.src.unistd.ftruncate
libc.src.unistd.getcwd
libc.src.unistd.geteuid
+ libc.src.unistd.getpagesize
libc.src.unistd.getpid
libc.src.unistd.getppid
libc.src.unistd.gettid
diff --git a/libc/docs/configure.rst b/libc/docs/configure.rst
index 3db750b1aed214..036e8fc7a93832 100644
--- a/libc/docs/configure.rst
+++ b/libc/docs/configure.rst
@@ -32,6 +32,7 @@ to learn about the defaults for your platform and target.
- ``LIBC_CONF_ERRNO_MODE``: The implementation used for errno, acceptable values are LIBC_ERRNO_MODE_DEFAULT, LIBC_ERRNO_MODE_UNDEFINED, LIBC_ERRNO_MODE_THREAD_LOCAL, LIBC_ERRNO_MODE_SHARED, LIBC_ERRNO_MODE_EXTERNAL, and LIBC_ERRNO_MODE_SYSTEM.
* **"general" options**
- ``LIBC_ADD_NULL_CHECKS``: Add nullptr checks in the library's implementations to some functions for which passing nullptr is undefined behavior.
+ - ``LIBC_CONF_ALLOC_TYPE``: The implementation used for allocations, acceptable values are LIBC_ALLOC_TYPE_EXTERN, LIBC_ALLOC_TYPE_SCUDO, LIBC_ALLOC_TYPE_ARENA.
* **"math" options**
- ``LIBC_CONF_FREXP_INF_NAN_EXPONENT``: The value written back to the second parameter when calling frexp/frexpf/frexpl` with `+/-Inf`/`NaN` is unspecified. Configue an explicit exp value for Inf/NaN inputs.
- ``LIBC_CONF_MATH_OPTIMIZATIONS``: Configures optimizations for math functions. Values accepted are LIBC_MATH_SKIP_ACCURATE_PASS, LIBC_MATH_SMALL_TABLES, LIBC_MATH_NO_ERRNO, LIBC_MATH_NO_EXCEPT, and LIBC_MATH_FAST.
@@ -60,3 +61,5 @@ to learn about the defaults for your platform and target.
- ``LIBC_CONF_STRING_UNSAFE_WIDE_READ``: Read more than a byte at a time to perform byte-string operations like strlen.
* **"time" options**
- ``LIBC_CONF_TIME_64BIT``: Force the size of time_t to 64 bits, even on platforms where compatibility considerations would otherwise make it 32-bit.
+* **"unistd" options**
+ - ``LIBC_CONF_PAGE_SIZE``: The value to use for the system page size, acceptable values are LIBC_CONF_PAGE_SIZE_SYSTEM, LIBC_CONF_PAGE_SIZE_4K, LIBC_CONF_PAGE_SIZE_16K, LIBC_CONF_PAGE_SIZE_64K.
diff --git a/libc/hdrgen/yaml/unistd.yaml b/libc/hdrgen/yaml/unistd.yaml
index c6441c04ce3a3d..a044a4290aed85 100644
--- a/libc/hdrgen/yaml/unistd.yaml
+++ b/libc/hdrgen/yaml/unistd.yaml
@@ -141,6 +141,12 @@ functions:
- type: int
- type: __getoptargv_t
- type: const char *
+ - name: getpagesize
+ standards:
+ - POSIX
+ return_type: int
+ arguments:
+ - type: void
- name: getpid
standards:
- POSIX
diff --git a/libc/src/__support/CMakeLists.txt b/libc/src/__support/CMakeLists.txt
index 4e90aad9a45b40..45f4dde264122a 100644
--- a/libc/src/__support/CMakeLists.txt
+++ b/libc/src/__support/CMakeLists.txt
@@ -370,3 +370,5 @@ add_subdirectory(HashTable)
add_subdirectory(fixed_point)
add_subdirectory(time)
+
+add_subdirectory(alloc)
diff --git a/libc/src/__support/alloc/CMakeLists.txt b/libc/src/__support/alloc/CMakeLists.txt
new file mode 100644
index 00000000000000..f7142bb0d7c0b8
--- /dev/null
+++ b/libc/src/__support/alloc/CMakeLists.txt
@@ -0,0 +1,57 @@
+add_object_library(
+ base
+ SRCS
+ base.cpp
+ HDRS
+ base.h
+ DEPENDS
+ libc.hdr.types.size_t
+ libc.src.__support.macros.config
+)
+
+if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${LIBC_TARGET_OS})
+ add_subdirectory(${LIBC_TARGET_OS})
+endif()
+
+if(TARGET libc.src.__support.alloc.${LIBC_TARGET_OS}.page)
+ add_object_library(
+ page
+ ALIAS
+ DEPENDS
+ .${LIBC_TARGET_OS}.page
+ )
+endif()
+
+add_object_library(
+ arena
+ SRCS
+ arena.cpp
+ HDRS
+ arena.h
+ COMPILE_OPTIONS
+ -DLIBC_PAGE_SIZE=${LIBC_CONF_PAGE_SIZE}
+ DEPENDS
+ .base
+ .page
+ libc.include.llvm-libc-macros.stdint_macros
+ libc.src.string.memmove
+ libc.src.unistd.getpagesize
+)
+
+if(NOT ${LIBC_CONF_ALLOC_TYPE} MATCHES "LIBC_ALLOC_TYPE_SCUDO" AND NOT ${LIBC_CONF_ALLOC_TYPE} MATCHES "LIBC_ALLOC_TYPE_EXTERN")
+ string(TOLOWER ${LIBC_CONF_ALLOC_TYPE} LIBC_CONF_ALLOC_TYPE_NAME)
+ string(REPLACE "libc_alloc_type_" "" LIBC_CONF_ALLOC_TYPE_NAME "${LIBC_CONF_ALLOC_TYPE_NAME}")
+ if(TARGET libc.src.__support.alloc.${LIBC_CONF_ALLOC_TYPE_NAME})
+ add_object_library(
+ alloc
+ SRCS
+ alloc.cpp
+ HDRS
+ alloc.h
+ COMPILE_OPTIONS
+ -DLIBC_CONF_ALLOC_TYPE=${LIBC_CONF_ALLOC_TYPE_NAME}
+ DEPENDS
+ .${LIBC_CONF_ALLOC_TYPE_NAME}
+ )
+ endif()
+endif()
diff --git a/libc/src/__support/alloc/alloc.cpp b/libc/src/__support/alloc/alloc.cpp
new file mode 100644
index 00000000000000..f5f1832bb58a8f
--- /dev/null
+++ b/libc/src/__support/alloc/alloc.cpp
@@ -0,0 +1,13 @@
+#include <src/__support/alloc/alloc.h>
+#include <src/__support/alloc/arena.h>
+
+namespace LIBC_NAMESPACE_DECL {
+
+#define CONCAT(a, b) a##b
+#define EXPAND_AND_CONCAT(a, b) CONCAT(a, b)
+
+#define ALLOCATOR EXPAND_AND_CONCAT(LIBC_CONF_ALLOC_TYPE, _allocator)
+
+BaseAllocator *allocator = reinterpret_cast<BaseAllocator *>(&ALLOCATOR);
+
+} // namespace LIBC_NAMESPACE_DECL
diff --git a/libc/src/__support/alloc/alloc.h b/libc/src/__support/alloc/alloc.h
new file mode 100644
index 00000000000000..88b5fbfb84680f
--- /dev/null
+++ b/libc/src/__support/alloc/alloc.h
@@ -0,0 +1,22 @@
+//===-- libc-wide allocator -------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIBC_SRC___SUPPORT_ALLOC_ALLOC_H
+#define LLVM_LIBC_SRC___SUPPORT_ALLOC_ALLOC_H
+
+#include "src/__support/alloc/base.h"
+#include "src/__support/macros/config.h"
+
+namespace LIBC_NAMESPACE_DECL {
+
+// The primary allocator to use
+extern BaseAllocator *allocator;
+
+} // namespace LIBC_NAMESPACE_DECL
+
+#endif
diff --git a/libc/src/__support/alloc/arena.cpp b/libc/src/__support/alloc/arena.cpp
new file mode 100644
index 00000000000000..f39d80630f2477
--- /dev/null
+++ b/libc/src/__support/alloc/arena.cpp
@@ -0,0 +1,71 @@
+#include "src/__support/alloc/arena.h"
+#include "src/__support/alloc/page.h"
+#include "src/__support/common.h"
+#include "src/__support/macros/config.h"
+#include "src/__support/macros/page_size.h"
+#include "src/__support/memory_size.h"
+#include "src/string/memmove.h"
+#include "src/unistd/getpagesize.h"
+
+namespace LIBC_NAMESPACE_DECL {
+
+void *arena_allocate(BaseAllocator *base, size_t alignment, size_t size) {
+ ArenaAllocator *self = reinterpret_cast<ArenaAllocator *>(base);
+
+ if (self->buffer == nullptr) {
+ self->buffer = reinterpret_cast<uint8_t *>(page_allocate(1));
+ self->buffer_size = self->get_page_size();
+ }
+
+ uintptr_t curr_ptr = (uintptr_t)self->buffer + (uintptr_t)self->curr_offset;
+ uintptr_t offset = internal::align_forward<uintptr_t>(curr_ptr, alignment);
+ offset -= (uintptr_t)self->buffer;
+
+ if (offset + size > self->buffer_size) {
+ self->buffer = reinterpret_cast<uint8_t *>(
+ page_expand(self->buffer, self->buffer_size / self->get_page_size()));
+ self->buffer_size += self->get_page_size();
+ }
+
+ if (offset + size <= self->buffer_size) {
+ void *ptr = &self->buffer[offset];
+ self->prev_offset = offset;
+ self->curr_offset = offset + size;
+ return ptr;
+ }
+ return nullptr;
+}
+
+void *arena_expand(BaseAllocator *base, void *ptr, size_t alignment,
+ size_t size) {
+ ArenaAllocator *self = reinterpret_cast<ArenaAllocator *>(base);
+
+ if (self->buffer + self->prev_offset == ptr) {
+ self->curr_offset = self->prev_offset + size;
+ return ptr;
+ } else {
+ void *new_mem = arena_allocate(base, alignment, size);
+ memmove(new_mem, ptr, size);
+ return new_mem;
+ }
+ return nullptr;
+}
+
+bool arena_free(BaseAllocator *base, void *ptr) {
+ (void)base;
+ (void)ptr;
+ return true;
+}
+
+size_t ArenaAllocator::get_page_size() {
+ if (page_size == LIBC_PAGE_SIZE_SYSTEM) {
+ page_size = getpagesize();
+ }
+ return page_size;
+}
+
+static ArenaAllocator default_arena_allocator(LIBC_PAGE_SIZE,
+ 2 * sizeof(void *));
+BaseAllocator *block_allocator = &default_arena_allocator;
+
+} // namespace LIBC_NAMESPACE_DECL
diff --git a/libc/src/__support/alloc/arena.h b/libc/src/__support/alloc/arena.h
new file mode 100644
index 00000000000000..45b6915139e712
--- /dev/null
+++ b/libc/src/__support/alloc/arena.h
@@ -0,0 +1,47 @@
+//===-- An arena allocator using pages. -------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIBC_SRC___SUPPORT_ALLOC_ARENA_H
+#define LLVM_LIBC_SRC___SUPPORT_ALLOC_ARENA_H
+
+#include "hdr/types/size_t.h"
+#include "include/llvm-libc-macros/stdint-macros.h"
+#include "src/__support/alloc/base.h"
+
+namespace LIBC_NAMESPACE_DECL {
+
+void *arena_allocate(BaseAllocator *base, size_t alignment, size_t size);
+void *arena_expand(BaseAllocator *base, void *ptr, size_t alignment,
+ size_t size);
+bool arena_free(BaseAllocator *base, void *ptr);
+
+class ArenaAllocator : public BaseAllocator {
+public:
+ uint8_t *buffer;
+ size_t buffer_size;
+ size_t prev_offset;
+ size_t curr_offset;
+
+private:
+ size_t page_size;
+
+public:
+ constexpr ArenaAllocator(size_t page_size, size_t default_alignment)
+ : BaseAllocator(arena_allocate, arena_expand, arena_free,
+ default_alignment),
+ buffer(nullptr), buffer_size(0), prev_offset(0), curr_offset(0),
+ page_size(page_size) {}
+
+ size_t get_page_size();
+};
+
+extern BaseAllocator *arena_allocator;
+
+} // namespace LIBC_NAMESPACE_DECL
+
+#endif
diff --git a/libc/src/__support/alloc/baremetal/CMakeLists.txt b/libc/src/__support/alloc/baremetal/CMakeLists.txt
new file mode 100644
index 00000000000000..845d2f5287bfbf
--- /dev/null
+++ b/libc/src/__support/alloc/baremetal/CMakeLists.txt
@@ -0,0 +1,13 @@
+add_object_library(
+ page
+ SRCS
+ page.cpp
+ HDRS
+ ../page.h
+ DEPENDS
+ libc.src.__support.alloc.base
+ libc.src.sys.mman.mmap
+ libc.src.sys.mman.mremap
+ libc.src.sys.mman.munmap
+ libc.src.unistd.getpagesize
+)
diff --git a/libc/src/__support/alloc/baremetal/page.cpp b/libc/src/__support/alloc/baremetal/page.cpp
new file mode 100644
index 00000000000000..4d92141bcc8276
--- /dev/null
+++ b/libc/src/__support/alloc/baremetal/page.cpp
@@ -0,0 +1,22 @@
+#include "src/__support/alloc/page.h"
+#include "src/__suport/macros/config.h"
+
+namespace LIBC_NAMESPACE_DECL {
+
+extern "C" void *__llvm_libc_page_allocate(size_t n_pages);
+extern "C" void *__llvm_libc_page_expand(void *ptr, size_t n_pages);
+extern "C" bool __llvm_libc_page_free(void *ptr, size_t n_pages);
+
+void *page_allocate(size_t n_pages) {
+ return __llvm_libc_page_allocate(n_pages);
+}
+
+void *page_expand(void *ptr, size_t n_pages) {
+ return __llvm_libc_page_expand(ptr, n_pages);
+}
+
+bool page_free(void *ptr, size_t n_pages) {
+ return __llvm_libc_page_free(ptr, n_pages);
+}
+
+} // namespace LIBC_NAMESPACE_DECL
diff --git a/libc/src/__support/alloc/base.cpp b/libc/src/__support/alloc/base.cpp
new file mode 100644
index 00000000000000..93684d457ee42d
--- /dev/null
+++ b/libc/src/__support/alloc/base.cpp
@@ -0,0 +1,16 @@
+#include "src/__support/alloc/base.h"
+#include "src/__support/macros/config.h"
+
+namespace LIBC_NAMESPACE_DECL {
+
+void *BaseAllocator::alloc(size_t alignment, size_t size) {
+ return impl_alloc(this, alignment, size);
+}
+
+void *BaseAllocator::expand(void *ptr, size_t alignment, size_t size) {
+ return impl_expand(this, ptr, alignment, size);
+}
+
+bool BaseAllocator::free(void *ptr) { return impl_free(this, ptr); }
+
+} // namespace LIBC_NAMESPACE_DECL
diff --git a/libc/src/__support/alloc/base.h b/libc/src/__support/alloc/base.h
new file mode 100644
index 00000000000000..59370352953d3e
--- /dev/null
+++ b/libc/src/__support/alloc/base.h
@@ -0,0 +1,44 @@
+//===-- A generic base allocator. -------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIBC_SRC___SUPPORT_ALLOC_BASE_H
+#define LLVM_LIBC_SRC___SUPPORT_ALLOC_BASE_H
+
+#include "hdr/types/size_t.h"
+#include "src/__support/macros/config.h"
+
+namespace LIBC_NAMESPACE_DECL {
+
+class BaseAllocator {
+public:
+ using AllocFunc = void *(BaseAllocator *self, size_t, size_t);
+ using ExpandFunc = void *(BaseAllocator *self, void *, size_t, size_t);
+ using FreeFunc = bool(BaseAllocator *self, void *);
+
+private:
+ // Implementation specific functions
+ AllocFunc *impl_alloc;
+ ExpandFunc *impl_expand;
+ FreeFunc *impl_free;
+
+public:
+ constexpr BaseAllocator(AllocFunc *ia, ExpandFunc *ie, FreeFunc *ifr,
+ size_t default_alignment)
+ : impl_alloc(ia), impl_expand(ie), impl_free(ifr),
+ default_alignment(default_alignment) {}
+
+ size_t default_alignment;
+
+ void *alloc(size_t alignment, size_t size);
+ void *expand(void *ptr, size_t alignment, size_t size);
+ bool free(void *ptr);
+};
+
+} // namespace LIBC_NAMESPACE_DECL
+
+#endif // LLVM_LIBC_SRC___SUPPORT_ALLOC_BASE_H
diff --git a/libc/src/__support/alloc/linux/CMakeLists.txt b/libc/src/__support/alloc/linux/CMakeLists.txt
new file mode 100644
index 00000000000000..4484a5d60c9635
--- /dev/null
+++ b/libc/src/__support/alloc/linux/CMakeLists.txt
@@ -0,0 +1,16 @@
+add_object_library(
+ page
+ SRCS
+ page.cpp
+ HDRS
+ ../page.h
+ DEPENDS
+ libc.hdr.types.size_t
+ libc.include.llvm-libc-macros.stdlib_macros
+ libc.include.llvm-libc-macros.stdint_macros
+ libc.src.__support.alloc.base
+ libc.src.sys.mman.mmap
+ libc.src.sys.mman.mremap
+ libc.src.sys.mman.munmap
+ libc.src.unistd.getpagesize
+)
diff --git a/libc/src/__support/alloc/linux/page.cpp b/libc/src/__support/alloc/linux/page.cpp
new file mode 100644
index 00000000000000..1d9f04e951856e
--- /dev/null
+++ b/libc/src/__support/alloc/linux/page.cpp
@@ -0,0 +1,41 @@
+#include "src/__support/alloc/page.h"
+#include "include/llvm-libc-macros/stdint-macros.h"
+#include "include/llvm-libc-macros/stdlib-macros.h"
+#include "src/__support/macros/config.h"
+#include "src/__support/memory_size.h"
+#include "src/sys/mman/mmap.h"
+#include "src/sys/mman/mremap.h"
+#include "src/sys/mman/munmap.h"
+#include "src/unistd/getpagesize.h"
+
+namespace LIBC_NAMESPACE_DECL {
+
+static void *alloc_hint = NULL;
+
+void *page_allocate(size_t n_pages) {
+ size_t page_size = getpagesize();
+ size_t size = n_pages * page_size;
+ size_t aligned_size = internal::SafeMemSize(size).align_up(page_size);
+
+ void *ptr = mmap(&alloc_hint, aligned_size, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ if (ptr == NULL)
+ return nullptr;
+
+ alloc_hint = (void *)(((uintptr_t)ptr) + aligned_size);
+ return ptr;
+}
+
+void *page_expand(void *ptr, size_t n_pages) {
+ (void)ptr;
+ (void)n_pages;
+ return nullptr;
+}
+
+bool page_free(void *ptr, size_t n_pages) {
+ (void)ptr;
+ (void)n_pages;
+ return false;
+}
+
+} // namespace LIBC_NAMESPACE_DECL
diff --git a/libc/src/__support/alloc/page.h b/libc/src/__support/alloc/page.h
new file mode 100644
index 00000000000000..bb627b50cd6eb4
--- /dev/null
+++ b/libc/src/__support/alloc/page.h
@@ -0,0 +1,23 @@
+//===-- Page allocations ----------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIBC_SRC___SUPPORT_ALLOC_PAGE_H
+#define LLVM_LIBC_SRC___SUPPORT_ALLOC_PAGE_H
+
+#include "hdr/types/size_t.h"
+#include "src/__support/macros/config.h"
+
+namespace LIBC_NAMESPACE_DECL {
+
+void *page_allocate(size_t n_pages);
+void *page_expand(void *ptr, size_t n_pages);
+bool page_free(void *ptr, size_t n_pages);
+
+} // namespace LIBC_NAMESPACE_DECL
+
+#endif // LLVM_LIBC_SRC___SUPPORT_ALLOC_BASE_H
diff --git a/libc/src/__support/macros/CMakeLists.txt b/libc/src/__support/macros/CMakeLists.txt
index 99d4f640f283a4..5ef209f4f9c8f6 100644
--- a/libc/src/__support/macros/CMakeLists.txt
+++ b/libc/src/__support/macros/CMakeLists.txt
@@ -20,6 +20,12 @@ add_header_library(
libc.src.__support.macros.properties.compiler
)
+add_header_library(
+ page_size
+ HDRS
+ page_size.h
+)
+
add_header_library(
sanitizer
HDRS
diff --git a/libc/src/__support/macros/page_size.h b/libc/src/__support/macros/page_size.h
new file mode 100644
index 00000000000000..bc45dcc672d345
--- /dev/null
+++ b/libc/src/__support/macros/page_size.h
@@ -0,0 +1,17 @@
+//===-- Convenient page size macros -----------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIBC_SRC___SUPPORT_MACROS_PAGE_SIZE_H
+#define LLVM_LIBC_SRC___SUPPORT_MACROS_PAGE_SIZE_H
+
+#define LIBC_PAGE_SIZE_SYSTEM 0
+#define LIBC_PAGE_SIZE_4K (4 * 1024)
+#define LIBC_PAGE_SIZE_16K (16 * 1024)
+#define LIBC_PAGE_SIZE_64K (64 * 1024)
+
+#endif // LLVM_LIBC_SRC___SUPPORT_MACROS_PAGE_SIZE_H
diff --git a/libc/src/__support/memory_size.h b/libc/src/__support/memory_size.h
index cdd6a10222de10..a9df814be7156b 100644
--- a/libc/src/__support/memory_size.h
+++ b/libc/src/__suppo...
[truncated]
|
RossComputerGuy
force-pushed
the
feat/libc-alloc1
branch
from
bce6b17 to
934e001
Compare
RossComputerGuy
force-pushed
the
feat/libc-alloc1
branch
18 times, most recently
from
caabf59 to
e6d3de8
Compare
|
There's already an implementation of a |
Comment on lines
+24
to
+23
| #else | ||
| return LIBC_PAGE_SIZE; | ||
| #endif |
There was a problem hiding this comment.
I don't think we should allow for the user to hardcode their page size for hosted environments.
There was a problem hiding this comment.
Why? There's malloc implementations out there which support hardcoded page sizes.
There was a problem hiding this comment.
Why? There's malloc implementations out there which support hardcoded page sizes.
And there's programs that just hardcode the page size and don't use getpagesize or sysconf(_SC_PAGESIZE) and those programs wind up with portability issues when they join the 21st century where the page size isn't 4k.
There was a problem hiding this comment.
I'm aware, I've ran into those problems. I'm thinking we can drop support for changing the page size on Linux but keep the option for baremetal. Does that sound like the best option?
Comment on lines
20
to
23
| int r = (int)LIBC_NAMESPACE::getauxval(AT_PAGESZ); | ||
| if (r == 0) | ||
| return (int)LIBC_NAMESPACE::sysconf(_SC_PAGESIZE); | ||
| return r; |
There was a problem hiding this comment.
Smells a lot like libc/src/unistd/linux/sysconf.cpp...probably this could just be:
Suggested change
| int r = (int)LIBC_NAMESPACE::getauxval(AT_PAGESZ); | |
| if (r == 0) | |
| return (int)LIBC_NAMESPACE::sysconf(_SC_PAGESIZE); | |
| return r; | |
| return static_cast<int>(LIBC_NAMESPACE::sysconf(_SC_PAGESIZE)); |
Make sure to use C++ style casts in llvm-libc, please.
There was a problem hiding this comment.
This is done along with rebasing the PR.
libc/hdrgen/yaml/unistd.yaml
Outdated
| - POSIX | ||
| return_type: int | ||
| arguments: | ||
| - type: void |
There was a problem hiding this comment.
Consider breaking getpagesize out into its own PR that's orthogonal to a new memory allocator.
There was a problem hiding this comment.
Will do, I didn't want PR's to get tied up depending on other PR's.
There was a problem hiding this comment.
Sorry, did you move it to a distinct PR? or not yet?
RossComputerGuy
force-pushed
the
feat/libc-alloc1
branch
from
e6d3de8 to
63c5bf8
Compare
mysterymath
reviewed
Jan 8, 2025
There was a problem hiding this comment.
Can you speak a bit more as to how this particular allocation approach helps for baremetal systems? These tend to be more space-constrained than time-constrained. AFAICT, this allocator can never free, which prioritizes time over space. (It also is likely less code, but the existing allocator is relatively small for what it does, and RAM is usually more expensive than ROM.)
Minimizing time and ROM usage and maximizing RAM usage may be a reasonable trade-off to support, but just to double-check, is there a specific use-case that exists to drive those requirements?
| }, | ||
| "LIBC_CONF_ALLOC_TYPE": { | ||
| "value": "LIBC_ALLOC_TYPE_ARENA", | ||
| "doc": "The implementation used for allocations, acceptable values are LIBC_ALLOC_TYPE_EXTERN, LIBC_ALLOC_TYPE_SCUDO, LIBC_ALLOC_TYPE_ARENA." |
There was a problem hiding this comment.
How does this interact with the existing embedded allocator present in the libc? Immediately it's unusual that it isn't present in a list of allocators.
The allocator doesn't currently work for Linux, but that's something I'm planning to fix in the near future. It's just for lack of a __morecore abstraction hooked up to e.g. sbrk, plus the plumbing for that sycall.
There was a problem hiding this comment.
I'm not familiar with existing allocators being implemented. I could hook them up to support the option and general interface.
mysterymath
reviewed
Jan 8, 2025
|
|
||
| if (offset + size > self->buffer_size) { | ||
| self->buffer = reinterpret_cast<uint8_t *>( | ||
| page_expand(self->buffer, self->buffer_size / self->get_page_size())); |
There was a problem hiding this comment.
What happens if page_expand fails? It appears to summarily return nullptr on Linux.
There was a problem hiding this comment.
Oh yeah, woops. I'll fix this when I can.
I wasn't familiar with any existing allocators and I'm not sure how to have free in an arena allocator. To me, this would be useful for UEFI until we have a proper block allocator. |
Not sure what you mean by this; there is a dlmalloc-style block allocator already present in the libc. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for multiple allocator implementations. This will be beneficial for distros and embedded systems where different kinds of allocators can be better than others. To start out with, I just added a primitive arena allocator. This PR also adds
getpagesizewhich supports dynamic and static page sizes.