Skip to content

[libc++] Add remaining benchmarks from [alg.modifying.operations] #127354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Mar 17, 2025

Conversation

ldionne
Copy link
Member

@ldionne ldionne commented Feb 15, 2025

This patch adds benchmarks for all the remaining algorithms in [alg.modifying.operations] that we didn't already have a benchmark for.

@ldionne ldionne requested a review from a team as a code owner February 15, 2025 23:42
@llvmbot llvmbot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Feb 15, 2025
@llvmbot
Copy link
Member

llvmbot commented Feb 15, 2025

@llvm/pr-subscribers-libcxx

Author: Louis Dionne (ldionne)

Changes

This patch adds benchmarks for all the remaining algorithms in [alg.modifying.operations] that we didn't already have a benchmark for.


Patch is 93.48 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/127354.diff

28 Files Affected:

  • (modified) libcxx/include/module.modulemap (+2-1)
  • (removed) libcxx/test/benchmarks/algorithms/fill.bench.cpp (-51)
  • (added) libcxx/test/benchmarks/algorithms/modifying/fill.bench.cpp (+86)
  • (added) libcxx/test/benchmarks/algorithms/modifying/fill_n.bench.cpp (+86)
  • (added) libcxx/test/benchmarks/algorithms/modifying/generate.bench.cpp (+59)
  • (added) libcxx/test/benchmarks/algorithms/modifying/generate_n.bench.cpp (+59)
  • (added) libcxx/test/benchmarks/algorithms/modifying/remove.bench.cpp (+133)
  • (added) libcxx/test/benchmarks/algorithms/modifying/remove_copy.bench.cpp (+109)
  • (added) libcxx/test/benchmarks/algorithms/modifying/remove_copy_if.bench.cpp (+119)
  • (added) libcxx/test/benchmarks/algorithms/modifying/remove_if.bench.cpp (+143)
  • (added) libcxx/test/benchmarks/algorithms/modifying/replace.bench.cpp (+107)
  • (added) libcxx/test/benchmarks/algorithms/modifying/replace_if.bench.cpp (+117)
  • (added) libcxx/test/benchmarks/algorithms/modifying/reverse.bench.cpp (+57)
  • (added) libcxx/test/benchmarks/algorithms/modifying/reverse_copy.bench.cpp (+61)
  • (added) libcxx/test/benchmarks/algorithms/modifying/rotate.bench.cpp (+59)
  • (added) libcxx/test/benchmarks/algorithms/modifying/rotate_copy.bench.cpp (+65)
  • (added) libcxx/test/benchmarks/algorithms/modifying/sample.bench.cpp (+68)
  • (added) libcxx/test/benchmarks/algorithms/modifying/shift_left.bench.cpp (+56)
  • (added) libcxx/test/benchmarks/algorithms/modifying/shift_right.bench.cpp (+56)
  • (added) libcxx/test/benchmarks/algorithms/modifying/shuffle.bench.cpp (+56)
  • (added) libcxx/test/benchmarks/algorithms/modifying/swap_ranges.bench.cpp (+64)
  • (added) libcxx/test/benchmarks/algorithms/modifying/transform.binary.bench.cpp (+73)
  • (added) libcxx/test/benchmarks/algorithms/modifying/transform.unary.bench.cpp (+68)
  • (added) libcxx/test/benchmarks/algorithms/modifying/unique.bench.cpp (+132)
  • (added) libcxx/test/benchmarks/algorithms/modifying/unique_copy.bench.cpp (+105)
  • (added) libcxx/test/benchmarks/algorithms/modifying/unique_copy_pred.bench.cpp (+121)
  • (added) libcxx/test/benchmarks/algorithms/modifying/unique_pred.bench.cpp (+144)
  • (removed) libcxx/test/benchmarks/algorithms/reverse.bench.cpp (-48)
diff --git a/libcxx/include/module.modulemap b/libcxx/include/module.modulemap
index b0720703bd0de..6af6e96af7d75 100644
--- a/libcxx/include/module.modulemap
+++ b/libcxx/include/module.modulemap
@@ -700,6 +700,7 @@ module std [system] {
     }
     module ranges_remove {
       header "__algorithm/ranges_remove.h"
+      export std.ranges.subrange // return type
     }
     module ranges_replace_copy_if {
       header "__algorithm/ranges_replace_copy_if.h"
@@ -724,7 +725,7 @@ module std [system] {
     }
     module ranges_rotate_copy {
       header "__algorithm/ranges_rotate_copy.h"
-      export std.algorithm.in_out_result
+      export std.ranges.subrange // return type
     }
     module ranges_rotate                          { header "__algorithm/ranges_rotate.h" }
     module ranges_sample                          { header "__algorithm/ranges_sample.h" }
diff --git a/libcxx/test/benchmarks/algorithms/fill.bench.cpp b/libcxx/test/benchmarks/algorithms/fill.bench.cpp
deleted file mode 100644
index 6a48b25b7eb63..0000000000000
--- a/libcxx/test/benchmarks/algorithms/fill.bench.cpp
+++ /dev/null
@@ -1,51 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
-
-#include <algorithm>
-#include <benchmark/benchmark.h>
-#include <vector>
-
-static void bm_fill_n_vector_bool(benchmark::State& state) {
-  std::vector<bool> vec1(state.range());
-  for (auto _ : state) {
-    benchmark::DoNotOptimize(vec1);
-    benchmark::DoNotOptimize(std::fill_n(vec1.begin(), vec1.size(), false));
-  }
-}
-BENCHMARK(bm_fill_n_vector_bool)->DenseRange(1, 8)->Range(16, 1 << 20);
-
-static void bm_ranges_fill_n_vector_bool(benchmark::State& state) {
-  std::vector<bool> vec1(state.range());
-  for (auto _ : state) {
-    benchmark::DoNotOptimize(vec1);
-    benchmark::DoNotOptimize(std::ranges::fill_n(vec1.begin(), vec1.size(), false));
-  }
-}
-BENCHMARK(bm_ranges_fill_n_vector_bool)->DenseRange(1, 8)->Range(16, 1 << 20);
-
-static void bm_fill_vector_bool(benchmark::State& state) {
-  std::vector<bool> vec1(state.range());
-  for (auto _ : state) {
-    benchmark::DoNotOptimize(vec1);
-    std::fill(vec1.begin(), vec1.end(), false);
-  }
-}
-BENCHMARK(bm_fill_vector_bool)->DenseRange(1, 8)->Range(16, 1 << 20);
-
-static void bm_ranges_fill_vector_bool(benchmark::State& state) {
-  std::vector<bool> vec1(state.range());
-  for (auto _ : state) {
-    benchmark::DoNotOptimize(vec1);
-    benchmark::DoNotOptimize(std::ranges::fill(vec1, false));
-  }
-}
-BENCHMARK(bm_ranges_fill_vector_bool)->DenseRange(1, 8)->Range(16, 1 << 20);
-
-BENCHMARK_MAIN();
diff --git a/libcxx/test/benchmarks/algorithms/modifying/fill.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/fill.bench.cpp
new file mode 100644
index 0000000000000..64c7364be6549
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/fill.bench.cpp
@@ -0,0 +1,86 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+#include "test_macros.h"
+
+template <class Container, class Operation>
+void bm(std::string operation_name, Operation fill) {
+  auto bench = [fill](auto& st) {
+    std::size_t const size = st.range(0);
+    using ValueType        = typename Container::value_type;
+    ValueType x            = Generate<ValueType>::random();
+    ValueType y            = Generate<ValueType>::random();
+    Container c(size, y);
+
+    for ([[maybe_unused]] auto _ : st) {
+      fill(c.begin(), c.end(), x);
+      std::swap(x, y);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+template <class Operation>
+void bm_vector_bool(std::string operation_name, Operation fill) {
+  auto bench = [fill](auto& st) {
+    std::size_t const size = st.range(0);
+    bool x                 = true;
+    bool y                 = false;
+    std::vector<bool> c(size, y);
+
+    for ([[maybe_unused]] auto _ : st) {
+      fill(c.begin(), c.end(), x);
+      std::swap(x, y);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_fill    = [](auto first, auto last, auto const& value) { return std::fill(first, last, value); };
+  auto ranges_fill = [](auto first, auto last, auto const& value) { return std::ranges::fill(first, last, value); };
+
+  // std::fill
+  bm<std::vector<int>>("std::fill(vector<int>)", std_fill);
+  bm<std::deque<int>>("std::fill(deque<int>)", std_fill);
+  bm<std::list<int>>("std::fill(list<int>)", std_fill);
+  bm_vector_bool("std::fill(vector<bool>)", std_fill);
+
+  // ranges::fill
+  bm<std::vector<int>>("ranges::fill(vector<int>)", ranges_fill);
+  bm<std::deque<int>>("ranges::fill(deque<int>)", ranges_fill);
+  bm<std::list<int>>("ranges::fill(list<int>)", ranges_fill);
+#if TEST_STD_VER >= 23 // vector<bool>::iterator is not an output_iterator before C++23
+  bm_vector_bool("ranges::fill(vector<bool>)", ranges_fill);
+#endif
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/fill_n.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/fill_n.bench.cpp
new file mode 100644
index 0000000000000..aaeb0982956f9
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/fill_n.bench.cpp
@@ -0,0 +1,86 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+#include "test_macros.h"
+
+template <class Container, class Operation>
+void bm(std::string operation_name, Operation fill_n) {
+  auto bench = [fill_n](auto& st) {
+    std::size_t const size = st.range(0);
+    using ValueType        = typename Container::value_type;
+    ValueType x            = Generate<ValueType>::random();
+    ValueType y            = Generate<ValueType>::random();
+    Container c(size, y);
+
+    for ([[maybe_unused]] auto _ : st) {
+      fill_n(c.begin(), size, x);
+      std::swap(x, y);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+template <class Operation>
+void bm_vector_bool(std::string operation_name, Operation fill_n) {
+  auto bench = [fill_n](auto& st) {
+    std::size_t const size = st.range(0);
+    bool x                 = true;
+    bool y                 = false;
+    std::vector<bool> c(size, y);
+
+    for ([[maybe_unused]] auto _ : st) {
+      fill_n(c.begin(), size, x);
+      std::swap(x, y);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_fill_n    = [](auto out, auto n, auto const& value) { return std::fill_n(out, n, value); };
+  auto ranges_fill_n = [](auto out, auto n, auto const& value) { return std::ranges::fill_n(out, n, value); };
+
+  // std::fill_n
+  bm<std::vector<int>>("std::fill_n(vector<int>)", std_fill_n);
+  bm<std::deque<int>>("std::fill_n(deque<int>)", std_fill_n);
+  bm<std::list<int>>("std::fill_n(list<int>)", std_fill_n);
+  bm_vector_bool("std::fill_n(vector<bool>)", std_fill_n);
+
+  // ranges::fill_n
+  bm<std::vector<int>>("ranges::fill_n(vector<int>)", ranges_fill_n);
+  bm<std::deque<int>>("ranges::fill_n(deque<int>)", ranges_fill_n);
+  bm<std::list<int>>("ranges::fill_n(list<int>)", ranges_fill_n);
+#if TEST_STD_VER >= 23 // vector<bool>::iterator is not an output_iterator before C++23
+  bm_vector_bool("ranges::fill_n(vector<bool>)", ranges_fill_n);
+#endif
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/generate.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/generate.bench.cpp
new file mode 100644
index 0000000000000..15b039a4d3009
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/generate.bench.cpp
@@ -0,0 +1,59 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+
+template <class Container, class Operation>
+void bm(std::string operation_name, Operation generate) {
+  auto bench = [generate](auto& st) {
+    std::size_t const size = st.range(0);
+    Container c(size);
+    using ValueType = typename Container::value_type;
+    ValueType x     = Generate<ValueType>::random();
+
+    for ([[maybe_unused]] auto _ : st) {
+      auto f = [&x] { return x; };
+      generate(c.begin(), c.end(), f);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_generate    = [](auto first, auto last, auto f) { return std::generate(first, last, f); };
+  auto ranges_generate = [](auto first, auto last, auto f) { return std::ranges::generate(first, last, f); };
+
+  // std::generate
+  bm<std::vector<int>>("std::generate(vector<int>)", std_generate);
+  bm<std::deque<int>>("std::generate(deque<int>)", std_generate);
+  bm<std::list<int>>("std::generate(list<int>)", std_generate);
+
+  // ranges::generate
+  bm<std::vector<int>>("ranges::generate(vector<int>)", ranges_generate);
+  bm<std::deque<int>>("ranges::generate(deque<int>)", ranges_generate);
+  bm<std::list<int>>("ranges::generate(list<int>)", ranges_generate);
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/generate_n.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/generate_n.bench.cpp
new file mode 100644
index 0000000000000..75b088411810f
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/generate_n.bench.cpp
@@ -0,0 +1,59 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+
+template <class Container, class Operation>
+void bm(std::string operation_name, Operation generate_n) {
+  auto bench = [generate_n](auto& st) {
+    std::size_t const size = st.range(0);
+    Container c(size);
+    using ValueType = typename Container::value_type;
+    ValueType x     = Generate<ValueType>::random();
+
+    for ([[maybe_unused]] auto _ : st) {
+      auto f = [&x] { return x; };
+      generate_n(c.begin(), size, f);
+      benchmark::DoNotOptimize(c);
+      benchmark::DoNotOptimize(x);
+      benchmark::ClobberMemory();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_generate_n    = [](auto out, auto n, auto f) { return std::generate_n(out, n, f); };
+  auto ranges_generate_n = [](auto out, auto n, auto f) { return std::ranges::generate_n(out, n, f); };
+
+  // std::generate_n
+  bm<std::vector<int>>("std::generate_n(vector<int>)", std_generate_n);
+  bm<std::deque<int>>("std::generate_n(deque<int>)", std_generate_n);
+  bm<std::list<int>>("std::generate_n(list<int>)", std_generate_n);
+
+  // ranges::generate_n
+  bm<std::vector<int>>("ranges::generate_n(vector<int>)", ranges_generate_n);
+  bm<std::deque<int>>("ranges::generate_n(deque<int>)", ranges_generate_n);
+  bm<std::list<int>>("ranges::generate_n(list<int>)", ranges_generate_n);
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/remove.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/remove.bench.cpp
new file mode 100644
index 0000000000000..764dfe73ab70c
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/remove.bench.cpp
@@ -0,0 +1,133 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <iterator>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+
+// Create a sequence of the form xxxxxxxxxxyyyyyyyyyy and remove
+// the prefix of x's from it.
+//
+// We perform this benchmark in a batch because we need to restore the
+// state of the container after the operation.
+template <class Container, class Operation>
+void bm_prefix(std::string operation_name, Operation remove) {
+  auto bench = [remove](auto& st) {
+    std::size_t const size          = st.range(0);
+    constexpr std::size_t BatchSize = 10;
+    using ValueType                 = typename Container::value_type;
+    Container c[BatchSize];
+    ValueType x = Generate<ValueType>::random();
+    ValueType y = Generate<ValueType>::random();
+    for (std::size_t i = 0; i != BatchSize; ++i) {
+      c[i]      = Container(size);
+      auto half = size / 2;
+      std::fill_n(std::fill_n(c[i].begin(), half, x), half, y);
+    }
+
+    while (st.KeepRunningBatch(BatchSize)) {
+      for (std::size_t i = 0; i != BatchSize; ++i) {
+        auto result = remove(c[i].begin(), c[i].end(), x);
+        benchmark::DoNotOptimize(result);
+        benchmark::DoNotOptimize(c[i]);
+        benchmark::DoNotOptimize(x);
+        benchmark::ClobberMemory();
+      }
+
+      st.PauseTiming();
+      for (std::size_t i = 0; i != BatchSize; ++i) {
+        auto half = size / 2;
+        std::fill_n(std::fill_n(c[i].begin(), half, x), half, y);
+      }
+      st.ResumeTiming();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+// Create a sequence of the form xyxyxyxyxyxyxyxyxyxy and remove
+// the x's from it.
+//
+// We perform this benchmark in a batch because we need to restore the
+// state of the container after the operation.
+template <class Container, class Operation>
+void bm_sprinkled(std::string operation_name, Operation remove) {
+  auto bench = [remove](auto& st) {
+    std::size_t const size          = st.range(0);
+    constexpr std::size_t BatchSize = 10;
+    using ValueType                 = typename Container::value_type;
+    Container c[BatchSize];
+    ValueType x    = Generate<ValueType>::random();
+    ValueType y    = Generate<ValueType>::random();
+    auto alternate = [&](auto out, auto n) {
+      for (std::size_t i = 0; i != n; ++i) {
+        *out++ = (i % 2 == 0 ? x : y);
+      }
+    };
+    for (std::size_t i = 0; i != BatchSize; ++i) {
+      c[i] = Container(size);
+      alternate(c[i].begin(), size);
+    }
+
+    while (st.KeepRunningBatch(BatchSize)) {
+      for (std::size_t i = 0; i != BatchSize; ++i) {
+        auto result = remove(c[i].begin(), c[i].end(), x);
+        benchmark::DoNotOptimize(result);
+        benchmark::DoNotOptimize(c[i]);
+        benchmark::DoNotOptimize(x);
+        benchmark::ClobberMemory();
+      }
+
+      st.PauseTiming();
+      for (std::size_t i = 0; i != BatchSize; ++i) {
+        alternate(c[i].begin(), size);
+      }
+      st.ResumeTiming();
+    }
+  };
+  benchmark::RegisterBenchmark(operation_name, bench)->Arg(32)->Arg(1024)->Arg(8192);
+}
+
+int main(int argc, char** argv) {
+  auto std_remove    = [](auto first, auto last, auto const& value) { return std::remove(first, last, value); };
+  auto ranges_remove = [](auto first, auto last, auto const& value) { return std::ranges::remove(first, last, value); };
+
+  // std::remove
+  bm_prefix<std::vector<int>>("std::remove(vector<int>) (prefix)", std_remove);
+  bm_sprinkled<std::vector<int>>("std::remove(vector<int>) (sprinkled)", std_remove);
+
+  bm_prefix<std::deque<int>>("std::remove(deque<int>) (prefix)", std_remove);
+  bm_sprinkled<std::deque<int>>("std::remove(deque<int>) (sprinkled)", std_remove);
+
+  bm_prefix<std::list<int>>("std::remove(list<int>) (prefix)", std_remove);
+  bm_sprinkled<std::list<int>>("std::remove(list<int>) (sprinkled)", std_remove);
+
+  // ranges::remove
+  bm_prefix<std::vector<int>>("ranges::remove(vector<int>) (prefix)", ranges_remove);
+  bm_sprinkled<std::vector<int>>("ranges::remove(vector<int>) (sprinkled)", ranges_remove);
+
+  bm_prefix<std::deque<int>>("ranges::remove(deque<int>) (prefix)", ranges_remove);
+  bm_sprinkled<std::deque<int>>("ranges::remove(deque<int>) (sprinkled)", ranges_remove);
+
+  bm_prefix<std::list<int>>("ranges::remove(list<int>) (prefix)", ranges_remove);
+  bm_sprinkled<std::list<int>>("ranges::remove(list<int>) (sprinkled)", ranges_remove);
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/modifying/remove_copy.bench.cpp b/libcxx/test/benchmarks/algorithms/modifying/remove_copy.bench.cpp
new file mode 100644
index 0000000000000..88c05384eb7bd
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/modifying/remove_copy.bench.cpp
@@ -0,0 +1,109 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-...
[truncated]

@siya100
Copy link
Contributor

siya100 commented Feb 18, 2025

can i work on this issue

@ldionne ldionne force-pushed the review/benchmark-add-all-modifying branch from 7d00280 to 55c1ebe Compare February 19, 2025 21:37
Copy link
Member

@mordante mordante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM!

}
})->Range(8, 1 << 20);
};
bm.operator()<std::vector<int>>("std::move(vector<int>)", std_move);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the copy family of algorithms I think it would be interesting to test some non-trivial type. Specifically, that the compiler would not be able to see through the operation. That could be achieved by an out-of-line definition or a [[gnu::noinline]] on the function. Maybe just an optimization barrier would work to.

Copy link
Member Author

@ldionne ldionne Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here too, I would like to defer to a separate patch where I can focus on doing just that for the copy and the move algorithms.

#128737

->Arg(1024)
->Arg(8192);
};
bm.operator()<std::vector<int>>("std::transform(vector<int>, vector<int>)", std_transform);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also add benchmarks for cases where the output is a segmented iterator (i.e. output to a deque) for the _copy versions of algorithms.

Copy link
Member Author

@ldionne ldionne Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but after consideration I would like to tackle that as a separate patch. Indeed, adding this variation of the benchmark requires writing a new batched benchmark since I can't just alternate between the source and the destination range. I'd like to do that in a separate patch that also tackles the benchmarks for std::copy (which have already landed).

#128733

@ldionne ldionne force-pushed the review/benchmark-add-all-modifying branch from 5c563b5 to fd313e5 Compare March 17, 2025 13:36
@ldionne
Copy link
Member Author

ldionne commented Mar 17, 2025

I'm going to land this now since I applied all of @philnik777 's comments except #128733 and #128737. We can do another pass once he's back from vacation, but landing this now will at least ensure we don't have merge conflicts.

@ldionne ldionne merged commit 24e88b0 into llvm:main Mar 17, 2025
85 checks passed
@ldionne ldionne deleted the review/benchmark-add-all-modifying branch March 17, 2025 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants