Skip to content

Commit 9290806

Browse files
committed
[libc++][hardening] Use bounded iterators in std::vector and std::string
This partially restores parity with the old, since removed debug build. We now can re-enable a bunch of the disabled tests. Some things of note: - bounded_iter's converting constructor has never worked. It needs a friend declaration to access the other bound_iter instantiation's private fields. - The old debug iterators also checked that callers did not try to compare iterators from different objects. bounded_iter does not currently do this, so I've left those disabled. However, I think we probably should add those. See #78771 (comment) - The std::vector iterators are bounded up to capacity, not size. This makes for a weaker safety check. This is because the STL promises not to invalidate iterators when appending up to the capacity. Since we cannot retroactively update all the iterators on push_back(), I've instead sized it to the capacaity. This is not as good, but at least will stop the iterator from going off the end of the buffer. There was also no test for this, so I've added one in the std directory. - std::string has two ambiguities to deal with. First, I opted not to size it against the capacity. https://eel.is/c++draft/string.require#4 says iterators are invalided on an non-const operation. Second, whether the iterator can reach the NUL terminator. The previous debug tests and the special-case in https://eel.is/c++draft/string.access#2 suggest no. If either of these causes widespread problems, I figure we can revisit. - resize_and_overwrite.pass.cpp assumed std::string's iterator supported s.begin().base(), but I see no promise of this in the standard. GCC also doesn't support this. I fixed the test to use std::to_address. - alignof.compile.pass.cpp's pointer isn't enough of a real pointer. (It needs to satisfy NullablePointer, LegacyRandomAccessIterator, and LegacyContiguousIterator.) __bounded_iter seems to instantiate enough to notice. I've added a few more bits to satisfy it. Fixes #78805
1 parent 49d9168 commit 9290806

17 files changed

+214
-44
lines changed

libcxx/include/__iterator/bounded_iter.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -214,6 +214,8 @@ struct __bounded_iter {
214214
private:
215215
template <class>
216216
friend struct pointer_traits;
217+
template <class, class>
218+
friend struct __bounded_iter;
217219
_Iterator __current_; // current iterator
218220
_Iterator __begin_, __end_; // valid range represented as [begin, end]
219221
};

libcxx/include/string

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -579,6 +579,7 @@ basic_string<char32_t> operator""s( const char32_t *str, size_t len );
579579
#include <__functional/unary_function.h>
580580
#include <__fwd/string.h>
581581
#include <__ios/fpos.h>
582+
#include <__iterator/bounded_iter.h>
582583
#include <__iterator/distance.h>
583584
#include <__iterator/iterator_traits.h>
584585
#include <__iterator/reverse_iterator.h>
@@ -736,9 +737,16 @@ public:
736737
"[allocator.requirements] states that rebinding an allocator to the same type should result in the "
737738
"original allocator");
738739

739-
// TODO: Implement iterator bounds checking without requiring the global database.
740+
#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
741+
// The pointer must be passed through __wrap_iter because
742+
// __alloc_traits::pointer may not be detected as a continguous iterator on
743+
// its own.
744+
typedef __bounded_iter<__wrap_iter<pointer>> iterator;
745+
typedef __bounded_iter<__wrap_iter<const_pointer>> const_iterator;
746+
#else
740747
typedef __wrap_iter<pointer> iterator;
741748
typedef __wrap_iter<const_pointer> const_iterator;
749+
#endif
742750
typedef std::reverse_iterator<iterator> reverse_iterator;
743751
typedef std::reverse_iterator<const_iterator> const_reverse_iterator;
744752

@@ -868,11 +876,27 @@ private:
868876
__init_with_sentinel(std::move(__first), std::move(__last));
869877
}
870878

879+
#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
880+
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) {
881+
return __make_bounded_iter(
882+
__wrap_iter<pointer>(__p),
883+
__wrap_iter<pointer>(__get_pointer()),
884+
__wrap_iter<pointer>(__get_pointer() + size()));
885+
}
886+
887+
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_iterator __make_const_iterator(const_pointer __p) const {
888+
return __make_bounded_iter(
889+
__wrap_iter<const_pointer>(__p),
890+
__wrap_iter<const_pointer>(__get_pointer()),
891+
__wrap_iter<const_pointer>(__get_pointer() + size()));
892+
}
893+
#else
871894
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) { return iterator(__p); }
872895

873896
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_iterator __make_const_iterator(const_pointer __p) const {
874897
return const_iterator(__p);
875898
}
899+
#endif
876900

877901
public:
878902
_LIBCPP_TEMPLATE_DATA_VIS static const size_type npos = -1;

libcxx/include/vector

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -326,6 +326,7 @@ template<class T, class charT> requires is-vector-bool-reference<T> // Since C++
326326
#include <__functional/hash.h>
327327
#include <__functional/unary_function.h>
328328
#include <__iterator/advance.h>
329+
#include <__iterator/bounded_iter.h>
329330
#include <__iterator/distance.h>
330331
#include <__iterator/iterator_traits.h>
331332
#include <__iterator/reverse_iterator.h>
@@ -399,9 +400,16 @@ public:
399400
typedef typename __alloc_traits::difference_type difference_type;
400401
typedef typename __alloc_traits::pointer pointer;
401402
typedef typename __alloc_traits::const_pointer const_pointer;
402-
// TODO: Implement iterator bounds checking without requiring the global database.
403+
#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
404+
// The pointer must be passed through __wrap_iter because
405+
// __alloc_traits::pointer may not be detected as a continguous iterator on
406+
// its own.
407+
typedef __bounded_iter<__wrap_iter<pointer>> iterator;
408+
typedef __bounded_iter<__wrap_iter<const_pointer>> const_iterator;
409+
#else
403410
typedef __wrap_iter<pointer> iterator;
404411
typedef __wrap_iter<const_pointer> const_iterator;
412+
#endif
405413
typedef std::reverse_iterator<iterator> reverse_iterator;
406414
typedef std::reverse_iterator<const_iterator> const_reverse_iterator;
407415

@@ -796,10 +804,30 @@ private:
796804
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void __append(size_type __n);
797805
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void __append(size_type __n, const_reference __x);
798806
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI iterator __make_iter(pointer __p) _NOEXCEPT {
807+
#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
808+
// Bound the iterator according to the capacity, rather than the size.
809+
// Resizing a vector up to the capacity will not invalidate iterators, so,
810+
// Without a way to update all live iterators on resize, we must
811+
// conservatively bound the iterator by the capacity rather than the size.
812+
return __make_bounded_iter(
813+
__wrap_iter<pointer>(__p), __wrap_iter<pointer>(this->__begin_), __wrap_iter<pointer>(this->__end_cap()));
814+
#else
799815
return iterator(__p);
816+
#endif
800817
}
801818
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI const_iterator __make_iter(const_pointer __p) const _NOEXCEPT {
819+
#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
820+
// Bound the iterator according to the capacity, rather than the size.
821+
// Resizing a vector up to the capacity will not invalidate iterators, so,
822+
// Without a way to update all live iterators on resize, we must
823+
// conservatively bound the iterator by the capacity rather than the size.
824+
return __make_bounded_iter(
825+
__wrap_iter<const_pointer>(__p),
826+
__wrap_iter<const_pointer>(this->__begin_),
827+
__wrap_iter<const_pointer>(this->__end_cap()));
828+
#else
802829
return const_iterator(__p);
830+
#endif
803831
}
804832
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void
805833
__swap_out_circular_buffer(__split_buffer<value_type, allocator_type&>& __v);

libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,36 +10,48 @@
1010

1111
// Add to iterator out of bounds.
1212

13-
// REQUIRES: has-unix-headers
14-
// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
13+
// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
14+
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
1515

1616
#include <vector>
1717
#include <cassert>
1818

1919
#include "check_assertion.h"
2020
#include "min_allocator.h"
2121

22+
template <typename T, typename A>
23+
void fill_to_capacity(std::vector<T, A>& vec) {
24+
// Fill vec up to its capacity. Our bounded iterators currently unable to
25+
// catch accesses between size and capacity due to iterator stability
26+
// guarantees. This function clears those effects.
27+
while (vec.size() < vec.capacity()) {
28+
vec.push_back(T());
29+
}
30+
}
31+
2232
int main(int, char**) {
2333
{
2434
typedef int T;
2535
typedef std::vector<T> C;
2636
C c(1);
37+
fill_to_capacity(c);
2738
C::iterator i = c.begin();
28-
i += 1;
39+
i += c.size();
2940
assert(i == c.end());
3041
i = c.begin();
31-
TEST_LIBCPP_ASSERT_FAILURE(i + 2, "Attempted to add/subtract an iterator outside its valid range");
42+
TEST_LIBCPP_ASSERT_FAILURE(i + 2, "__bounded_iter::operator+=: Attempt to advance an iterator past the end");
3243
}
3344

3445
{
3546
typedef int T;
3647
typedef std::vector<T, min_allocator<T> > C;
3748
C c(1);
49+
fill_to_capacity(c);
3850
C::iterator i = c.begin();
39-
i += 1;
51+
i += c.size();
4052
assert(i == c.end());
4153
i = c.begin();
42-
TEST_LIBCPP_ASSERT_FAILURE(i + 2, "Attempted to add/subtract an iterator outside its valid range");
54+
TEST_LIBCPP_ASSERT_FAILURE(i + 2, "__bounded_iter::operator+=: Attempt to advance an iterator past the end");
4355
}
4456

4557
return 0;

libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@
1010

1111
// Decrement iterator prior to begin.
1212

13-
// REQUIRES: has-unix-headers
14-
// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
13+
// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
14+
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
1515

1616
#include <vector>
1717
#include <cassert>
@@ -27,7 +27,7 @@ int main(int, char**) {
2727
C::iterator i = c.end();
2828
--i;
2929
assert(i == c.begin());
30-
TEST_LIBCPP_ASSERT_FAILURE(--i, "Attempted to decrement a non-decrementable iterator");
30+
TEST_LIBCPP_ASSERT_FAILURE(--i, "__bounded_iter::operator--: Attempt to rewind an iterator past the start");
3131
}
3232

3333
{
@@ -37,7 +37,7 @@ int main(int, char**) {
3737
C::iterator i = c.end();
3838
--i;
3939
assert(i == c.begin());
40-
TEST_LIBCPP_ASSERT_FAILURE(--i, "Attempted to decrement a non-decrementable iterator");
40+
TEST_LIBCPP_ASSERT_FAILURE(--i, "__bounded_iter::operator--: Attempt to rewind an iterator past the start");
4141
}
4242

4343
return 0;

libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,29 +10,41 @@
1010

1111
// Dereference non-dereferenceable iterator.
1212

13-
// REQUIRES: has-unix-headers
14-
// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
13+
// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
14+
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
1515

1616
#include <vector>
1717

1818
#include "check_assertion.h"
1919
#include "min_allocator.h"
2020

21+
template <typename T, typename A>
22+
void fill_to_capacity(std::vector<T, A>& vec) {
23+
// Fill vec up to its capacity. Our bounded iterators currently unable to
24+
// catch accesses between size and capacity due to iterator stability
25+
// guarantees. This function clears those effects.
26+
while (vec.size() < vec.capacity()) {
27+
vec.push_back(T());
28+
}
29+
}
30+
2131
int main(int, char**) {
2232
{
2333
typedef int T;
2434
typedef std::vector<T> C;
2535
C c(1);
36+
fill_to_capacity(c);
2637
C::iterator i = c.end();
27-
TEST_LIBCPP_ASSERT_FAILURE(*i, "Attempted to dereference a non-dereferenceable iterator");
38+
TEST_LIBCPP_ASSERT_FAILURE(*i, "__bounded_iter::operator*: Attempt to dereference an iterator at the end");
2839
}
2940

3041
{
3142
typedef int T;
3243
typedef std::vector<T, min_allocator<T> > C;
3344
C c(1);
45+
fill_to_capacity(c);
3446
C::iterator i = c.end();
35-
TEST_LIBCPP_ASSERT_FAILURE(*i, "Attempted to dereference a non-dereferenceable iterator");
47+
TEST_LIBCPP_ASSERT_FAILURE(*i, "__bounded_iter::operator*: Attempt to dereference an iterator at the end");
3648
}
3749

3850
return 0;

libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,34 +10,46 @@
1010

1111
// Increment iterator past end.
1212

13-
// REQUIRES: has-unix-headers
14-
// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
13+
// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
14+
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
1515

1616
#include <vector>
1717
#include <cassert>
1818

1919
#include "check_assertion.h"
2020
#include "min_allocator.h"
2121

22+
template <typename T, typename A>
23+
void fill_to_capacity(std::vector<T, A>& vec) {
24+
// Fill vec up to its capacity. Our bounded iterators currently unable to
25+
// catch accesses between size and capacity due to iterator stability
26+
// guarantees. This function clears those effects.
27+
while (vec.size() < vec.capacity()) {
28+
vec.push_back(T());
29+
}
30+
}
31+
2232
int main(int, char**) {
2333
{
2434
typedef int T;
2535
typedef std::vector<T> C;
2636
C c(1);
37+
fill_to_capacity(c);
2738
C::iterator i = c.begin();
28-
++i;
39+
i += c.size();
2940
assert(i == c.end());
30-
TEST_LIBCPP_ASSERT_FAILURE(++i, "Attempted to increment a non-incrementable iterator");
41+
TEST_LIBCPP_ASSERT_FAILURE(++i, "__bounded_iter::operator++: Attempt to advance an iterator past the end");
3142
}
3243

3344
{
3445
typedef int T;
3546
typedef std::vector<T, min_allocator<T> > C;
3647
C c(1);
48+
fill_to_capacity(c);
3749
C::iterator i = c.begin();
38-
++i;
50+
i += c.size();
3951
assert(i == c.end());
40-
TEST_LIBCPP_ASSERT_FAILURE(++i, "Attempted to increment a non-incrementable iterator");
52+
TEST_LIBCPP_ASSERT_FAILURE(++i, "__bounded_iter::operator++: Attempt to advance an iterator past the end");
4153
}
4254

4355
return 0;

libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,32 +10,50 @@
1010

1111
// Index iterator out of bounds.
1212

13-
// REQUIRES: has-unix-headers
14-
// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
13+
// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
14+
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
1515

1616
#include <vector>
1717
#include <cassert>
1818

1919
#include "check_assertion.h"
2020
#include "min_allocator.h"
2121

22+
template <typename T, typename A>
23+
void fill_to_capacity(std::vector<T, A>& vec) {
24+
// Fill vec up to its capacity. Our bounded iterators currently unable to
25+
// catch accesses between size and capacity due to iterator stability
26+
// guarantees. This function clears those effects.
27+
while (vec.size() < vec.capacity()) {
28+
vec.push_back(T());
29+
}
30+
}
31+
2232
int main(int, char**) {
2333
{
2434
typedef int T;
2535
typedef std::vector<T> C;
2636
C c(1);
37+
fill_to_capacity(c);
2738
C::iterator i = c.begin();
2839
assert(i[0] == 0);
29-
TEST_LIBCPP_ASSERT_FAILURE(i[1], "Attempted to subscript an iterator outside its valid range");
40+
TEST_LIBCPP_ASSERT_FAILURE(
41+
i[c.size()], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
42+
TEST_LIBCPP_ASSERT_FAILURE(
43+
i[c.size() + 1], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
3044
}
3145

3246
{
3347
typedef int T;
3448
typedef std::vector<T, min_allocator<T> > C;
3549
C c(1);
50+
fill_to_capacity(c);
3651
C::iterator i = c.begin();
3752
assert(i[0] == 0);
38-
TEST_LIBCPP_ASSERT_FAILURE(i[1], "Attempted to subscript an iterator outside its valid range");
53+
TEST_LIBCPP_ASSERT_FAILURE(
54+
i[c.size()], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
55+
TEST_LIBCPP_ASSERT_FAILURE(
56+
i[c.size() + 1], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
3957
}
4058

4159
return 0;

libcxx/test/libcxx/strings/basic.string/alignof.compile.pass.cpp

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010

1111
// UNSUPPORTED: c++03
1212

13+
#include <iterator>
1314
#include <string>
1415

1516
#include "test_macros.h"
@@ -18,6 +19,14 @@
1819

1920
template <class T>
2021
class small_pointer {
22+
public:
23+
using value_type = T;
24+
using difference_type = std::int16_t;
25+
using pointer = small_pointer;
26+
using reference = T&;
27+
using iterator_category = std::random_access_iterator_tag;
28+
29+
private:
2130
std::uint16_t offset;
2231
};
2332

0 commit comments

Comments
 (0)