BUG: test_constructor_does_not_cast_to_float fails on aarch64, armv7 and armv6

[ggardet]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

Run test_constructor_does_not_cast_to_float on aarch64, armv7 and armv6

Issue Description

On openSUSE Tumbleweed test_constructor_does_not_cast_to_float fails for aarch64, armv7 and armv6.
Log:

[ 8767s] =================================== FAILURES ===================================
[ 8767s] ___________ TestUInt64Index.test_constructor_does_not_cast_to_float ____________
[ 8767s] [gw1] linux -- Python 3.9.7 /usr/bin/python3.9
[ 8767s] [XPASS(strict)] https://github.com/numpy/numpy/issues/19146

Expected Behavior

Test should pass on aarch64, armv7 and armv6.

Installed Versions

Numpy 1.21.2