Open
Description
- calls to
ArrayBuffer#ensureSize(this.size + n)without ensuring that the sum doesn't overflowInt.MaxValue.- see:
- the API of
ArrayBuffer#ensureSizeonly taking a single parameter is poor for many use cases, as it encourages the unsafe behaviour of addingInts together to get the new size. changing the parameter to aLongwouldn't help, as the implicit conversion fromInttoLongwill happen after the unsafe sum is performed.
ArrayBuffer.ensureSize(and probablyArrayBuffer.downsizeas well) checks bounds poorly.- the current number of elements (before resizing) is named
end, which is confusing. - we throw an exception iff the current size (not the size that we're ensuring) is exactly
Int.MaxValue, regardless of whether or not the size we're ensuring is or exceedsInt.MaxValue. this is inconsistent at best.- the exception message is
s"Collections can not have more than ${Int.MaxValue} elements", but we actually checked if the (wrong) size wasInt.MaxValue, not if it exceeded it. - the parameter
nfor the size we're ensuring is anInt(as isend), and therefore by definition cannot ever exceedInt.MaxValue; consequently we cannot actually detect overflow anyway.
- the exception message is
- if the new array size (which attempts to be a power of 2, but may not be—see below) exceeds
Int.MaxValue, we drop it down toInt.MaxValue. hotspot allows a maximum array size ofInt.MaxValue - 2, and thus throw anOutOfMemoryErroreven if you had enough memory for the array. this is poor behaviour for the API if the actual size we're trying to ensure is less than the VM array size limit.scala> new Array[Int](Int.MaxValue) java.lang.OutOfMemoryError: Requested array size exceeds VM limit ... 32 elided scala> new Array[Int](Int.MaxValue - 1) java.lang.OutOfMemoryError: Requested array size exceeds VM limit scala> new Array[Int](Int.MaxValue - 2) // my REPL doesn't have enough memory for this java.lang.OutOfMemoryError: Java heap space
- the current number of elements (before resizing) is named
- if
new ArrayBuffer(initialSize)is called with a size that is—orArrayBuffer.fromis called with a collection with aknownSizethat is—greater thanArrayBuffer.DefaultInitialSizebut not a power of 2, the buffer will be backed by an array that's not a power of 2. additionally calls to bothArrayBuffer.ensureSizeandArrayBuffer.downsizeare likely to resize the backing array to non-power-of-2 sizes.
there is also some argument to be made that:
ArrayBuffer#trimToSize()should allow the backing array to have non-power-of-2-sizes (perhaps even less thanArrayBuffer.DefaultInitialValue?)ArrayBuffer#sizeHint(Int)should perhaps allow the backing array to have non-power-of-2-sizes as well? (it also could use a better name that's not like theBuilderAPI since it is no longer aBuilderas of 2.13, but that's not fixable soon)
however, I don't think those necessarily need to be part of the ticket (they're just reasonable to change if we're already in the area making fixes)