build bindings for blosc, lz4, and zstd are apparently not functional (failing the test suite)

[mcepl]

Problem description

When building the package of numcodecs for openSUSE, we run the test suite and it fails miserably, because apparently it cannot find bindings for blosc, lz4, and zstd:

[  128s] ==================================== ERRORS ====================================
[  128s] ________________ ERROR collecting numcodecs/tests/test_blosc.py ________________
[  128s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/numcodecs-0.6.4/numcodecs/tests/test_blosc.py'.
[  128s] Hint: make sure your test modules/packages have valid Python names.
[  128s] Traceback:
[  128s] numcodecs/tests/test_blosc.py:12: in <module>
[  128s]     from numcodecs import blosc
[  128s] E   ImportError: cannot import name blosc
[  128s] _________________ ERROR collecting numcodecs/tests/test_lz4.py _________________
[  128s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/numcodecs-0.6.4/numcodecs/tests/test_lz4.py'.
[  128s] Hint: make sure your test modules/packages have valid Python names.
[  128s] Traceback:
[  128s] numcodecs/tests/test_lz4.py:9: in <module>
[  128s]     from numcodecs.lz4 import LZ4
[  128s] E   ImportError: No module named lz4
[  128s] ________________ ERROR collecting numcodecs/tests/test_zstd.py _________________
[  128s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/numcodecs-0.6.4/numcodecs/tests/test_zstd.py'.
[  128s] Hint: make sure your test modules/packages have valid Python names.
[  128s] Traceback:
[  128s] numcodecs/tests/test_zstd.py:9: in <module>
[  128s]     from numcodecs.zstd import Zstd
[  128s] E   ImportError: No module named zstd
[  128s] !!!!!!!!!!!!!!!!!!! Interrupted: 3 errors during collection !!!!!!!!!!!!!!!!!!!!
[  128s] ====================== 4 skipped, 3 error in 1.03 seconds ======================
[  128s] error: Bad exit status from /var/tmp/rpm-tmp.ZF7poj (%check)
[  128s] 
[  128s]

Version and installation information

Please provide the following:

• Value of numcodecs.__version__ 0.6.4
• Version of Python interpreter 2.7.17 (or 3.7.3)
• Operating system (Linux/Windows/Mac) Linux/openSUSE/Tumbledweed
• How NumCodecs was installed (e.g., "using pip into virtual environment", or "using conda")
  Building from scratch using the tarball from PyPI. See this full build log with all details.

When doing ldd on the build library, it looks reasonably well:

# ldd blosc.cpython-37m-x86_64-linux-gnu.so 
	linux-vdso.so.1 (0x00007ffe841bd000)
	libpython3.7m.so.1.0 => /usr/lib64/libpython3.7m.so.1.0 (0x00007f48fc56c000)
	libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f48fc38a000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f48fc370000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f48fc34e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f48fc187000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f48fc182000)
	libutil.so.1 => /lib64/libutil.so.1 (0x00007f48fc17b000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f48fc036000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f48fc9b1000)
#

However, when I try to import the library into Python, I get:

stitny:/home/abuild/rpmbuild/BUILD/numcodecs-0.6.4 # python3
Python 3.7.3 (default, Apr 09 2019, 05:18:21) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numcodecs.blosc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numcodecs.blosc'
>>>

[jakirkham]

Have you initialized the Git submodules?

[mcepl]

Have you initialized the Git submodules?

Built it from the tarball on PyPI (and besides, build system for openSUSE doesn't have access to the network, for understandable reasons).

[jakirkham]

Why is PYTHONPATH being set in the build?

[mcepl]

Why is PYTHONPATH being set in the build?

Because the library is installed in the temporary root, where the RPM package will be collected from.

[jakirkham]

Is it possible to not use PYTHONPATH?

[mcepl]

Is it possible to not use PYTHONPATH?

Where would the test suite found those libraries? In the build tree?

[jakirkham]

Normally the Python interpreter knows to search for site-packages relative to where it is. So setting PYTHONPATH shouldn't be needed.

[jakirkham]

Could you please share the .spec file as well?

[mcepl]

https://build.opensuse.org/package/show/home:mcepl:branches:devel:languages:python:numeric/python-numcodecs

[QuLogic]

The build log shows you are building for x86_64 and installed to /usr/lib64, e.g.,

[  126s] creating /home/abuild/rpmbuild/BUILDROOT/python-numcodecs-0.6.4-0.x86_64/usr/lib64/python3.7/site-packages/numcodecs

But your PYTHONPATH is set to /usr/lib:

[  127s] + PYTHONPATH=:/home/abuild/rpmbuild/BUILDROOT/python-numcodecs-0.6.4-0.x86_64/usr/lib/python2.7/site-packages

So this is a downstream packaging bug, and since OBS is building fine now, I think the can be closed.