Controll about what is installed into binary wheel

[peter-urban]

Currently "pip wheel -w dist ." installs also all headers and static libraries into the binary wheel for my project.
This is unnecessary and also lets the builds fail on windows ('installing libraries (in my case a pkg-config file) is only supported in POSIX so far').

Is it possibly to control this behavior?

In "meson install" I can control what is install using tags e.g. meson install --tags python-runtime.
Maybe supporting this option would be a solution?

If you share your thoughts on this and give me some directions I'd be willing to work on such a feature.

Thanks for this great Project!
Peter

[rgommers]

@peter-urban thanks, this is interesting. Is your project public? Would be nice to see it. Otherwise, could you expand on how your meson.build files look like for the feature that's triggering this check, and why you don't need it installed?

Is it possibly to control this behavior?

IIUC, not right now. But it sounds like it should be possible.

[peter-urban]

@rgommers: thx for your response
The project is public here: https://github.com/themachinethatgoesping/themachinethatgoesping
It is very new; we are currently building up it's infrastructure.
The relevant meson.build are within this subproject: https://github.com/themachinethatgoesping/tools
I simply marked the targets I want to be installed in my python environment using mesons "install_tag : 'python-runtime'"

Admittedly I am new python packaging. So it is possible that I misunderstand some of the concepts.
But to me it seems unnecessary clutter to install c++ headers and static libraries within a wheel that contains a build c++ binary module that works without these files.

Otherwise my request was triggered by a current practical problem:
Mesonpy fails with 'Bundling libraries in wheel is currently only supported in POSIX!' if it tries to copy e.g. a librarie's pkg-config file into the wheel on windows or mac.

[FFY00]

Hi @peter-urban, thank you for raising this issue. I think this use-case could be meet by allowing users to specify Meson options, which is something we do want to do, but the design and UX around such settings is complex and so, I haven't got around to settling on a design.

[rgommers]

Admittedly I am new python packaging. So it is possible that I misunderstand some of the concepts.
But to me it seems unnecessary clutter to install c++ headers and static libraries within a wheel that contains a build c++ binary module that works without these files.

So this is the meson.build file that contains things you don't want in your wheel: https://github.com/themachinethatgoesping/tools/blob/a11c2b18274386919c5d00c95458b40df0c2b974/src/tools/meson.build

One question I have is: why is that stuff there inside your top-level Python project, and why do you want it installed when you invoke meson directly but not installed when you do pip install .? The structure is a little unusual.

[peter-urban]

So this is the meson.build file that contains things you don't want in your wheel: https://github.com/themachinethatgoesping/tools/blob/a11c2b18274386919c5d00c95458b40df0c2b974/src/tools/meson.build

Yes, but I need to correct the file. I understood now that the install tags ('devel' for static libraries 'runtime' for shared libraries) are already implicitly set by meson (https://mesonbuild.com/Installing.html#installation-tags)

We implemented a c++ library (src/tools) (which can be used independent of the python components) and a pybind11 module interface (src/tools_pymodule) in the same overarching meson project. You can theoretically use the library completely without any python related stuff by setting the correct build option.
I like the idea that you can configure what you want to install using mesons install tags. E.g. meson install --tags runtime,python-runtime for a python module without dev headers.
Or meson install --tags runtime,devel for a dev build that allows to use the library within a c++ project

[rgommers]

Yes, but I need to correct the file. I understood now that the install tags ('devel' for static libraries 'runtime' for shared libraries) are already implicitly set by meson (https://mesonbuild.com/Installing.html#installation-tags)

Ah, that's fairly new, interesting. Had not seen that yet. For static_library the default is already install: false (see https://mesonbuild.com/Reference-manual_functions.html#arguments45), so if it's explicitly set to true then I think it is reasonable for python-meson to try to install it by default (and hence, install all tags). And then provide an interface to select tags for further per-package customization (as part of a generic "pass options to meson"), as mentioned by @FFY00 above.

[peter-urban]

That sounds like very reasonable behavior.
Thanks for your responses. I'll just until mesonpy has the possibility to pass through option.
If there is something I can do, let me know

[peter-urban]

Hi @FFY00,

if I understand correctly this PR (from you) may solve my problem: ##167
However, I don't yet understand how to apply it.

Is it now possible to tell meson-python to set the "--tags python-runtime" option for "meson install" command in the pyproject.toml? And if so, could you show how? :-)

Best,
Peter

[peter-urban]

Ok, I figured out my previous question. Unfortunately this does not yet solve the problem

Since meson-python 0.11 ( PR: #167 ) it is possible to pass custom build options to meson.
It is thus now possible pass tags to meson install: https://mesonbuild.com/Installing.html#installation-tags

The following section in the pyproject.toml will pass "--tags python_runtime,runtime" to the meson install command:

[tool.meson-python.args]
dist = []
setup = []
compile = []
install = ['--tags=runtime,python-runtime'] 

This causes meson-python call meson install --tags runtime,python-runtime which only installs targets that are tagged with 'runtime' or 'python-runtime'

So far so good. Unfortunately, meson-python 0.11 does not only copy the files that are installed by the 'meson install' command.
Rather, it scans through the 'meson-info/into-install_plan.json' and copies all files that can be found within this file. It does thereby not react on the target tags that are also specified within this file

I currently see two possible solutions:

1. The logic of meson-python could be changed to only copy the files installed with 'meson install'
2. The _install_plan function/property could be changed to filter the install_plan by user specified install-tags

@FFY00: In my fork I implemented the second solution. Does it make sense to create a pull request for this, or do you prefer a different solution anyways?

[rgommers]

Is this completely addressed now that gh-288 is merged? If so, do we need some docs?

[peter-urban]

This is issue is completely addressed now (closing).
Documentation would be nice I suppose. I can prepare a draft (once I find some time) possibly here? https://github.com/mesonbuild/meson-python/blob/main/docs/how-to-guides/meson-args.rst

Until this documentation exists, I'll put it here:

---

It is now (meson-python 0.14?) possible to use meson install_tags (https://mesonbuild.com/Installing.html#installation-tags) to select what is to be installed into the wheel. All meson targets have a default install_tag (e.g. 'runtime' for shared libraries, 'devel' for static libraries, or 'python-runtime' for python sources (there is currently an issue with the last statement (mesonbuild/meson#11352). Install tags can be manually overwritten for each target by using the install_tag option.

If a user wants meson-python to only install targets with specific tags (e.g. only 'runtime' and 'python-runtime') into the binary wheel, this can be achieved by adding the following lines to the pyproject.toml

[tool.meson-python.args]
dist = []
setup = []
compile = []
install = ['--tags=runtime,python-runtime']

Note: currently all four option categories (dist, setup, compile and install) must be present in the [tool.meson-python.args] section even if no arguments are supplied (empty list: [])