mirror of
https://gitlab.com/buildroot.org/buildroot.git
synced 2026-08-08 08:30:47 -09:00
docs/manual: rename *.txt as *.adoc
by using this standard extension `adoc`, these files are rendered on gitlab & github Signed-off-by: Francois Perrad <francois.perrad@gadz.org> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit is contained in:
Francois Perrad
committed by
Thomas Petazzoni
Thomas Petazzoni
parent
b1f9c51162
commit
32cec3be97
259
docs/manual/adding-packages-python.adoc
Normal file
259
docs/manual/adding-packages-python.adoc
Normal file
@@ -0,0 +1,259 @@
|
||||
// -*- mode:doc; -*-
|
||||
// vim: set syntax=asciidoc:
|
||||
|
||||
=== Infrastructure for Python packages
|
||||
|
||||
This infrastructure applies to Python packages that use the standard
|
||||
Python setuptools or pep517 mechanisms as their build system, generally
|
||||
recognizable by the usage of a +setup.py+ script or +pyproject.toml+
|
||||
file.
|
||||
|
||||
[[python-package-tutorial]]
|
||||
|
||||
==== +python-package+ tutorial
|
||||
|
||||
First, let's see how to write a +.mk+ file for a Python package,
|
||||
with an example :
|
||||
|
||||
------------------------
|
||||
01: ################################################################################
|
||||
02: #
|
||||
03: # python-foo
|
||||
04: #
|
||||
05: ################################################################################
|
||||
06:
|
||||
07: PYTHON_FOO_VERSION = 1.0
|
||||
08: PYTHON_FOO_SOURCE = python-foo-$(PYTHON_FOO_VERSION).tar.xz
|
||||
09: PYTHON_FOO_SITE = http://www.foosoftware.org/download
|
||||
10: PYTHON_FOO_LICENSE = BSD-3-Clause
|
||||
11: PYTHON_FOO_LICENSE_FILES = LICENSE
|
||||
12: PYTHON_FOO_ENV = SOME_VAR=1
|
||||
13: PYTHON_FOO_DEPENDENCIES = libmad
|
||||
14: PYTHON_FOO_SETUP_TYPE = distutils
|
||||
15:
|
||||
16: $(eval $(python-package))
|
||||
------------------------
|
||||
|
||||
On line 7, we declare the version of the package.
|
||||
|
||||
On line 8 and 9, we declare the name of the tarball (xz-ed tarball
|
||||
recommended) and the location of the tarball on the Web. Buildroot
|
||||
will automatically download the tarball from this location.
|
||||
|
||||
On line 10 and 11, we give licensing details about the package (its
|
||||
license on line 10, and the file containing the license text on line
|
||||
11).
|
||||
|
||||
On line 12, we tell Buildroot to pass custom options to the Python
|
||||
+setup.py+ script when it is configuring the package.
|
||||
|
||||
On line 13, we declare our dependencies, so that they are built
|
||||
before the build process of our package starts.
|
||||
|
||||
On line 14, we declare the specific Python build system being used. In
|
||||
this case the +distutils+ Python build system is used. The four
|
||||
supported ones are +distutils+, +flit+, +pep517+ and +setuptools+.
|
||||
|
||||
Finally, on line 16, we invoke the +python-package+ macro that
|
||||
generates all the Makefile rules that actually allow the package to be
|
||||
built.
|
||||
|
||||
[[python-package-reference]]
|
||||
|
||||
==== +python-package+ reference
|
||||
|
||||
As a policy, packages that merely provide Python modules should all be
|
||||
named +python-<something>+ in Buildroot. Other packages that use the
|
||||
Python build system, but are not Python modules, can freely choose
|
||||
their name (existing examples in Buildroot are +scons+ and
|
||||
+supervisor+).
|
||||
|
||||
The main macro of the Python package infrastructure is
|
||||
+python-package+. It is similar to the +generic-package+ macro. It is
|
||||
also possible to create Python host packages with the
|
||||
+host-python-package+ macro.
|
||||
|
||||
Just like the generic infrastructure, the Python infrastructure works
|
||||
by defining a number of variables before calling the +python-package+
|
||||
or +host-python-package+ macros.
|
||||
|
||||
All the package metadata information variables that exist in the
|
||||
xref:generic-package-reference[generic package infrastructure] also
|
||||
exist in the Python infrastructure: +PYTHON_FOO_VERSION+,
|
||||
+PYTHON_FOO_SOURCE+, +PYTHON_FOO_PATCH+, +PYTHON_FOO_SITE+,
|
||||
+PYTHON_FOO_SUBDIR+, +PYTHON_FOO_DEPENDENCIES+, +PYTHON_FOO_LICENSE+,
|
||||
+PYTHON_FOO_LICENSE_FILES+, +PYTHON_FOO_INSTALL_STAGING+, etc.
|
||||
|
||||
Note that:
|
||||
|
||||
* It is not necessary to add +python+ or +host-python+ in the
|
||||
+PYTHON_FOO_DEPENDENCIES+ variable of a package, since these basic
|
||||
dependencies are automatically added as needed by the Python
|
||||
package infrastructure.
|
||||
|
||||
* Similarly, it is not needed to add +host-setuptools+ to
|
||||
+PYTHON_FOO_DEPENDENCIES+ for setuptools-based packages, since it's
|
||||
automatically added by the Python infrastructure as needed.
|
||||
|
||||
One variable specific to the Python infrastructure is mandatory:
|
||||
|
||||
* +PYTHON_FOO_SETUP_TYPE+, to define which Python build system is used
|
||||
by the package. The four supported values are +distutils+, +flit+,
|
||||
+pep517+ and +setuptools+. If you don't know which one is used in
|
||||
your package, look at the +setup.py+ or +pyproject.toml+ file in your
|
||||
package source code, and see whether it imports things from the
|
||||
+distutils+, +flit+ module or the +setuptools+ module. If the package
|
||||
is using a +pyproject.toml+ file without any build-system requires
|
||||
and with a local in-tree backend-path one should use +pep517+.
|
||||
|
||||
A few additional variables, specific to the Python infrastructure, can
|
||||
optionally be defined, depending on the package's needs. Many of them
|
||||
are only useful in very specific cases, typical packages will
|
||||
therefore only use a few of them, or none.
|
||||
|
||||
* +PYTHON_FOO_SUBDIR+ may contain the name of a subdirectory inside the
|
||||
package that contains the main +setup.py+ or +pyproject.toml+ file.
|
||||
This is useful, if for example, the main +setup.py+ or +pyproject.toml+
|
||||
file is not at the root of the tree extracted by the tarball. If
|
||||
+HOST_PYTHON_FOO_SUBDIR+ is not specified, it defaults to
|
||||
+PYTHON_FOO_SUBDIR+.
|
||||
|
||||
* +PYTHON_FOO_ENV+, to specify additional environment variables to
|
||||
pass to the Python +setup.py+ script (for distutils/setuptools
|
||||
packages) or the +support/scripts/pyinstaller.py+ script (for
|
||||
flit/pep517 packages) for both the build and install steps. Note
|
||||
that the infrastructure is automatically passing several standard
|
||||
variables, defined in +PKG_PYTHON_DISTUTILS_ENV+ (for distutils
|
||||
target packages), +HOST_PKG_PYTHON_DISTUTILS_ENV+ (for distutils
|
||||
host packages), +PKG_PYTHON_SETUPTOOLS_ENV+ (for setuptools target
|
||||
packages), +HOST_PKG_PYTHON_SETUPTOOLS_ENV+ (for setuptools host
|
||||
packages), +PKG_PYTHON_PEP517_ENV+ (for flit/pep517 target packages)
|
||||
and +HOST_PKG_PYTHON_PEP517_ENV+ (for flit/pep517 host packages).
|
||||
|
||||
* +PYTHON_FOO_BUILD_OPTS+, to specify additional options to pass to the
|
||||
Python +setup.py+ script during the build step, this generally only
|
||||
makes sense to use for distutils/setuptools based packages as
|
||||
flit/pep517 based packages do not pass these options to a +setup.py+
|
||||
script but instead pass them to +support/scripts/pyinstaller.py+.
|
||||
For target distutils packages, the +PKG_PYTHON_DISTUTILS_BUILD_OPTS+
|
||||
options are already passed automatically by the infrastructure.
|
||||
|
||||
* +PYTHON_FOO_INSTALL_TARGET_OPTS+, +PYTHON_FOO_INSTALL_STAGING_OPTS+,
|
||||
+HOST_PYTHON_FOO_INSTALL_OPTS+ to specify additional options to pass
|
||||
to the Python +setup.py+ script (for distutils/setuptools packages)
|
||||
or +support/scripts/pyinstaller.py+ (for flit/pep517 packages) during
|
||||
the target installation step, the staging installation step or the
|
||||
host installation, respectively. Note that the infrastructure is
|
||||
automatically passing some options, defined in
|
||||
+PKG_PYTHON_DISTUTILS_INSTALL_TARGET_OPTS+ or
|
||||
+PKG_PYTHON_DISTUTILS_INSTALL_STAGING_OPTS+ (for target distutils
|
||||
packages), +HOST_PKG_PYTHON_DISTUTILS_INSTALL_OPTS+ (for host
|
||||
distutils packages), +PKG_PYTHON_SETUPTOOLS_INSTALL_TARGET_OPTS+ or
|
||||
+PKG_PYTHON_SETUPTOOLS_INSTALL_STAGING_OPTS+ (for target setuptools
|
||||
packages), +HOST_PKG_PYTHON_SETUPTOOLS_INSTALL_OPTS+ (for host
|
||||
setuptools packages) and +PKG_PYTHON_PEP517_INSTALL_TARGET_OPTS+ or
|
||||
+PKG_PYTHON_PEP517_INSTALL_STAGING_OPTS+ (for target flit/pep517
|
||||
packages).
|
||||
|
||||
With the Python infrastructure, all the steps required to build and
|
||||
install the packages are already defined, and they generally work well
|
||||
for most Python-based packages. However, when required, it is still
|
||||
possible to customize what is done in any particular step:
|
||||
|
||||
* By adding a post-operation hook (after extract, patch, configure,
|
||||
build or install). See xref:hooks[] for details.
|
||||
|
||||
* By overriding one of the steps. For example, even if the Python
|
||||
infrastructure is used, if the package +.mk+ file defines its own
|
||||
+PYTHON_FOO_BUILD_CMDS+ variable, it will be used instead of the
|
||||
default Python one. However, using this method should be restricted
|
||||
to very specific cases. Do not use it in the general case.
|
||||
|
||||
[[scanpypi]]
|
||||
|
||||
==== Generating a +python-package+ from a PyPI repository
|
||||
|
||||
If the Python package for which you would like to create a Buildroot
|
||||
package is available on PyPI, you may want to use the +scanpypi+ tool
|
||||
located in +utils/+ to automate the process.
|
||||
|
||||
You can find the list of existing PyPI packages
|
||||
https://pypi.python.org[here].
|
||||
|
||||
+scanpypi+ requires Python's +setuptools+ package to be installed on
|
||||
your host.
|
||||
|
||||
When at the root of your buildroot directory just do :
|
||||
|
||||
-----------------------
|
||||
utils/scanpypi foo bar -o package
|
||||
-----------------------
|
||||
|
||||
This will generate packages +python-foo+ and +python-bar+ in the package
|
||||
folder if they exist on https://pypi.python.org.
|
||||
|
||||
Find the +external python modules+ menu and insert your package inside.
|
||||
Keep in mind that the items inside a menu should be in alphabetical order.
|
||||
|
||||
Please keep in mind that you'll most likely have to manually check the
|
||||
package for any mistakes as there are things that cannot be guessed by
|
||||
the generator (e.g. dependencies on any of the python core modules
|
||||
such as BR2_PACKAGE_PYTHON_ZLIB). Also, please take note that the
|
||||
license and license files are guessed and must be checked. You also
|
||||
need to manually add the package to the +package/Config.in+ file.
|
||||
|
||||
If your Buildroot package is not in the official Buildroot tree but in
|
||||
a br2-external tree, use the -o flag as follows:
|
||||
|
||||
-----------------------
|
||||
utils/scanpypi foo bar -o other_package_dir
|
||||
-----------------------
|
||||
|
||||
This will generate packages +python-foo+ and +python-bar+ in the
|
||||
+other_package_directory+ instead of +package+.
|
||||
|
||||
Option +-h+ will list the available options:
|
||||
|
||||
-----------------------
|
||||
utils/scanpypi -h
|
||||
-----------------------
|
||||
|
||||
[[python-package-cffi-backend]]
|
||||
|
||||
==== +python-package+ CFFI backend
|
||||
|
||||
C Foreign Function Interface for Python (CFFI) provides a convenient
|
||||
and reliable way to call compiled C code from Python using interface
|
||||
declarations written in C. Python packages relying on this backend can
|
||||
be identified by the appearance of a +cffi+ dependency in the
|
||||
+install_requires+ field of their +setup.py+ file.
|
||||
|
||||
Such a package should:
|
||||
|
||||
* add +python-cffi+ as a runtime dependency in order to install the
|
||||
compiled C library wrapper on the target. This is achieved by adding
|
||||
+select BR2_PACKAGE_PYTHON_CFFI+ to the package +Config.in+.
|
||||
|
||||
------------------------
|
||||
config BR2_PACKAGE_PYTHON_FOO
|
||||
bool "python-foo"
|
||||
select BR2_PACKAGE_PYTHON_CFFI # runtime
|
||||
------------------------
|
||||
|
||||
* add +host-python-cffi+ as a build-time dependency in order to
|
||||
cross-compile the C wrapper. This is achieved by adding
|
||||
+host-python-cffi+ to the +PYTHON_FOO_DEPENDENCIES+ variable.
|
||||
|
||||
------------------------
|
||||
################################################################################
|
||||
#
|
||||
# python-foo
|
||||
#
|
||||
################################################################################
|
||||
|
||||
...
|
||||
|
||||
PYTHON_FOO_DEPENDENCIES = host-python-cffi
|
||||
|
||||
$(eval $(python-package))
|
||||
------------------------
|
||||
Reference in New Issue
Block a user