72 Commits

Author SHA1 Message Date
Peter Korsgaard
279cbbdb64 utils/check-package: add a check for 'default n' in Config.in files
Boolean Config.in symbols default to 'n', so we typically do not add such
redundant lines.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
2025-11-15 21:56:10 +01:00
Titouan Christophe
72c95d947f package/aufs{-util}: remove package
Remove support for aufs, as it is outdated and unmaintained in Buildroot.
Users are encouraged to use overlayfs in place of aufs.

This patch removes support for aufs package providing sources for
the kernel extension, the related Buildroot kernel extension configuration,
as well as the userspace utilities.

This also allows to remove some special cases in configuration utilities.

Signed-off-by: Titouan Christophe <titouan.christophe@mind.be>
[Romain: fix legacy handling text]
Signed-off-by: Romain Naour <romain.naour@smile.fr>
2025-09-10 23:57:12 +02:00
Romain Naour
4f78d362aa utils/checkpackagelib: fix shellcheck tests after update to shellcheck 0.9.0
shellcheck tests needs to be updated after shellcheck 0.9.0 update due
to changes in errors/warning reporting:

  utils/checkpackagelib/test_tool.py::test_Shellcheck[missing shebang-empty.sh--expected0] FAILED [ 98%]
  utils/checkpackagelib/test_tool.py::test_Shellcheck[2 warnings-unused.sh-unused=""-expected3] FAILED [ 99%]
  utils/checkpackagelib/test_tool.py::test_Shellcheck[tab-tab.sh-\t#!/bin/sh-expected4] FAILED [100%]

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2025-02-09 12:53:35 +01:00
Yann E. MORIN
f9e6d39e8e utils/check-package: require exactly 1 TAB and 2 SP on help text 1st line
Test that the first line starts exactly with one TAB and exactly two
spaces before the text.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Florian Fainelli <f.fainelli@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@datacom.com.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-09-14 22:24:55 +02:00
Yann E. MORIN
32b4912a75 utils/checkpackagelib: instruct shellcheck to follow included files
We do have a few shell scripts that have source directives (either with
'source' or with '.'), and they all are currently either globally
ignored in .checkpackageignore, or have shellcheck directives to ignore
the source statement (SC1090).

So, we can safely instruct shellcheck where to lookup for sourced files;
that does not trigger any new error.

This will alow fixing some shellcheck errors in later commits.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@datacom.com.br>
[Arnout: use long option]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-07-14 10:09:07 +02:00
Yann E. MORIN
fc2dedcfc6 utils/check-package: add option to run the test-suite
Currently, how to run the check-package test-suite is thoroughly under-
documented. There is one hint in the commit log for commit fc254881e6
(utils/checkpackagelib: add unit tests), and another in commit
242e9d72e7 (utils/docker-run: new script); the former is hard to find,
and the latter is about an unrelated script, so harder yet to find...

Add a new option to check-package, that will make it run its test-suite.
Since pytest is only needed for the test-suite, only import it in that
case.

pytest will be default create a cache of the tests (not sure what it is
about, though), so instruct it not to (esp. since the tree can be
read-only).

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@datacom.com.br>
[Arnout: move the code into checkpackagelib.base]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-07-14 10:09:06 +02:00
Yann E. MORIN
5836b79762 utils/checkpackagelib: extend hint about unprefixed variables
User may get confused when they see the current hint, and take that as
the proper replacement, while we're only reporting the stem of the
variable name:

    .../foo.mk:16: possible typo: BLA -> *FOO*

There is usually no easy way to actually suggest the proper variable
name, though, so let's make it a little bit more obvious that we meant
the variable was improperly prefixed:

    .../foo.mk:16: possible typo, variable not properly prefixed: BLA -> *FOO_XXX*

And while at it, throw in the URL to the corresponding manual entry.

Adapt the test accordingly.

Reported-by: "Frager, Neal" <neal.frager@amd.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@datacom.com.br>
Reviewed-by: Neal Frager <neal.frager@amd.com>
[Arnout: also update new test, scoped -> prefixed]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-07-14 08:53:40 +02:00
Yann E. MORIN
b0964df557 utils/checkpackagelib: really check variable name
Currently, when a package defines an unprefixed variable, but its value
contains a properly prefixed expansion (or even just the name of a
variable), there is not error reported (e.g. with the recently fixed
composer issue):

    BASE_SITE = https://getcomposer.org/download/$(COMPOSER_VERSION)/composer.phar

The reason is that he check is done on the whole line, rather than on
the variable that is being set.

We fix that by really looking at the variable we found, instead of
looking in the whole line.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@datacom.com.br>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-07-14 08:53:36 +02:00
Ricardo Martincoski
ccc12c0f24 utils/check-package: check linux-tools
Each linux tool uses a fragment of a .mk file, named, for instance:
package/linux-tools/linux-tool-cpupower.mk.in
So currently check-package does not check these files.

Add the support in check-package script.
At the same time, factor out a function to derive package prefix from
the filename being checked, so the fix (calling os.path.splitext twice)
can be applied in a single place.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: add docstring and explain double splitext to
         get_package_prefix_from_filename]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-07-12 20:30:40 +02:00
Ricardo Martincoski
cca315f814 utils/check-package: fix check for <pkg>_REDISTRIBUTE
Commit "8e87d76c12 check-package: check *.mk files" added a typo that
prevents check-package from checking <PKG>_REDISTRIBUTE default value.

Fix the typo: _INSTALL_REDISTRIBUTE -> _REDISTRIBUTE

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-07-12 20:30:36 +02:00
Yann E. MORIN
7931f78c16 utils/check-package: new check for Buildroot's defconfig files
Now that we do have support for checking hashes for custom versions
(for the few packages for which we do support custom versions, like the
kernel, some bootloaders...), we want to ensure that our defconfig
files, when they enable one or more such custom version, do enable
checking the hashes for those versions, and thus we want to require all
our defconfigs do enable BR2_DOWNLOAD_FORCE_CHECK_HASHES.

Add a check for that condition.

We need to be careful that we only check Buildroot's defconfig, whether
in-tree or in a br2-external, and not kernel or other kconfig-based
defconfig files, like those in board/ sub-directories. So we only match
defconfig files that are in a configs/ directory, whether at the
toplevel (for in-tree defconfigs), or not (for br2-external defconfigs).

Since we only have two defconfigs that check hashes for custom versions,
regnerate .checkpackageignore to ignore all so-far broken defconfigs.

Suggested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@datacom.com.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-05-01 21:41:27 +02:00
Yann E. MORIN
81bb14a935 utils/check-package: don't ignore check-package
Commit 32934b526b (utils/checkpackagelib: check for Upstream trailers)
introduced a new python module to check Upstream tags in patch files. In
doing so, it introduced a flake8 coding style issue. That was not caught
when applying the change, and neither was it caught by our daily checks,
because the .checkpackagefile was regenerated right just in the next
commit, to apply ignore patterns to existing patch files.

It is a bit sad that one of our checks does not itself passes all our
checks...

Fix that trivial issue now.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-03-31 22:21:52 +02:00
Yann E. MORIN
fcf465d04b utils/checkpackagelib: add check for CPE variables set to default values
Now that we can specify that the default values for the CPE_ID variables
are valid, without having to actually set one (or more) to their
default, add a check-package check that validates that the CPE_ID
variables are indeed not set to their default.

It also validates that CPE_ID_VALID is not set when another CPE_ID
variable is set to a non-default value.

Add an anchor in the manual so that we can easily point to it.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 16:30:59 +01:00
Yann E. MORIN
5eac4f81eb utils/check-package: check ignored files exist
When an ignored file is removed (e.g. a package patch is no longer
needed after a version bump), the corresponding entry in the ignore list
is no longer needed.

However, we currently only validate that an ignored *test* still fails,
not that a ignore files is now missing.

Add a new test to check-package that does that check, and add a
test-case for that check.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
2023-05-13 12:08:25 +02:00
Vincent Fazio
32934b526b utils/checkpackagelib: check for Upstream trailers
Implement a check-package check for an Upstream: trailer in patches
being applied to packages per a mailing list discussion [0].

No strict formatting checks are implemented for the contents within the
trailer as the needed level of detail will vary patch-to-patch.

Tested with: `./utils/docker-run python3 -m pytest utils/checkpackagelib`

[0] https://lists.buildroot.org/pipermail/buildroot/2023-March/666016.html

Signed-off-by: Vincent Fazio <vfazio@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-04-15 19:36:50 +02:00
Ricardo Martincoski
29a0dd4a30 utils/checkpackagelib: warn about $(HOST_DIR)/usr
It's been ages (5 years at the next release) that we've not installed
host packages in $(HOST_DIR)/usr, but we still have a few packages that
reference it or install things in there. See [1]

Add a new check_function that warns when a file is added installing to
or referencing $(HOST_DIR)/usr .

[1] "d9ff62c4cd pacakge: drop remnants of $(HOST_DIR)/usr"

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: exclude skeleton.mk with disable comment instead of explicit
         code]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-04-09 22:56:16 +02:00
Ricardo Martincoski
9df6503ed0 Makefile: merge check-flake8 into check-package
Teach check-package to detect python files by type and check them using
flake8.
Do not use subprocess to call 'python3 -m flake8' in order to avoid too
many spawned shells, which in its turn would slow down the check for
multiple files. (make check-package takes twice the time using a shell
for each flake8 call, when compared of importing the main application)

Expand the runtime test and the unit tests for check-package.

Remove check-flake8 from the makefile and also from the GitLab CI
because the exact same checks become part of check-package.

Suggested-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: add a comment to x-python to explain its purpose]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-04-09 22:33:24 +02:00
Ricardo Martincoski
29d6f319a0 utils/check-package: check files in board/
When a SysV init script is inside package/ it doesn't need to be
executable. However, when an init script is inside a fs_overlay, it
*does* need to be executable. Therefore, skip the NotExecutable test for
init scripts. We detect them based on the directory /etc/init.d

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: update .checkpackageignore]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-04-09 21:51:11 +02:00
Ricardo Martincoski
c5a3dfb9a4 utils/check-package: check all shell scripts
Currently only SysV init scripts are checked using shellcheck and a few
other rules (e.g. variable naming, file naming).

Extend the check using shellcheck to all shell scripts in the tree.
This is actually limited to the list of directories that check-package
knows that can check, but that list can be expanded later.

In order to apply the check to all shell scripts, use python3-magic to
determine the file type. Unfortunately, there are two different python
modules called "magic". Support both by detecting which one is installed
and defining get_filetype accordingly.

Keep testing first for name pattern, and only in the case there is no
match, check the file type. This ensures, for instance, that SysV
init scripts follow specific rules.

Apply these checks for shell scripts:
 - shellcheck;
 - trailing space;
 - consecutive empty lines;
 - empty last line on file;
 - newline at end of file.

Update the list of ignored warnings.

Do not add unit tests since no function was added, they were just
reused.
But expand the runtime test for check-package using as fixture a file
that generates a shellcheck warning.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: support both variants of the "magic" module]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-02-08 13:31:06 +01:00
Ricardo Martincoski
29c9b44355 utils/checkpackagelib: warn about ifdef on .mk
There are two legitimate cases to prefer ifdef over ifeq in package
recipes: command-line overrides are allowed for busybox and uclibc
configs.

Except for that, all package in tree already use ifeq, so warn the
developer adding/changing a package to use ifeq instead of ifdef, in
order to keep consistence across packages.
file.mk:2: use ifeq ($(SYMBOL),y) instead of ifdef SYMBOL
file.mk:5: use ifneq ($(SYMBOL),y) instead of ifndef SYMBOL

The difference between ifeq and ifdef is that ifdef doesn't expand
recursively.

Add comments to busybox and uclibc packages to avoid a warning in such
special cases.

Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-06 14:22:38 +01:00
Ricardo Martincoski
4bbda72743 utils/checkpackagelib: warn about redefined config
Warn the developer in the case the same config is declared more than
once in the same Config.in file.
But take into account the conditional code that lets the config be
visible and warn only when it is declared more than once in the same
conditions.
For instance, do not warn for:
 if BR2_PACKAGE_BUSYBOX
 config BR2_PACKAGE_BUSYBOX_SHOW_OTHERS
 endif
 if !BR2_PACKAGE_BUSYBOX # kconfig doesn't support else
 config BR2_PACKAGE_BUSYBOX_SHOW_OTHERS
 endif

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-06 11:26:41 +01:00
Fabrice Fontaine
4dc48e8b9d utils/checkpackagelib/lib_mk.py: check CONF_OPTS
As already done for {FOO}_DEPENDENCIES in commit
4910a175b3, check that {FOO}_CONF_OPTS are
never overridden in a conditional

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-03-07 23:15:46 +01:00
Arnout Vandecappelle (Essensium/Mind)
b42e87772a utils/checkpackagelib/test_tool.py: fix expectation
While committing the shellcheck feature, it was changed to output the
full shellcheck output even at verbosity level 1. However, the
expectation of the shellcheck test was not updated accordingly.

Do that now, simply merging all the shellcheck output in a single
string.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-02-06 19:31:44 +01:00
Ricardo Martincoski
d409fe0856 utils/checkpackagelib/lib_sysv: run shellcheck
For simplicity, when shellcheck returns one or more warnings, count it
as a single check-package warning.
The developer can get the full output either by running shellcheck
directly or by calling check-package with -v.

Examples:

|$ ./utils/docker-run utils/check-package --include Shellcheck package/polkit/S50polkit
|package/polkit/S50polkit:0: run 'shellcheck' and fix the warnings
|51 lines processed
|1 warnings generated

|$ ./utils/docker-run utils/check-package --include Shellcheck -v package/polkit/S50polkit
|package/polkit/S50polkit:0: run 'shellcheck' and fix the warnings
|In package/polkit/S50polkit line 43:
|< tab  >start|stop|restart|reload)
|                           ^----^ SC2221: This pattern always overrides a later one on line 45.
|In package/polkit/S50polkit line 45:
|< tab  >reload)
|        ^----^ SC2222: This pattern never matches because of a previous pattern on line 43.
|For more information:
|  https://www.shellcheck.net/wiki/SC2221 -- This pattern always overrides a l...
|  https://www.shellcheck.net/wiki/SC2222 -- This pattern never matches becaus...
|51 lines processed
|1 warnings generated

NOTICE: shellcheck results depends on the version of the tool.
This is why the examples above run inside the docker image.

Also update .gitlab-ci.yml with the docker image after the change of
the previous commit. We don't actually use shellcheck in CI, but the
image from .gitlab-ci.yml is used by the docker-run script and we use
that to run shellcheck.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: make sure a single -v is enough to get shellcheck output;
 update docker image tag in .gitlab-ci.yml]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-02-06 18:27:03 +01:00
Ricardo Martincoski
d020368eea utils/checkpackagelib/lib_sysv: check SysV init scripts
Enable the common checks:
 - consecutive empty lines
 - empty last line
 - missing new line at end of file
 - trailing space
 - warn for executable files, with the hint to instead use
   '$(INSTALL) -D -m 0755' in the .mk file

Check indent with tabs:
 - add a simple check function to warn only when the indent is done
   using spaces or a mix of tabs and spaces. It does not check indenting
   levels, but it already makes the review easier, since it
   diferentiates spaces and tabs.

Check variables:
 - check DAEMON is defined
 - when DAEMON is defined, check the filename is in the form S01daemon
 - when PIDFILE is defined, expect it to be in /var/run and defined
   using $DAEMON.

Also add unit test for this.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: avoid 'del NotExecutable_base' by importing the module instead
 of the class; refer to manual in warnings]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-02-06 18:27:00 +01:00
Ricardo Martincoski
7947328de4 utils/checkpackagelib: warn about executable files
Currently there are no .mk, Config.in, .patch or .hash files with
executable permissions in the tree.
But we don't want to have that.

So warn when a file checked by check-package has executable permission.

This check will be reused when testing SysV init scripts in the tree.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: use context manager for temp dir so it gets deleted]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-02-06 18:26:20 +01:00
Ricardo Martincoski
1734127e59 utils/check-package: prepare to run external tools
Some file formats have well-established syntax checkers.
One example of this is the tool 'shellcheck' that can analyse shell
scripts for common mistakes.

There is no reason to reimplement such tools in check-package, when we
can just call them.

Add the ability to check-package to call external tools that will run
once for each file to be analysed.
For simplicity, when the tool generated one or more warnings, count it
as a single warning from check-package, that can display something like
this:

|$ ./utils/check-package  package/unscd/S46unscd
|package/unscd/S46unscd:0: run 'shellcheck' and fix the warnings
|25 lines processed
|1 warnings generated

|$ ./utils/check-package -vvvvvvvvvvvvvvvv package/unscd/S46unscd
|package/unscd/S46unscd:0: run 'shellcheck' and fix the warnings
|In package/unscd/S46unscd line 9:
|        printf "Starting ${NAME}: "
|               ^------------------^ SC2059: Don't use variables in the printf format string. Use printf "..%s.." "$foo".
|In package/unscd/S46unscd line 11:
|        [ $? -eq 0 ] && echo "OK" || echo "FAIL"
|          ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.
|In package/unscd/S46unscd line 14:
|        printf "Stopping ${NAME}: "
|               ^------------------^ SC2059: Don't use variables in the printf format string. Use printf "..%s.." "$foo".
|In package/unscd/S46unscd line 16:
|        [ $? -eq 0 ] && echo "OK" || echo "FAIL"
|          ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.
|For more information:
|  https://www.shellcheck.net/wiki/SC2059 -- Don't use variables in the printf...
|  https://www.shellcheck.net/wiki/SC2181 -- Check exit code directly with e.g...
|25 lines processed
|1 warnings generated

In this first commit, add only the ability for check-package to call
external tools and not an example of such tool, as adding each tool to
call may need update to the docker image and can lead to it's own
discussion on how to implement.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-02-06 16:38:52 +01:00
Ricardo Martincoski
fc254881e6 utils/checkpackagelib: add unit tests
So anyone willing to contribute to check-package can run all tests in
less than 1 second by using:
$ python3 -m pytest -v utils/checkpackagelib/

Most test cases are in the form:
@pytest.mark.parametrize('testname,filename,string,expected', function)
 - testname: a short description of the scenario tested, added in order
   to improve readability of the log when some tests fail
 - filename: the filename the check-package function being tested thinks
   it is testing
 - string: the content of the file being sent to the function under test
 - expected: all expected warnings that a given function from
   check-package should generate for a given file named filename and
   with string as its content.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-02-06 15:35:19 +01:00
Yann E. MORIN
5da8218184 support/download: drop support for the 'none' hash
Commit 1ba85b7f87 (support/download: add explicit no-hash support)
introduced the 'none' hash type, in an attempt to make hash files
mandatory, but not failing on archives localy generated, like those
for git or svn repositories, especially for those packages where a
version choice was present, which would allow for either remote
archives for which we'd have a hash or VCS trees for which we could
not have a hash for the localy generated archive.

Indeed, back in the time, we did not have a mean to generate
reproducible archives, so having a hash file without a hash for
thosel ocally generated archives would trigger an error in the
hash-checking machinery.

But now, low-and-behold, we do know how to generate those archives,
and we have a mechanism to explicitly exclude some archives from being
hash-checked (e.g. when the version string itself can be user-provided).

As such, the 'none' hash type no longer has any raison d'être, we do not
use it in-tree, and its use in a br2-external tree is most probably
inexistent (as is the use of hash files alotgether most probably).

So we simply drop the support for that.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
[Thomas: drop support in checkpackagelib, as reported by Ricardo.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-01-11 21:58:01 +01:00
Heiko Thiery
f35a4b4ae2 utils/check-package: add a check for the new spacing convention
The seperation of the fields in the hash file should be 2 spaces for
consistency.

Since a large number of hash files still violate this rule, exclude it
from "make check-package" (and thus from CI).

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Arnout:
 - Move it to a separate class, so it can be excluded.
 - Exclude it from "make check-package"
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-01-09 23:01:53 +01:00
Ricardo Martincoski
5bbedea9c2 utils/checkpackagelib/lib_mk.py: fix check for overridden variable
Currently this .mk snippet results in unexpected behavior from
check-package:
|VAR_1 = VALUE1
|ifeq (condition)
|VAR_1 := $(VAR_1), VALUE2
|endif

Fix commit "163f160a8e utils/{check-package, checkpackagelib}:
consistently use raw strings for re.compile" that ended up doing this:
-    CONCATENATING = re.compile("^([A-Z0-9_]+)\s*(\+|:|)=\s*\$\(\\1\)")
+    CONCATENATING = re.compile(r"^([A-Z0-9_]+)\s*(\+|:|)=\s*\$\(\\1\)")

But raw strings do not expect escaping when referencing \1 and the
pattern ends up searching for a raw '\\1' instead of an occurrence of
the first pattern inside parenthesis.

|$ python3
|Python 3.8.10 (default, Sep 28 2021, 16:10:42)
|[GCC 9.3.0] on linux
|Type "help", "copyright", "credits" or "license" for more information.
|>>> import re
|>>> p1 = re.compile('(foo)bar\\1')
|>>> p2 = re.compile(r'(foo)bar\\1')
|>>> p3 = re.compile(r'(foo)bar\1')
|>>> s1 = 'foobarfoo'
|>>> s2 = 'foobar\\1'
|>>> print(p1.search(s1))
|<re.Match object; span=(0, 9), match='foobarfoo'>
|>>> print(p2.search(s1))
|None
|>>> print(p3.search(s1))
|<re.Match object; span=(0, 9), match='foobarfoo'>
|>>> print(p1.search(s2))
|None
|>>> print(p2.search(s2))
|<re.Match object; span=(0, 8), match='foobar\\1'>
|>>> print(p3.search(s2))
|None
|>>>

So use '\1' instead of '\\1' in the raw string.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-12-10 20:01:40 +01:00
Fabrice Fontaine
4910a175b3 utils/checkpackagelib/lib_mk.py: check DEPENDENCIES
Check that {FOO}_DEPENDENCIES are never overriden in a conditional

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-12-10 19:26:30 +01:00
Fabrice Fontaine
fba568a16e utils/checkpackagelib/lib_mk.py: check typo in define
Check typo in define to detect SMAKE_LINUX_CONFIG_FIXUPS in smack
(fixed by 41e2132fbe)

The new expression will catch "SMAKE_CONF_OPTS" as well as
"define SMAKE_LINUX_CONFIG_FIXUPS"

Two modifications were made:
 - add (define\s+)? which will match "define " but also an empty value.
   Thanks to this, the second group will always contain the variable
   name.
 - remove \s*(\+|)= which seems superfluous

Also, add GCC_TARGET in ALLOWED variable to avoid the following
warnings:

arch/arch.mk:12: possible typo: GCC_TARGET_ARCH -> *ARCH*
arch/arch.mk:13: possible typo: GCC_TARGET_ABI -> *ARCH*
arch/arch.mk:14: possible typo: GCC_TARGET_NAN -> *ARCH*
arch/arch.mk:15: possible typo: GCC_TARGET_FP32_MODE -> *ARCH*
arch/arch.mk:16: possible typo: GCC_TARGET_CPU -> *ARCH*
arch/arch.mk:17: possible typo: GCC_TARGET_FPU -> *ARCH*
arch/arch.mk:18: possible typo: GCC_TARGET_FLOAT_ABI -> *ARCH*
arch/arch.mk:19: possible typo: GCC_TARGET_MODE -> *ARCH*

Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-12-06 22:12:34 +01:00
Thomas De Schampheleire
a1bb132a81 utils/checkpackagelib/lib_mk.py: handle 'else' and 'elif' statements
An 'else' or 'elif' clause inside a make conditional should not be indented
in the same way as the if/endif clause. check-package did not recognize the
else statement and expected an indentation.

For example:

ifdef FOOBAR
	interesting
else
	more interesting
endif

would, according to check-package, need to become:

ifdef FOOBAR
	interesting
	else
	more interesting
endif

Treat 'else' and 'elif' the same as if-like keywords in the Indent test, but
take into account that 'else' is also valid shell, so we need to correctly
handle line continuation to prevent complaining about the 'else' in:

ifdef FOOBAR
	if true; \
	    ... \
	else \
	    ... \
	fi
endif

We don't add the 'else' and 'elif' statements to start_conditional, because
it would cause incorrect nesting counting in class OverriddenVariable.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-02 13:54:59 +01:00
Thomas Petazzoni
163f160a8e utils/{check-package, checkpackagelib}: consistently use raw strings for re.compile
Raw strings need to be used when calling re.compile() otherwise Python
3.x flake8 complains with:

W605 invalid escape sequence '\s'

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-08-14 21:55:59 +02:00
Yann E. MORIN
76772cefd8 utils/check-package: ignore ACLOCAL_PATH
The aclocal program is provided by the automake package, so it makes
sense to define aclocal-related variables in automake.mk.

Add an exception to check-package to ignore that variable.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-02-04 17:15:19 +01:00
Yann E. MORIN
1e03a2e89e utils/check-package: report := that appends to variables
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-27 10:35:06 +01:00
Jerzy Grzegorek
31fc6a1fe4 utils/checkpackagelib: CommentsMenusPackagesOrder: add more Config.in files to check
Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: remove boot/Config.in, it is not ordered correctly yet.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-07 23:39:21 +02:00
Jerzy Grzegorek
16271ec356 utils/checkpackagelib: CommentsMenusPackagesOrder: initialize 'menu_of_packages' array
'source' without a previous 'menu' is common in package/Config.in in
br2-externals.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-07 23:39:21 +02:00
Jerzy Grzegorek
b47590fdb0 utils/checkpackagelib: CommentsMenusPackagesOrder: initialize in before()
This makes sure the state from a previous run (previous file) can never
leak over into the next file.

Also order the initializations alphabetically.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-07 23:39:21 +02:00
Jerzy Grzegorek
6a5f6ea7e2 utils/checkpackagelib: CommentsMenusPackagesOrder: use '-' to describe state
Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-07 23:39:21 +02:00
Jerzy Grzegorek
0625899e3b utils/checkpackagelib: CommentsMenusPackagesOrder: separate comment/if/menu cases
The handling of 'comment...', 'if ...' and 'menu ...' lines have almost
nothing in common, and subsequent patches will give them even less in
common. Therefore, completely separate their handling in top-level
conditions. The only code that gets duplicated in the different branches
is the 'self.initialize_level_elements(text)' call.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-07 23:39:21 +02:00
Jerzy Grzegorek
dd251d68e5 utils/checkpackagelib: CommentsMenusPackagesOrder: remove '-comment' state before the '-menu' one
A comment is considered an alternative delimiter like a menu. I.e.,
a menu that comes after a comment should not be considered a submenu of
that comment. Therefore, remove the '-comment' state before adding the
'-menu' one.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-07 23:39:21 +02:00
Jerzy Grzegorek
c82e0092f5 utils/checkpackagelib: CommentsMenusPackagesOrder: change the type of variable 'new_package'
Change the type of variable 'new_package' to make it a class member.
It will be used not only locally. Also initialize it.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-25 22:24:42 +02:00
Jerzy Grzegorek
7197d3cb80 utils/checkpackagelib: CommentsMenusPackagesOrder: add functions to initialize arrays elements
Factor out two functions to initialize arrays elements. They will be
reused by followup patches.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-25 22:24:42 +02:00
Jerzy Grzegorek
4e88b37274 utils/checkpackagelib: CommentsMenusPackagesOrder: get value of variable 'level'
Get value of variable 'level' only just after the state change.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-25 22:24:42 +02:00
Jerzy Grzegorek
6748e42ace utils/checkpackagelib: CommentsMenusPackagesOrder: change the type of variable 'level'
Change the type of variable "level" to make it a class member.
It will be used not only locally.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: initialize self.level in the before() method, as suggested by
Ricardo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-25 22:24:42 +02:00
Jerzy Grzegorek
dd99dc5763 utils/checkpackagelib: CommentsMenusPackagesOrder: rename variable 'm'
Rename variable 'm' for better readability.

Signed-off-by: Jerzy Grzegorek <jerzy.m.grzegorek@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-25 22:24:39 +02:00
Ricardo Martincoski
1842bb1470 check-package: fix check of file in current dir with -b
One of the possible usages of check-package is to first cd to the
directory that contains the files to test (e.g. a package directory) and
then call the script passing the files in the current dir.
It already works when used for intree files, but for files in a
br2-external it throws an exception because some check functions (from
utils/checkpackagelib/lib_*.py) do need the name of the file being
processed and assume there will be a slash before the name.

Fix all check functions that assume that the full filename being checked
contains a slash. Do not use regexps to extract the filename, use
os.path functions instead.

Notice RemoveDefaultPackageSourceVariable and TypoInPackageVariable lead
to an exception in this case, but ApplyOrder instead generates a false
warning.

Fixes bug #11271.

Reported-by: Vitaliy Lotorev <lotorev@gmail.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vitaliy Lotorev <lotorev@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-03 17:17:29 +02:00
Arnout Vandecappelle (Essensium/Mind)
d5990601ba utils/checkpackagelib: CommentsMenusPackagesOrder: properly initialize levels
Fix an issue introduced by Arnout while committing. Jerzy originally
initialized the menu_of_packages, package and print_package_warning
members like they should be, but Arnout thought it wasn't needed and
removed that.

It is actually needed, to make sure the top level (level 0) works.

Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/264383157

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-02 18:43:52 +02:00