The Python module gets built during the 'make install' command even if
'make all' was run prior for build.
This is the same issue as described in the previous commit
"pylibfdt/meson.build: fix Python library being rebuilt during install".
Remove the '--build-lib' flag so setuptools can find the build module.
Signed-off-by: Brandon Maier <brandon.maier@gmail.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
User @sharkcz noted that when the '--quiet' flag is removed from
./setup.py, the following can be seen from the `./setup.py install`
stage.
Running custom install script 'dtc/g/pylibfdt/../setup.py --top-builddir \
dtc/g/redhat-linux-build install --prefix=/usr/local --root=$DESTDIR'
running install
...
building '_libfdt' extension
swigging dtc/g/pylibfdt/../pylibfdt/libfdt.i to \
dtc/g/pylibfdt/../pylibfdt/libfdt_wrap.c
swig -python -Idtc/g/pylibfdt/../libfdt -o \
dtc/g/pylibfdt/../pylibfdt/libfdt_wrap.c dtc/g/pylibfdt/../pylibfdt/libfdt.i
gcc -fno-strict-overflow -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 \
-DNDEBUG -fexceptions -fexceptions -fexceptions -fPIC -DPY_SSIZE_T_CLEAN \
-Idtc/g/pylibfdt/../libfdt -I/usr/include/python3.12 -c \
dtc/g/pylibfdt/../pylibfdt/libfdt_wrap.c -o \
build/temp.linux-ppc64le-cpython-312dtc/g/pylibfdt/../pylibfdt/libfdt_wrap.o
creating build/lib.linux-ppc64le-cpython-312
gcc -shared build/temp.linux-ppc64le-cpython-312dtc/g/pylibfdt/../pylibfdt/libfdt_wrap.o \
-Ldtc/g/redhat-linux-build/libfdt -L/usr/lib64 -lfdt -o \
build/lib.linux-ppc64le-cpython-312/_libfdt.cpython-312-powerpc64le-linux-gnu.so
copying dtc/g/pylibfdt/../pylibfdt/libfdt.py -> build/lib.linux-ppc64le-cpython-312
Meaning the python library is getting recompiled during the `meson
install` phase. This causes build issues as Meson does not set the
compiler and linker flags during the install phase.
The reason the library is getting rebuilt is during the normal build
with "build_ext", the `--build-lib` flag gets passed which changes the
default output build directory. But there is no equivalent option for
the "install" command. Install instead looks in the default directory
"./build" and so does not find the previously built library.
Since we can't fix the "install" command, drop the --build-lib flag.
This causes setup.py to compile the libraries at
`<meson-build>/build/lib.linux-x86_64-cpython-312/`. We must also then
fix run_tests.sh to find the library build directory as it's
machine-dependent.
Fixes: #135
Signed-off-by: Brandon Maier <brandon.maier@gmail.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
If the tests are run without a full compile they will fail. For example
with the following.
> rm -rf build/
> meson setup build/
> meson test -C build/
This is because the tests rely on the devicetree tools and test
executables.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The python library requires libfdt to build. This would intermittently
fail depending on what order targets compiled.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The test for get_mem_rsv fails on newer versions of Python with the
following error.
> Traceback (most recent call last):
> File "/__w/dtc/dtc/tests/pylibfdt_tests.py", line 421, in testReserveMap
> self.assertEqual([ 0xdeadbeef00000000, 0x100000],
> AssertionError: Lists differ: [16045690981097406464, 1048576] != [0, 16045690981097406464, 1048576]
>
> First differing element 0:
> 16045690981097406464
> 0
>
> Second list contains 1 additional elements.
> First extra element 2:
> 1048576
>
> - [16045690981097406464, 1048576]
> + [0, 16045690981097406464, 1048576]
> ? +++
It appears this is because the PyTuple_GET_SIZE() function that was used
to build the fdt_get_mem_rsv() return value has changed. It now is
returning a non-zero value when it's passed an integer, which causes the
SWIG wrapper to append the returned arguments to the return error rather
then ignore them.
This is valid behaviour per Python's documentation, which says it will
"Return the size of the tuple p, which must be non-NULL and point to a
tuple; no error checking is performed"[1]. As passing an integer is not
a tuple, its return value is undefined.
Fix this issue on older and newer versions by avoiding
PyTuple_GET_SIZE() entirely. Always append the arguments to the list,
and instead use the wrapper python function to check the first argument
and then splice the last two arguments as the return value.
[1] https://docs.python.org/3/c-api/tuple.html#c.PyTuple_GET_SIZE
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Boolean properties are unusual in that their presense or absence
indicates the value of the property. This makes them a little painful to
support using the existing getprop() support.
Add new methods to deal with booleans specifically.
Signed-off-by: Simon Glass <sjg@chromium.org>
Message-ID: <20230912182716.248253-1-sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Fixes the following warning
> pylibfdt/meson.build:2: WARNING: Project targets '>=0.56.0' but uses
feature deprecated since '0.55.0': ExternalProgram.path. use
ExternalProgram.full_path() instead
Do not use full_path() as suggested. setup_py is being called as a
command by custom_target() which understands how to properly inherit the
object returned by find_program().
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This also enables us to test the -NOSPACE condition by adding a test
setting size_hint=1 so this path is taken.
Message-Id: <20230201181112.1644842-1-luca@z3ntu.xyz>
Signed-off-by: Luca Weiss <luca@z3ntu.xyz>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add a new Python method wrapping fdt_get_path() from the C API.
Also add a test for the new method.
Signed-off-by: Luca Weiss <luca@z3ntu.xyz>
Message-Id: <20220419194537.63170-1-luca@z3ntu.xyz>
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add new methods to handle decoding of int32, uint32, int64 and uint64
arrays.
Also add tests for the new methods.
Signed-off-by: Luca Weiss <luca@z3ntu.xyz>
Message-Id: <20211225132558.167123-3-luca@z3ntu.xyz>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add a new method for decoding a string list property, useful for e.g.
the "reg-names" property.
Also add a test for the new method.
Signed-off-by: Luca Weiss <luca@z3ntu.xyz>
Message-Id: <20211225132558.167123-2-luca@z3ntu.xyz>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Fatal Python error: none_dealloc: deallocating None
Python runtime state: finalizing (tstate=0x000055c9bac70920)
Current thread 0x00007fbe34e47740 (most recent call first):
<no Python frame>
Aborted (core dumped)
This is caused by a missing Py_INCREF on the returned Py_None, as
demonstrated e.g. in https://github.com/mythosil/swig-python-incref or
described at https://edcjones.tripod.com/refcount.html ("Remember to
INCREF Py_None!")
A PoC for triggering this crash is uploaded to
https://github.com/z3ntu/pylibfdt-crash .
With this patch applied to pylibfdt the crash does not happen.
Signed-off-by: Luca Weiss <luca@z3ntu.xyz>
Message-Id: <20211224102811.70695-1-luca@z3ntu.xyz>
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Since Python 2.5 the argument parsing functions when parsing expressions
such as s# (string plus length) expect the length to be an int or a
ssize_t, depending on whether PY_SSIZE_T_CLEAN is defined or not.
Python 3.8 deprecated the use of int, and with Python 3.10 this symbol
must be defined and ssize_t used[1].
Define the magic symbol when building the extension, and cast the ints
from the libfdt API to ssize_t as appropriate.
[1] https://docs.python.org/3.10/whatsnew/3.10.html#id2
Signed-off-by: Ross Burton <ross.burton@arm.com>
Message-Id: <20211111160536.2516573-1-ross.burton@arm.com>
[dwg: Adjust for new location of setup.py]
Tested-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Using 'pip' and several setup.py sub-commands currently don't work with
pylibfdt. The primary reason is Python packaging has opinions on the
directory structure of repositories and one of those appears to be the
inability to reference source files outside of setup.py's subtree. This
means a sdist cannot be created with all necessary source components
(i.e. libfdt headers). Moving setup.py to the top-level solves these
problems.
With this change. the following commands now work:
Creating packages for pypi.org:
./setup.py sdist bdist_wheel
Using pip for installs:
pip install .
pip install git+http://github.com/robherring/dtc.git@pypi-v2
Signed-off-by: Rob Herring <robh@kernel.org>
Message-Id: <20211111011135.2386773-5-robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The 'author' field in setup.py is supposed to be just the name. The
email address goes in 'author_email' field.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Rob Herring <robh@kernel.org>
Message-Id: <20211111011135.2386773-4-robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The DTC version in version_gen.h causes a warning with setuptools:
setuptools/dist.py:501: UserWarning: The version specified ('1.6.1-g5454474d') \
is an invalid version, this may not work as expected with newer versions of \
setuptools, pip, and PyPI. Please see PEP 440 for more details.
It also creates an unnecessary dependency on the rest of the build
system(s). Switch to use setuptools_scm instead to get the version for
pylibfdt.
Signed-off-by: Rob Herring <robh@kernel.org>
Message-Id: <20211111011135.2386773-3-robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The use of setuptools is favored over distutils. setuptools is needed to
support building Python 'wheels' and for pip support.
Signed-off-by: Rob Herring <robh@kernel.org>
Message-Id: <20211111011135.2386773-2-robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Clang has -Wself-assign enabled by default under -Wall and so when
building with -Werror we would get an error here. Inspired by Linux
kernel git commit a21151b9d81a ("tools/build: tweak unused value
workaround") make use of the fact that both Clang and GCC support
casting to `void` as the method to note that something is intentionally
unused.
Signed-off-by: Tom Rini <trini@konsulko.com>
Message-Id: <20210524154910.30523-1-trini@konsulko.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The meson build system allows projects to "vendor" dtc easily, thanks to
subproject(). QEMU has recently switched to meson, and adding meson
support to dtc will help to handle the QEMU submodule.
meson rules are arguably simpler to write and maintain than
the hand-crafted/custom Makefile. meson support various backends, and
default build options (including coverage, sanitizer, debug/release
etc, see: https://mesonbuild.com/Builtin-options.html)
Compare to the Makefiles, the same build targets should be built and
installed and the same tests should be run ("meson test" can be provided
extra test arguments for running the equivalent of checkm/checkv).
There is no support EXTRAVERSION/LOCAL_VERSION/CONFIG_LOCALVERSION,
instead the version is simply set with project(), and vcs_tag() is
used for git/dirty version reporting (This is most common and is
hopefully enough. If necessary, configure-time options could be added
for extra versioning.).
libfdt shared library is build following regular naming conventions:
instead of libfdt.so.1 -> libfdt-1.6.0.so (with current build-sys),
libfdt.so.1 -> libfdt.so.1.6.0. I am not sure why the current build
system use an uncommon naming pattern. I also included a libfdt.pc
pkg-config file, as convenience.
Both Linux native build and mingw cross-build pass. CI pass. Tests are
only run on native build.
The current Makefiles are left in-tree, and make/check still work.
Eventually, the Makefiles could be marked as deprecated, to start a
transition period and avoid having to maintain 2 build systems in the
near future.
(run_tests.sh could eventually be replaced by the meson test runner,
which would have several advantages in term of flexibility/features,
but this is left for another day)
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20201012073405.1682782-3-marcandre.lureau@redhat.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
With meson, we have to support out-of-tree build.
Introduce a --top-builddir option, which will default to the current
directory to lookup generated filed such as version_gen.h and output
directories.
Other source paths are derived from the location of the setup.py script
in the source tree.
--build-lib is changed to be relative to the current directory, instead
of relative to setup.py. This has less surprising results!
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20201012073405.1682782-2-marcandre.lureau@redhat.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This function should use a void * type, not char *. This causes an error:
TypeError: in method 'fdt_property_stub', argument 3 of type 'char const *'
Fix it and update the tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
Message-Id: <20191025010226.34378-1-sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The default Python version for pylibfdt is already Python 3 but if
called without specifiying an interpreter, the setup.py script gets
called with Python 2.
It's of course still possible to call setup.py with python2 directly.
Signed-off-by: Luca Weiss <luca@z3ntu.xyz>
Message-Id: <20190907152530.25102-1-luca@z3ntu.xyz>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add this into the class to simplify use of this function.
Signed-off-by: Appana Durga Kedareswara rao <appana.durga.rao@xilinx.com>
Message-Id: <1562130487-27028-1-git-send-email-appana.durga.rao@xilinx.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present this example is incorrect since it is missing the call to
finish_reservemap() and does not add a root node. Fix these problems.
Signed-off-by: Simon Glass <sjg@chromium.org>
Message-Id: <20190703000815.102459-1-sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Python2 is deprecated upstream, lets try to move forwards. Along with it
generalize the .gitignore file so we ignore the .pyc files in the new
location that Python3 uses.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Petr Viktorin <pviktori@redhat.com>
Message-Id: <20190218164856.23861-3-frenzy@frenzy.cz>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
A list passed as an argument to check_err() means that
there is no error code to check and therefore it should
be returned back.
Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Message-Id: <20190218164856.23861-2-frenzy@frenzy.cz>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The dtc makefiles have support for building into a separate directory from
the sources... except that it's broken and probably always has been.
Remove the pretense.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For no particularly good reason, the install target for the Python library
uses a different PREFIX variable to give the installation destination
to the rest of dtc & libfdt. Make it use the same one.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Move it to the subdir Makefile, generalize some of the patterns, remove
the 'build' directory made by setup.py and __pycache__ directory made by
Python3.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Python 2 is still the default but it can be changed by
setting environment variable PYTHON before build/test.
Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently setup.py depends on being invoked from the right directory
(specifically it needs to be run from the root of the project). That's a
bit confusing.
This updates setup.py to no longer depend on the invoking directory by
instead having it change directory to the location of the script itself,
then using internal paths relative to that.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
This function no longer does anything useful, so get rid of it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
Currently setup.py expects the library version in a VERSION environment
variable, or it exctracts the version from the Makefile. The latter is
for the case where the script is run standalone, rather than from make.
But parsing the Makefile is ugly and fragile, and won't always get the
same version we put into the C code.
This changes to instead extracting the version from the trivial .h file we
already generate to put the version into C code. It's still slightly ugly,
but it's simpler and since we can control the precise format of that .h,
not as fragile.
This lets us remove the remains of the makefile parsing code from setup.py.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
At the moment we unconditionally pass --quiet to setup.py. Change that to
get more debugging output from it when V=1 is passed to make.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
This points to the Python setup script, since we reference it in a couple
of places. While we're there correct two small problems:
1) setup.py is part of the checked in sources and so lives in
$(PYLIBFDT_srcdir) not $(PYLIBFDT_objdir) [this only worked because
those are the same by default]
2) The module itself should depend on the setup script so it is rebuilt
if the script is changed
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
At the moment we have some fiddly code to either pass in make's CPPFLAGS to
setup.py, or have setup.py extract them from the Makefile. But really the
only thing we need from here is the include paths. We already know what
include paths we need (libfdt/) so we can just set that directly in
setup.py.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
Currently we build the Python extension module from all the libfdt source
files as well as the swig wrapper file. This is a bit silly, since we've
already compiled libfdt itself.
This changes the build to instead build the extension module from just the
swig wrapper, linking it against the libfdt.a we've already build.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
Our Makefile currently passes PYLIBFDT_objdir into setup.py in an attempt
to set the correct place to put the Python extension module output. But
that gets passed in the 'package_dir' map in distutils.
But that's basically not what package_dir controls. What actually makes us
find the module in the right place is the --inplace passed to setup.py
(causing the module to go into the current directory), and the following
'mv' in the Makefile to move it into the right final location.
We can simplify setup.py by dropping the useless objdir stuff, and get the
module put in the right place straight way by instead using the --build-lib
setup.py option.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
pylibfdt/setup.py currently adds include flags to the extension module
build to allow include files in the base dtc directory. But pylibfdt
doesn't rely on any headers there, only on headers in libfdt/ - it also
shouldn't rely on dtc headers at any future time.
So, remove that from the include list, allowing some simplifications to
setup.py.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
These methods are needed to permit larger changes to the device tree blob.
Add two new methods and an associate test.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present this method always raised an exception when an error occurs.
Add a 'quiet' argument so it matches the other methods.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Fix typemap for fdt_get_mem_rsv so it returns 64-bit values.
Fixes https://github.com/dgibson/dtc/issues/15.
Signed-off-by: Dan Horák <dan@danny.cz>
[dwg: Adjusted commit message for typo and context]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In libfdt.i we set the handling of uint64_t parameters to use
PyLong_AsUnsignedLong. But for 32-bit platforms, where an unsigned long
is 32-bits, this will truncate the value we need.
It turns out swig's default typemapping for uint64_t correctly handles
conversions both to python ints and python longs, so we don't need this
typemap at all.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
It is useful to be able to create a device tree from scratch using
software. This is supported in libfdt but not currently available in the
Python bindings.
Add a new FdtSw class to handle this, with various methods corresponding
to the libfdt functions. When the tree is complete, calling AsFdt() will
return the completed device-tree object.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Brandon Maier