6dcb8ba4 "libfdt: Add helpers for accessing unaligned words" introduced
the fdt32_ld() and fdt64_ld() helpers for loading values from the FDT blob
which might not be naturally aligned. This matters for ARM, where
attempting a plain unaligned load will often cause an exception.
However, it seems the memcpy() we used here was surprisingly expensive,
making libfdt nearly 6x slower on at least some ARM platforms.
This patch takes an alternative approach, using a bunch of 1-byte loads
and shifts to implement the helpers.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The entire check_msg function is under the if condition except for
va_start/va_end. Move these and invert the if condition saving a level
of indentation.
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Provide the new command-line option:
--annotate (abbreviated -T)
--annotate provides one or more filenames and line numbers indicating
the origin of a given line. The filename is expressed relative the the
filename provided on the command line. Nothing is printed for overlays,
etc.
-T can be repeated giving more verbose annotations. These consist of
one or more tuples of: filename, starting line, starting column, ending
line ending column. The full path is given for the file name.
Overlays, etc are annotated with <no-file>:<no-line>.
The verbose annotations may be too verbose for normal use.
There are numerous changes in srcpos.c to provide the relative filenames
(variables initial_path, initial_pathlen and initial_cpp, new functions
set_initial_path and shorten_to_initial_path, and changes in
srcfile_push and srcpos_set_line). The change in srcpos_set_line takes
care of the case where cpp is used as a preprocessor. In that case the
initial file name is not the one provided on the command line but the
one found at the beginnning of the cpp output.
shorten_to_initial_path only returns a string if it has some shortening
to do. Otherwise it returns NULL and relies on the caller to use the
initial string. This simplifies memory management, by making clear to
the caller whether a new string is allocated.
The new functions srcpos_string_comment, srcpos_string_first, and
srcpos_string_last print the annotations. srcpos_string_comment is
recursive to print a list of source file positions.
Various changes are sprinkled throughout treesource.c to print the
annotations.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Extend the parser to record positions, in build_node,
build_node_delete, and build_property.
srcpos structures are added to the property and node types, and to the
parameter lists of the above functions that construct these types.
Nodes and properties that are created by the compiler rather than from
parsing source code have NULL as the srcpos value.
merge_nodes, defined in livetree.c, uses srcpos_extend to combine
multiple positions, resulting in a list of positions. srcpos_extend
is defined in srcpos.c. New elements are added at the end. This
requires the srcpos type, define in srcpos.h, to be a list structure
with a next field. This next field is initialized to NULL in
srcpos.h, in the macro YYLLOC_DEFAULT invoked implicitly by the
generated parser code.
Another change to srcpos.c is to make srcpos_copy always do a full
copy, including a copy of the file substructure. This is required
because when dtc is used on the output of cpp, the successive detected
file names overwrite the file name in the file structure. The next
field does not need to be deep copied, because it is always NULL when
srcpos_copy is called; an assert checks for this. File names are only
updated in uncopied position structures.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For unclear reasons we had some code to copy a transcript of "make checkm"
runs to a vglog.XXX file. It's not really clear why this was there, and
it had the nasty side effect of discarding errors from run_tests.sh,
meaning that an error on the valgrind run wouldn't show up clearly in
Travis CI builds. Remove that logic so that we see errors more clearly.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The logic in wrap_test() was effectively squashing valgrind errors into
the "FAIL" bucket rather than their own bucket as intended. Correct it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This reverts commit baa1d2cf78.
Turns out this introduced memory badness. valgrind picks it up on
x86, but it straight out SEGVs on x86.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
If the SPI bus controller is being used for 'spi-slave' mode some of the
checks we have need to change:
In 'spi-slave' mode #address-cells should be 0, as any children don't
have a reg property.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Extend the parser to record positions, in build_node, build_node_delete,
and build_property.
srcpos structures are added to the property and node types, and to the
parameter lists of the above functions that construct these types.
Nodes and properties that are created by the compiler rather than from
parsing source code have NULL as the srcpos value.
merge_nodes, defined in livetree.c, uses srcpos_extend to combine
multiple positions, resulting in a list of positions. srcpos_extend is
defined in srcpos.c. New elements are added at the end. The srcpos
type, define in srcpos.h, is now a list structure with a next field.
Another change to srcpos.c is to make srcpos_copy always do a full copy,
including a copy of the file substructure. This is required because
when dtc is used on the output of cpp, the successive detected file
names overwrite the file name in the file structure. The next field
does not need to be deep copied, because it is only updated in newly
copied positions and the positions to which it points have also been
copied. File names are only updated in uncopied position structures.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
According to the device tree specification, the default value for
#size-cells is 1, but fdt_size_cells() was returning 2 if this property
was not present.
This patch also makes fdt_address_cells() and fdt_size_cells() conform
to the behaviour documented in libfdt.h. The defaults are only returned
if fdt_getprop() returns -FDT_ERR_NOTFOUND, otherwise the actual error
is returned.
Signed-off-by: John Clarke <johnc@kirriwa.net>
Use ptrdiff_t modifier (%tx) for printing a difference between 2 pointers. Currently
%zx (size_t) is used, but it fails on platforms where size_t and ptrdiff_t are
defined differently (like s390).
Comes from
f3da2d1b00?branch=master
originally.
Signed-off-by: Dan Horák <dan@danny.cz>
If we have a phandle in the middle of a sequence of numbers and
it is not bracketed (e.g. <0x1234 &phandle 0x5678>), the dts output will
be corrupted due to missing a space between the phandle value and the
following number.
Fixes: 8c59a97ce0 ("Fix missing labels when emitting dts format")
Cc: Grant Likely <grant.likely@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Otherwise the FAIL results won't be accounted for in the summary.
Easily testable by artifically causing them to fail:
- if [ $(($size % $align)) -eq 0 ] ;then
+ if [ $(($size % $align)) -eq 666 ] ;then
Signed-off-by: Lubomir Rintel <lkundrak@v3.sk>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Commit 8c59a97ce0 ("Fix missing labels when emitting dts format")
fixed label output, but broke output when there is a REF_PATH marker.
The problem is a REF_PATH marker causes a zero length string to be
emitted. The write_propval_string() function requires a length of at
least 1 (including the terminating '\0'), but that was not being
checked.
For the integer output, a length of 0 is valid as it is possible to have
labels inside the starting '<':
int-prop = < start: 0x1234>;
REF_PHANDLE is another marker that we don't explicitly handle, but it
doesn't cause a problem as it is fundamentally just an int.
Fixes: 8c59a97ce0 ("Fix missing labels when emitting dts format")
Reported-by: Kumar Gala <kumar.gala@linaro.org>
Cc: Grant Likely <grant.likely@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This commit adds test cases for commits "Correct overlay syntactic
sugar for generating target-path fragments" and "Merge nodes with
local target label references".
It verifies that target path references are not resolved locally and
that target label references that can be resolved locally are.
Signed-off-by: Fredrik Markstrom <fredrik.markstrom@gmail.com>
[dwg: Fixed some whitespace problems]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This change makes sure that nodes with target label references doesn't
create additional fragments if the label can been resolved
locally. Target path references are not resolved locally and will
generate a fragment.
Previously the dts below would generate two fragments:
/dts-v1/;
/plugin/;
&x { a: a@0 {};};
&a { b {}; };
This commit essentially reverts part of the commit "Correct overlay
syntactic sugar for generating target-path fragments". The main reason
we want to do this is that it breaks consumers of dtbo:s that can't
resolve references between fragments in the same dtbo (like the linux
4.1 kernel). In addition creating a fragment for each label reference
substantially increases the size of the resulting dtbo for some use
cases.
Signed-off-by: Fredrik Markstrom <fredrik.markstrom@gmail.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently setup.py depends on being invoked from the right directory
(specifically it needs to be run from the root of the project). That's a
bit confusing.
This updates setup.py to no longer depend on the invoking directory by
instead having it change directory to the location of the script itself,
then using internal paths relative to that.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
This function no longer does anything useful, so get rid of it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
Currently setup.py expects the library version in a VERSION environment
variable, or it exctracts the version from the Makefile. The latter is
for the case where the script is run standalone, rather than from make.
But parsing the Makefile is ugly and fragile, and won't always get the
same version we put into the C code.
This changes to instead extracting the version from the trivial .h file we
already generate to put the version into C code. It's still slightly ugly,
but it's simpler and since we can control the precise format of that .h,
not as fragile.
This lets us remove the remains of the makefile parsing code from setup.py.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
At the moment we unconditionally pass --quiet to setup.py. Change that to
get more debugging output from it when V=1 is passed to make.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
This points to the Python setup script, since we reference it in a couple
of places. While we're there correct two small problems:
1) setup.py is part of the checked in sources and so lives in
$(PYLIBFDT_srcdir) not $(PYLIBFDT_objdir) [this only worked because
those are the same by default]
2) The module itself should depend on the setup script so it is rebuilt
if the script is changed
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
At the moment we have some fiddly code to either pass in make's CPPFLAGS to
setup.py, or have setup.py extract them from the Makefile. But really the
only thing we need from here is the include paths. We already know what
include paths we need (libfdt/) so we can just set that directly in
setup.py.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
Currently we build the Python extension module from all the libfdt source
files as well as the swig wrapper file. This is a bit silly, since we've
already compiled libfdt itself.
This changes the build to instead build the extension module from just the
swig wrapper, linking it against the libfdt.a we've already build.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
Our Makefile currently passes PYLIBFDT_objdir into setup.py in an attempt
to set the correct place to put the Python extension module output. But
that gets passed in the 'package_dir' map in distutils.
But that's basically not what package_dir controls. What actually makes us
find the module in the right place is the --inplace passed to setup.py
(causing the module to go into the current directory), and the following
'mv' in the Makefile to move it into the right final location.
We can simplify setup.py by dropping the useless objdir stuff, and get the
module put in the right place straight way by instead using the --build-lib
setup.py option.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
pylibfdt/setup.py currently adds include flags to the extension module
build to allow include files in the base dtc directory. But pylibfdt
doesn't rely on any headers there, only on headers in libfdt/ - it also
shouldn't rely on dtc headers at any future time.
So, remove that from the include list, allowing some simplifications to
setup.py.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Simon Glass <sjg@chromium.org>
Since commit 7975f64222 ("Fix widespread incorrect use of strneq(),
replace with new strprefixeq()") simple-bus checks have been silently
skipped. The problem was 'end - str' is one more than the string length
and the strnlen in strprefixeq fails. This can't be fixed simply by
subtracting one as it is possible to have multiple '\0' at the end of
the property. Fix this by making the 'compatible' property string list
check a dependency, and then we can assume the property is null
terminated and we can just use streq() for comparisons.
Add some tests so the problem doesn't happen again.
Fixes: 7975f64222 ("Fix widespread incorrect use of strneq(), replace with new strprefixeq()")
Reported-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
When there is a label inside a sequence of ints at the end of a
property, an assertion is hit because write_propval() expects all the
labels to be at the very end of the property data. This is clearly wrong
behaviour.
To reproduce run: "dtc -O dts tests/label01.dts". dtc fails on property
/randomnode/blob.
Fix by reworking the write_propval() loop to remove the separate
iterating over label markers. Instead handle the label markers as part
of the main marker iteration loop. This guarantees that each label
marker is handled at the right location, even if all the data markers
have already been handled, and has the added advantage of making the
code simpler.
However, a side effect of this code is that a label at the very end of
an int sequence will be emitted outside the sequence delimiters. For
example:
Input: intprop = < 1 2 L1: >, L2: < 3 4 L3: > L4:;
Output: intprop = < 1 2 >, L1: L2: < 3 4 > L3: L4:;
The two representations are equivalent in the data model, but the
current test case was looking for the former, but needed to be modified
to look for the later. The alternative would be to render labels before
closing the sequence, but that makes less sense syntactically because
labels between sequences are normally to point at the next one, not the
former. For example:
Input: intprop = < 1 2 L1: >, L2: < 3 4 L3: > L4:;
Output: intprop = < 1 2 L1: L2: >, < 3 4 L3: L4: >;
DTC doesn't current have the information to know if the label should be
inside or outside the sequence, but in common usage, it is more likely
that L1 & L2 refer to the second sequence, not the end of the first.
Fixes: 32b9c61307 ("Preserve datatype markers when emitting dts
format")
Reported-by: Łukasz Dobrowolski <lukasz.dobrowolski@nokia.com>
Signed-off-by: Grant Likely <grant.likely@arm.com>
Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Commit 32b9c61307 ("Preserve datatype markers when emitting dts format")
add spaces between <> and [] and the encapsulated numbers. Fix this to
keep the prior formatting and not break some users needlessly.
Fixes: 32b9c61307 ("Preserve datatype markers when emitting dts format")
Reported-by: Stewart Smith <stewart@linux.ibm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
YAML encoded DT is useful for validation of DTs using binding schemas.
The YAML encoding is an intermediate format used for validation and
is therefore subject to change as needed. The YAML output is dependent
on DTS input with type information preserved.
Signed-off-by: Grant Likely <grant.likely@arm.com>
[robh: make YAML support optional, build fixes, Travis CI test,
preserve type information in paths and phandles]
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Make type_marker_length available to other users of TYPE_* markers.
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
These methods are needed to permit larger changes to the device tree blob.
Add two new methods and an associate test.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present this method always raised an exception when an error occurs.
Add a 'quiet' argument so it matches the other methods.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Some platforms don't have valgrind support, and sometimes you simply might
not want to use valgrind. But at present, dtc, or more specifically its
testsuite, won't compile without valgrind because we use the valgrind
client interface in some places to improve our testing and suppress false
positives.
This adds some Makefile detection to correctly handle the case where
valgrind is not available.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Greg Kurz added a trivial test of the -I fs mode recently, which was
previously basically untested. This is an oversight, since we
recently had a bug which completely broke it.
This replaces Greg's test with a more thorough test of -I fs mode. We
use a test helper to create the familiar test_tree1 in "fs" form, then use
dtc -I fs to process it, and check that the results match what they
should.
We only check the content in -I fs -O dtb mode, since that's simplest,
but we do run -I fs -O dts mode as well to make sure it doesn't blow
up (the aforementioned bug caused just such a blow up, specific to -O
dts mode, for example).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For some upcoming tests we want to be able to test if two trees are
equal, but we don't care about the memory reservation map. So, this
adds an option to the dtbs_equal_unordered test helper which tells it
to ignore the reserve map.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Some recent changes caused '-I fs -O dts' to crash instantly when
emitting the first property holding actual data, ie, coming from
a non-empty file. This got fixed already by another patch.
This simply adds a test for the original problem.
Signed-off-by: Greg Kurz <groug@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Fix typemap for fdt_get_mem_rsv so it returns 64-bit values.
Fixes https://github.com/dgibson/dtc/issues/15.
Signed-off-by: Dan Horák <dan@danny.cz>
[dwg: Adjusted commit message for typo and context]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In libfdt.i we set the handling of uint64_t parameters to use
PyLong_AsUnsignedLong. But for 32-bit platforms, where an unsigned long
is 32-bits, this will truncate the value we need.
It turns out swig's default typemapping for uint64_t correctly handles
conversions both to python ints and python longs, so we don't need this
typemap at all.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Since commit 32b9c61307 "Preserve datatype markers when emitting dts
format", we no longer try to guess the value type. Instead, we reuse
the type of the datatype markers when they are present, if the type
is either TYPE_UINT* or TYPE_STRING.
This causes 'dtc -I fs' to crash:
Starting program: /root/dtc -q -f -O dts -I fs /proc/device-tree
/dts-v1/;
/ {
Program received signal SIGSEGV, Segmentation fault.
__strlen_power8 () at ../sysdeps/powerpc/powerpc64/power8/strlen.S:47
47 ld r12,0(r4) /* Load doubleword from memory. */
(gdb) bt
#0 __strlen_power8 () at ../sysdeps/powerpc/powerpc64/power8/strlen.S:47
#1 0x00007ffff7de3d10 in __GI__IO_fputs (str=<optimized out>,
fp=<optimized out>) at iofputs.c:33
#2 0x000000001000c7a0 in write_propval (prop=0x100525e0,
f=0x7ffff7f718a0 <_IO_2_1_stdout_>) at treesource.c:245
The offending line is:
fprintf(f, "%s", delim_start[emit_type]);
where emit_type is TYPE_BLOB and:
static const char *delim_start[] = {
[TYPE_UINT8] = "[",
[TYPE_UINT16] = "/bits/ 16 <",
[TYPE_UINT32] = "<",
[TYPE_UINT64] = "/bits/ 64 <",
[TYPE_STRING] = "",
};
/* Data blobs */
enum markertype {
TYPE_NONE,
REF_PHANDLE,
REF_PATH,
LABEL,
TYPE_UINT8,
TYPE_UINT16,
TYPE_UINT32,
TYPE_UINT64,
TYPE_BLOB,
TYPE_STRING,
};
Because TYPE_BLOB < TYPE_STRING and delim_start[] is a static array,
delim_start[emit_type] is 0x0. The glibc usually prints out "(null)"
when one passes 0x0 to %s, but it seems to call fputs() internally if
the format is exactly "%s", hence the crash.
TYPE_BLOB basically means the data comes from a file and we don't know
its type. We don't care for the former, and the latter is TYPE_NONE.
So let's drop TYPE_BLOB completely and use TYPE_NONE instead when reading
the file. Then, try to guess the data type at emission time, like the
code already does for refs and labels.
Instead of adding yet another check for TYPE_NONE, an helper is introduced
to check if the data marker has type information, ie, >= TYPE_UINT8.
Fixes: 32b9c61307
Suggested-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Path references are also a string, so add TYPE_STRING marker in addition
to REF_PATH.
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add SPI bus type detection and checks. The node name is the
preferred way to find SPI buses as there is no common compatible or
property which can be used. There are a few common properties used in
child nodes, so they can be used as a fallback detection method. This
lets us warn if the SPI controller is not properly named 'spi@...'.
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add I2C bus type detection and checks. The node name is used to find I2C
buses as there is no common compatible or property which can be used to
identify I2C controllers/buses. There are some common I2C properties,
but they are not used frequently enough to match on.
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We've accumulated a bunch of bugfixes, including considerable improvements
to libfdt's memory safety, so get ready for another release.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
vg_prepare_blob() assumes a valid return from fdt_num_mem_rsv() in order
to make sensible initialization of the valgrind mem checker. Usually
that's fine, but it breaks down on the (deliberately corrupted)
truncated_memrsv testcase.
That led to marking a negative-size (== enormously sized once cast to
size_t) as defined with VALGRIND_MAKE_MEM_DEFINED, which casued valgrind
to freeze up and consume ludicrous amounts of memory until OOMing.
This correction makes us robust in that case.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
You're not supposed to pass NULL to memcmp(), and some sanitizers complain
about it, even when the length is zero.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add internal fdt_cells() to avoid copy and paste. Test error cases and
default values. Fix typo in fdt_size_cells() documentation comment.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Generated phandle property values are a single cell, so set the type
marker to uint32. Otherwise, we default to uint8.
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
It is useful to be able to create a device tree from scratch using
software. This is supported in libfdt but not currently available in the
Python bindings.
Add a new FdtSw class to handle this, with various methods corresponding
to the libfdt functions. When the tree is complete, calling AsFdt() will
return the completed device-tree object.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson