dtc: Flexible tree checking infrastructure
Here, at last, is a substantial start on revising dtc's infrastructure
for checking the tree; this is the rework I've been saying was
necessary practically since dtc was first release.
In the new model, we have a table of "check" structures, each with a
name, references to checking functions, and status variables. Each
check can (in principle) be individually switched off or on (as either
a warning or error). Checks have a list of prerequisites, so if
checks need to rely on results from earlier checks to make sense (or
even to avoid crashing) they just need to list the relevant other
checks there.
For now, only the "structural" checks and the fixups for phandle
references are converted to the new mechanism. The rather more
involved semantic checks (which is where this new mechanism will
really be useful) will have to be converted in future patches.
At present, there's no user interface for turning on/off the checks -
the -f option now forces output even if "error" level checks fail.
Again, future patches will be needed to add the fine-grained control,
but that should be quite straightforward with the infrastructure
implemented here.
Also adds a testcase for the handling of bad references, which catches
a bug encountered while developing this patch.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds some options to the run_tests.sh script allowing it to
run all the testcases under valgrind to check for pointer corruption
bugs and memory leaks. Invoking "make checkm" will run the testsuite
with valgrind.
It include a mechanism for specifying valgrind errors to be suppressed
on a per-testcase basis, and adds a couple of such suppression files
for the mangle-layout and open_pack testcases which dump for use by
other testcases a buffer which may contain uninitialized sections. We
use suppressions rather than initializing the buffer so that valgrind
will catch any internal access s to the uninitialized data, which
would be a bug.
The patch also fixes one genuine bug caught by valgrind -
_packblocks() in fdt_rw.c was using memcpy() where it should have been
using memmove().
At present the valgrinding won't do anything useful for testcases
invoked via a shell script - which includes all the dtc testcases. I
plan to fix that later.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch removes a number of testcases from the testsuite that are
extremely unlikely to find any bugs that won't be found by the other
tests. This speeds up the testsuite.
- Both loops across the various tree block layouts run the
tree1_tests on the basic mangled tree. This is completely redundant,
so remove the second copy. This removes 456 testcases.
- We currently run tree1_tests on various trees manipulated by
move_and_save. We replace those with just a dtbs_equal_ordered test
to check that the manipulated tree is equal to the original. What
we're testing here is that fdt_move() operates correctly - it's very
unlikely it would succeed well enough for the ordered_equal test to
succeed, but the tree1_tests would fail on the result. This removes
162 testcases.
- Currently we re-ordered with mangle-layout both the basic
test_tree1.dtb and sw_tree1.test.dtb. Since we've already checked
that these dtbs are equivalent with dtbs_ordered_equal, it's very
unlikely that the tests would fail on one but not the other.
Therefore reduce this to only using test_tree1.dtb. This removes 828
testcases.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Lots of room for improvement here. Command line options, etc.
The script iterates over a hard-coded list of kernel DTS files.
Signed-off-by: Jon Loeliger <jdl@freescale.com>
The run_tests.sh script currently invokes the testcase binaries via
env(1). This behaviour is inherited from the libhugetlbfs testsuite
which uses this approach to easily set various configuration
environment variables in testcases.
We don't use that for dtc, and are unlikely to ever want to.
Therefore this patch removes that technique, which substantially
speeds up the testsuite.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds a group of testcases to check that dtc correctly
rejects trees with various structural errors.
To make things easier to test, we change dtc so that failing checks
(as opposed to other errors) result in exit code 2.
This patch also fixes an embarrasing bug uncovered by these new tests:
check_phandles() worked out if the tree's phandles were valid, then
throws that information away and returns success always.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The fdt_offset_ptr_typed() macro seemed like a good idea at the time.
However, it's not actually used all that often, it can silently throw
away const qualifications and it uses a gcc extension (typeof) which
I'd prefer to avoid for portability.
Therefore, this patch gets rid of it (and the fdt_offset_ptr_typed_w()
variant which was never used at all). It also makes a few variables
const in testcases, which always should have been const, but weren't
caught before because of the aforementioned silent discards.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In a number of places through libfdt and its tests, we have *_typed()
macro variants on functions which use gcc's typeof and statement
expression extensions to allow passing literals where the underlying
function takes a buffer and size.
These seemed like a good idea at the time, but in fact they have some
problems. They use typeof and statement expressions, extensions I'd
prefer to avoid for portability. Plus, they have potential gotchas -
although they'll deal with the size of the thing passed, they won't
deal with other representation issues (like endianness) and results
could be very strange if the type of the expression passed isn't what
you think it is.
In fact, the only users of these _typed() macros were when the value
passed is a single cell (32-bit integer). Therefore, this patch
removes all these _typed() macros and replaces them with explicit
_cell() variants which handle a single 32-bit integer, and which also
perform endian convesions as appropriate.
With this in place, it now becomes easy to use standardized big-endian
representation for integer valued properties in the testcases,
regardless of the platform we're running on. We therefore do that,
which has the additional advantage that all the example trees created
during a test run are now byte-for-byte identical regardless of
platform.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds fdt_get_phandle() and fdt_node_offset_by_phandle()
functions to libfdt. fdt_get_phandle() will retreive the phandle
value of a given node, and fdt_node_offset_by_phandle() will locate a
node given a phandle.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present, the Makefiles will not rebuild trees.o or the dtb files
derived from it if testdata.h is updated. This is incorrect, and is
because of missing dependency information.
This patch fixes the problem by making sure that dependency
information is generated from trees.S and dumptrees.c.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch alters the -Odts mode output so that it uses dts-v1 format.
This means that dtc -Idts -Odts used on a v0 dts file will convert
that file to v1.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
dtc: Switch dtc to C-style literals
This patch introduces a new version of dts file, distinguished from
older files by starting with the special token /dts-v1/. dts files in
the new version take C-style literals instead of the old bare hex or
OF-style base notation. In addition, the "range" for of memreserve entries
(/memreserve/ f0000-fffff) is no longer recognized in the new format.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
The current scheme of having CELLDATA and MEMRESERVE states to
recognize hex literals instead of node or property names is
arse-backwards. The patch switches things around so that literals are
lexed in normal states, and property/node names are only recognized in
the special PROPNODENAME state, which is only entered after a { or a
;, and is left as soon as we scan a property/node name or a keyword.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
This patch documents a few more functions in libfdt.h. All the
read-only functions are now documented.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch alters the main testcase, and the dts file corresponding to
it so that we at least trivially exercise dtc's bytestring and base
conversion features.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds a batch of testcases exercising dtc's -Odts mode.
Specifically it checks that using dtc to convert dtb->dts->dtb
preserves the original dtb for a number of example dtb files.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch changes -Odts mode output so that labels within property
values in the input are preserved in the output. Applied on top of
the earlier patch to preserve node and property labels in -Odts mode,
this means that dtc in -Idts -Odts mode will transfer all labels in
the input to the output.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently the main recursive tree printing function,
write_tree_source_node(), calls guess_type() to apply heuristics to
see how to print a property value, then calls the appropriate
write_propval_*() function to print it.
However, future heuristics for handling internal labels and the like
don't work well this way. Therefore, this patch refactors things to
have write_tree_source_node() call a new write_propval() function,
which incorporates the heurstic logic from guess_type() and also calls
the right function to do the actual printing.
No behavioural change.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There is a bug in the sw_tree1 testcase / utility which puts two
"compatible" properties in one node in the output tree. This patch
fixes the bug, and also adds a new test checking that the sw_tree1
output is equal to test_tree1.dtb as its supposed to be, which should
catch future errors of this type.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch documents a few more functions in libfdt.h. It also makes
a slight update to the description of the FDT_ERR_INTERNAL error code.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
libfdt: Add more documentation (patch the third)
This patch adds documentation in libfdt.h for a few more libfdt
functions. It also makes a slight update to the documentation of
fdt_get_name().
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
/usr/bin/install: cannot stat `fdt.h': No such file or directory
/usr/bin/install: cannot stat `libfdt.h': No such file or directory
Signed-off-by: Emil Medve <Emilian.Medve@Freescale.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
At present, defining a property as, say:
foo = [abcd], <ffffffff>;
Will cause dtc to insert 2 bytes of zeros between the abcd and the
ffffffff, to align the cell form data.
Doing so seemed like a good idea at the time, but I don't believe
there are any users who actually rely on this behaviour. Segher
claims that OF has some defined bindings which include properties an
unaligned subsection of which is interpreted as 32-bit ints (i.e. like
cell data).
Worse, this alignment will cause nothing but pain when we add
expression support to dtc (when celldata is included in a larger
bytestring expession, we won't know the size of the preceding chunk of
the expression until it's evaluated, so we would have to carry
alignment fixup information right through the expression evaluation
process).
Therefore, this patch kills off this alignment behaviour.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch changes -Odts mode output so that labels on properties,
nodes and memreserve entries in input source are preserved in the
output.
Preserving labels within property values is trickier - another patch
coming later.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Remove the uneccessary LDFLAGS from the top-level makefile. It only
added libfdt/ to the link path. dtc doesn't need libfdt at all, and
the testcases which do, already link libfdt.a by explicit path.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch moves the dtc code for checking the device tree its
processing into a new checks.c. The tree accessor functions from
livetree.c which the checks use are exported and added to dtc.h.
Another small step towards a flexible checking architecture.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently all the read/write functions in libfdt require that the
given tree be v17, and further, that the tree has the memory
reservation block, structure block and strings block stored in that
physical order.
This patch eases these constraints, by making fdt_open_int() reorder
the blocks, and/or convert the tree to v17, so that it will then be
ready for the other read-write functions.
It also extends fdt_pack() to actually remove any gaps between blocks
that might be present.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
If future dtb version > 17 are defined, that are still backwards
compatible with v16, libfdt will of course be able to read and
understand them. However, when modifying such a tree, it can't
guarantee that it won't clobber additional structure from the new
version which it doesn't know about. Therefore, before making
modifications to a tree of version >17, we must change it's version to
be exactly 17.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Previously, there were a few shift/reduce and reduce/reduce
errors in the grammar that were being handled by the not-so-popular
GLR Parser technique.
Flip a right-recursive stack-abusing rule into a left-recursive
stack-friendly rule and clear up three messes in one shot: No more
conflicts, no need for the GLR parser, and friendlier stackness.
Compensate by reversing the property list on the node.
Signed-off-by: Jon Loeliger <jdl@freescale.com>
At present, all the example dtbs we use in the testsuite are version
17 and have reservation map, then structure block then strings block
(the natural ordering based on alignment constraints). However, all
libfdt's read-only and in-place write functions should also work on
v16 trees, and on trees with other layouts.
This patch adds a testcase / utility function to rearrange the blocks
of a dtb and/or regress a v17 tree to v16, and uses it to run tests on
trees with different layouts and versions.
Signed-off-by: David Gibson <david@tgibson.dropbear.id.au>
The fdt_set_header() macro casts an arbitrary pointer into (struct
fdt_header *) to set fdt header fields. While we need to change the
type, so that we can use this macro on the usual (void *) used to
represent a device tree blob, the current macro also casts away any
const on the input pointer, which loses an important check.
This patch replaces the fdt_set_header() macro with a set of inline
functions, one for each header field which do a similar thing, but
which won't silently remove const from a given pointer. This approach
is also more in keeping with the individual accessor macros we use for
reading fdt header fields.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds some internal documentation in libfdt.h, in the form
of comments on the error codes and some functions. Only a couple of
functions are covered so far, leaving the documentation still woefully
inadequate, but hey, it's a start.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Although it's a low-level function that shouldn't normally be needed,
there are circumstances where it's useful for users of libfdt to use
the _fdt_next_tag() function. Therefore, this patch renames it to
fdt_next_tag() and publishes it in libfdt.h.
In addition, this patch adds a new testcase using fdt_next_tag(),
dtbs_equal_ordered. This testcase tests for structural equality of
two dtbs, including the order of properties and subnodes, but ignoring
NOP tags, the order of the dtb sections and the layout of strings in
the strings block. This will be useful for testing other dtc
functionality in the future.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
It's potentially useful for users of libfdt to sanity check a device
tree (or, rather, a blob of data which may or may not be a device
tree) before processing it in more detail with libfdt.
This patch renames the libfdt internal function _fdt_check_header() to
fdt_check_header() and makes it a published function, so it can now be
used for this purpose.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
I like to see the basis cases established early in
the rule sets, so place "empty" reduction first.
Purely cosmetic.
Signed-off-by: Jon Loeliger <jdl@freescale.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
At present, dtc makes a lot of semantic checks on the device tree by
default, and will refuse to produce output if they fail. This means
people tend to need -f to force output despite failing semantic checks
rather a lot.
This patch splits the device tree checks into structural checks (no
bad or duplicate names or phandles) and semantic checks (everything
else). By default, only the structural checks are performed, and are
fatal. -f will force output even with structural errors (using this
in -Idts mode would essentially always be a bad idea, but it might be
useful in -Idtb mode for examining a malformed dtb).
Semantic checks are only performed if the new -c command line option
is supplied, and are always warnings only. Semantic checks will never
be performed on a tree with structural errors.
This patch is only a stopgap before implementing proper fine-grained
error/warning handling, but it should at least get rid of the
far-too-frequent need for -f for the time being.
This patch removes the -f from the dtc testcases now that it's no
longer necessary.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
fdt_del_node(), unlike most of the rw functions does not check the
fdt's header with RW_CHECK_HEADER. However, it could make a mess of
things if the conditions in RW_CHECK_HEADER aren't met. So, this
patch adds the omitted check.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch makes the helper macros in trees.S use separate labels for
the end of each dt subblock, rather than using only start labels.
This means that the macros can now be used to create trees with the
subblocks in non-standard orders.
In addition, it adds a bunch of extra ; after lines of asm code in
macros, making them safe to use in nested macros.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, fdt_string() returns a (non-const) char *, despite taking a
const void *fdt. This is inconsistent with all the other read-only
functions which all return const pointers into the blob.
This patch fixes that.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch makes improvements to the way properties are printed when
in dtc is producing dts output.
- Characters which need escaping are now properly handled when
printing properties as strings
- The heuristics for what format to use for a property are
improved so that 'compatible' properties will be displayed as
expected.
- escapes.dts is altered to better demonstrate the changes,
and the string_escapes testcase is adjusted accordingly.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Recent commits 333542fabf and
fd1bf3a5ae added new testcases to dtc.
However, although the testcases were added to the Makefile and
run_tests.sh, one of the .c files for the testcase was omitted from
the patch in each case.
This patch restores the missing testcase code.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
dtc supports the use of C-style escapes (\n, \t and so forth) in
string property definitions via the data_copy_escape_string()
function. However, while it supports the most common escape
characters, it doesn't support the full set that C does, which is a
potential gotcha.
Worse, a bug in the lexer means that while data_copy_escape_string()
can handle the \" escape, a string with such an escape won't lex
correctly.
This patch fixes both problems, extending data_copy_escape_string() to
support the missing escapes, and fixing the regex for strings in the
lexer to handle internal escaped quotes.
This also adds a testcase for string escape functionality.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson