The recent change to the lexer to only recognize property and node
names in the appropriate context removed a number of lexical warts in
our language that would have gotten ugly as we add expression support
and so forth.
But there's one nasty one remaining: references can contain a full
path, including the various problematic node name characters (',', '+'
and '-', for example). This would cause trouble with expressions, and
it also causes trouble with the patch I'm working on to allow
expanding references to paths rather than phandles. This patch
therefore reworks the lexer to mitigate these problems.
- References to labels cause no problems. These are now
recognized separately from references to full paths. No syntax change
here.
- References to full paths, including problematic characters
are allowed by "quoting" the path with braces
e.g. &{/pci@10000/somedevice@3,8000}. The braces protect any internal
problematic characters from being confused with operators or whatever.
- For compatibility with existing dts files, in v0 dts files
we allow bare references to paths as before &/foo/bar/whatever - but
*only* if the path contains no troublesome characters. Specifically
only [a-zA-Z0-9_@/] are allowed.
This is an incompatible change to the dts-v1 format, but since AFAIK
no-one has yet switched to dts-v1 files, I think we can get away with
it. Better to make the transition when people to convert to v1, and
get rid of the problematic old syntax.
Strictly speaking, it's also an incompatible change to the v0 format,
since some path references that were allowed before are no longer
allowed. I suspect no-one has been using the no-longer-supported
forms (certainly none of the kernel dts files will cause trouble).
We might need to think about this harder, though.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
dtc: Flexible tree checking infrastructure
Here, at last, is a substantial start on revising dtc's infrastructure
for checking the tree; this is the rework I've been saying was
necessary practically since dtc was first release.
In the new model, we have a table of "check" structures, each with a
name, references to checking functions, and status variables. Each
check can (in principle) be individually switched off or on (as either
a warning or error). Checks have a list of prerequisites, so if
checks need to rely on results from earlier checks to make sense (or
even to avoid crashing) they just need to list the relevant other
checks there.
For now, only the "structural" checks and the fixups for phandle
references are converted to the new mechanism. The rather more
involved semantic checks (which is where this new mechanism will
really be useful) will have to be converted in future patches.
At present, there's no user interface for turning on/off the checks -
the -f option now forces output even if "error" level checks fail.
Again, future patches will be needed to add the fine-grained control,
but that should be quite straightforward with the infrastructure
implemented here.
Also adds a testcase for the handling of bad references, which catches
a bug encountered while developing this patch.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds some options to the run_tests.sh script allowing it to
run all the testcases under valgrind to check for pointer corruption
bugs and memory leaks. Invoking "make checkm" will run the testsuite
with valgrind.
It include a mechanism for specifying valgrind errors to be suppressed
on a per-testcase basis, and adds a couple of such suppression files
for the mangle-layout and open_pack testcases which dump for use by
other testcases a buffer which may contain uninitialized sections. We
use suppressions rather than initializing the buffer so that valgrind
will catch any internal access s to the uninitialized data, which
would be a bug.
The patch also fixes one genuine bug caught by valgrind -
_packblocks() in fdt_rw.c was using memcpy() where it should have been
using memmove().
At present the valgrinding won't do anything useful for testcases
invoked via a shell script - which includes all the dtc testcases. I
plan to fix that later.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch removes a number of testcases from the testsuite that are
extremely unlikely to find any bugs that won't be found by the other
tests. This speeds up the testsuite.
- Both loops across the various tree block layouts run the
tree1_tests on the basic mangled tree. This is completely redundant,
so remove the second copy. This removes 456 testcases.
- We currently run tree1_tests on various trees manipulated by
move_and_save. We replace those with just a dtbs_equal_ordered test
to check that the manipulated tree is equal to the original. What
we're testing here is that fdt_move() operates correctly - it's very
unlikely it would succeed well enough for the ordered_equal test to
succeed, but the tree1_tests would fail on the result. This removes
162 testcases.
- Currently we re-ordered with mangle-layout both the basic
test_tree1.dtb and sw_tree1.test.dtb. Since we've already checked
that these dtbs are equivalent with dtbs_ordered_equal, it's very
unlikely that the tests would fail on one but not the other.
Therefore reduce this to only using test_tree1.dtb. This removes 828
testcases.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Lots of room for improvement here. Command line options, etc.
The script iterates over a hard-coded list of kernel DTS files.
Signed-off-by: Jon Loeliger <jdl@freescale.com>
The run_tests.sh script currently invokes the testcase binaries via
env(1). This behaviour is inherited from the libhugetlbfs testsuite
which uses this approach to easily set various configuration
environment variables in testcases.
We don't use that for dtc, and are unlikely to ever want to.
Therefore this patch removes that technique, which substantially
speeds up the testsuite.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds a group of testcases to check that dtc correctly
rejects trees with various structural errors.
To make things easier to test, we change dtc so that failing checks
(as opposed to other errors) result in exit code 2.
This patch also fixes an embarrasing bug uncovered by these new tests:
check_phandles() worked out if the tree's phandles were valid, then
throws that information away and returns success always.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The fdt_offset_ptr_typed() macro seemed like a good idea at the time.
However, it's not actually used all that often, it can silently throw
away const qualifications and it uses a gcc extension (typeof) which
I'd prefer to avoid for portability.
Therefore, this patch gets rid of it (and the fdt_offset_ptr_typed_w()
variant which was never used at all). It also makes a few variables
const in testcases, which always should have been const, but weren't
caught before because of the aforementioned silent discards.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In a number of places through libfdt and its tests, we have *_typed()
macro variants on functions which use gcc's typeof and statement
expression extensions to allow passing literals where the underlying
function takes a buffer and size.
These seemed like a good idea at the time, but in fact they have some
problems. They use typeof and statement expressions, extensions I'd
prefer to avoid for portability. Plus, they have potential gotchas -
although they'll deal with the size of the thing passed, they won't
deal with other representation issues (like endianness) and results
could be very strange if the type of the expression passed isn't what
you think it is.
In fact, the only users of these _typed() macros were when the value
passed is a single cell (32-bit integer). Therefore, this patch
removes all these _typed() macros and replaces them with explicit
_cell() variants which handle a single 32-bit integer, and which also
perform endian convesions as appropriate.
With this in place, it now becomes easy to use standardized big-endian
representation for integer valued properties in the testcases,
regardless of the platform we're running on. We therefore do that,
which has the additional advantage that all the example trees created
during a test run are now byte-for-byte identical regardless of
platform.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds fdt_get_phandle() and fdt_node_offset_by_phandle()
functions to libfdt. fdt_get_phandle() will retreive the phandle
value of a given node, and fdt_node_offset_by_phandle() will locate a
node given a phandle.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present, the Makefiles will not rebuild trees.o or the dtb files
derived from it if testdata.h is updated. This is incorrect, and is
because of missing dependency information.
This patch fixes the problem by making sure that dependency
information is generated from trees.S and dumptrees.c.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
dtc: Switch dtc to C-style literals
This patch introduces a new version of dts file, distinguished from
older files by starting with the special token /dts-v1/. dts files in
the new version take C-style literals instead of the old bare hex or
OF-style base notation. In addition, the "range" for of memreserve entries
(/memreserve/ f0000-fffff) is no longer recognized in the new format.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
This patch alters the main testcase, and the dts file corresponding to
it so that we at least trivially exercise dtc's bytestring and base
conversion features.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds a batch of testcases exercising dtc's -Odts mode.
Specifically it checks that using dtc to convert dtb->dts->dtb
preserves the original dtb for a number of example dtb files.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch changes -Odts mode output so that labels within property
values in the input are preserved in the output. Applied on top of
the earlier patch to preserve node and property labels in -Odts mode,
this means that dtc in -Idts -Odts mode will transfer all labels in
the input to the output.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There is a bug in the sw_tree1 testcase / utility which puts two
"compatible" properties in one node in the output tree. This patch
fixes the bug, and also adds a new test checking that the sw_tree1
output is equal to test_tree1.dtb as its supposed to be, which should
catch future errors of this type.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently all the read/write functions in libfdt require that the
given tree be v17, and further, that the tree has the memory
reservation block, structure block and strings block stored in that
physical order.
This patch eases these constraints, by making fdt_open_int() reorder
the blocks, and/or convert the tree to v17, so that it will then be
ready for the other read-write functions.
It also extends fdt_pack() to actually remove any gaps between blocks
that might be present.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present, all the example dtbs we use in the testsuite are version
17 and have reservation map, then structure block then strings block
(the natural ordering based on alignment constraints). However, all
libfdt's read-only and in-place write functions should also work on
v16 trees, and on trees with other layouts.
This patch adds a testcase / utility function to rearrange the blocks
of a dtb and/or regress a v17 tree to v16, and uses it to run tests on
trees with different layouts and versions.
Signed-off-by: David Gibson <david@tgibson.dropbear.id.au>
Although it's a low-level function that shouldn't normally be needed,
there are circumstances where it's useful for users of libfdt to use
the _fdt_next_tag() function. Therefore, this patch renames it to
fdt_next_tag() and publishes it in libfdt.h.
In addition, this patch adds a new testcase using fdt_next_tag(),
dtbs_equal_ordered. This testcase tests for structural equality of
two dtbs, including the order of properties and subnodes, but ignoring
NOP tags, the order of the dtb sections and the layout of strings in
the strings block. This will be useful for testing other dtc
functionality in the future.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present, dtc makes a lot of semantic checks on the device tree by
default, and will refuse to produce output if they fail. This means
people tend to need -f to force output despite failing semantic checks
rather a lot.
This patch splits the device tree checks into structural checks (no
bad or duplicate names or phandles) and semantic checks (everything
else). By default, only the structural checks are performed, and are
fatal. -f will force output even with structural errors (using this
in -Idts mode would essentially always be a bad idea, but it might be
useful in -Idtb mode for examining a malformed dtb).
Semantic checks are only performed if the new -c command line option
is supplied, and are always warnings only. Semantic checks will never
be performed on a tree with structural errors.
This patch is only a stopgap before implementing proper fine-grained
error/warning handling, but it should at least get rid of the
far-too-frequent need for -f for the time being.
This patch removes the -f from the dtc testcases now that it's no
longer necessary.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch makes the helper macros in trees.S use separate labels for
the end of each dt subblock, rather than using only start labels.
This means that the macros can now be used to create trees with the
subblocks in non-standard orders.
In addition, it adds a bunch of extra ; after lines of asm code in
macros, making them safe to use in nested macros.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch makes improvements to the way properties are printed when
in dtc is producing dts output.
- Characters which need escaping are now properly handled when
printing properties as strings
- The heuristics for what format to use for a property are
improved so that 'compatible' properties will be displayed as
expected.
- escapes.dts is altered to better demonstrate the changes,
and the string_escapes testcase is adjusted accordingly.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Recent commits 333542fabf and
fd1bf3a5ae added new testcases to dtc.
However, although the testcases were added to the Makefile and
run_tests.sh, one of the .c files for the testcase was omitted from
the patch in each case.
This patch restores the missing testcase code.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
dtc supports the use of C-style escapes (\n, \t and so forth) in
string property definitions via the data_copy_escape_string()
function. However, while it supports the most common escape
characters, it doesn't support the full set that C does, which is a
potential gotcha.
Worse, a bug in the lexer means that while data_copy_escape_string()
can handle the \" escape, a string with such an escape won't lex
correctly.
This patch fixes both problems, extending data_copy_escape_string() to
support the missing escapes, and fixing the regex for strings in the
lexer to handle internal escaped quotes.
This also adds a testcase for string escape functionality.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds functions for dealing with the compatible property.
fdt_node_check_compatible() can be used to determine whether a node is
compatible with a given string and fdt_node_offset_by_compatible()
locates nodes with a given compatible string.
Testcases for these functions are also included.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The dtc/libfdt testsuite creates a number of .dtb files during its
run. To ensure a clean test run, these are currently deleted before
each group of tests.
This is, in fact, a mistake, since if something goes wrong in the
first group of tests, deleting the .dtb at the beginning of the second
group of tests makes it harder to figure out what the problem was.
This patch changes the script to only delete the files once, before
the whole test run.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds functions to libfdt for accessing the memory
reservation map section of a device tree blob. fdt_num_mem_rsv()
retreives the number of reservation entries in a dtb, and
fdt_get_mem_rsv() retreives a specific reservation entry.
fdt_add_mem_rsv() adds a new entry, and fdt_del_mem_rsv() removes a
specific numbered entry.
Testcases for these new functions are also included.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present, the fdt_subnode_offset() and fdt_path_offset() functions
in libfdt require the exact name of the nodes in question be passed,
including unit address.
This is contrary to traditional OF-like finddevice() behaviour, which
allows the unit address to be omitted (which is useful when the device
name is unambiguous without the address).
This patch introduces similar behaviour to
fdt_subnode_offset_namelen(), and hence to fdt_subnode_offset() and
fdt_path_offset() which are implemented in terms of the former. The
unit address can be omitted from the given node name. If this is
ambiguous, the first such node in the flattened tree will be selected
(this behaviour is consistent with IEEE1275 which specifies only that
an arbitrary node matching the given information be selected).
This very small change is then followed by many more diffs which
change the test examples and testcases to exercise this behaviour.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present, the testcases for read/write functions (setprop,
del_property and del_node) are only invoked on the single
asm-generated tree, not on any of the other tree images which should
be equivalent. The functions in question will (correctly) not work on
the "unfinished" tree output from sw_tree1, but should work on most of
the others.
This patch extends the run_tests script to invoke the r/w testcases on
more example trees. The testsuite still passes clean with this
addition.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This large patch removes all trailing whitespace from dtc (including
libfdt, the testsuite and documentation). It also removes a handful
of redundant blank lines (at the end of functions, or when there are
two blank lines together for no particular reason).
As well as anything else, this means that quilt won't whinge when I go
to convert the whole of libfdt into a patch to apply to the kernel.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds a handful of simple testcases for dtc. It adds a dts
file which should generate the same sample tree as is used for the
libfdt testcases, and tests invoking dtc on this dts, plus the
standard batch of libfdt cases on the resulting dtb, which effectively
checks that the dtb is correct.
Because the test framework assumes each testcase is an executable with
the right output conventions, we use a little shell script, dtc.sh, as
a wrapper around dtc itself. It simply invokes dtc and returns a PASS
or FAIL depending on whether dtc returned an error.
It's not much, but it's a start.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
run_tests.sh from the dtc testsuite currently has a facility ro run
just "functional" or just "stress" tests. This distinction is carried
over from libhugetlbfs where the test framework originated, and where
it made sense.
In dtc, we have no stress tests, so running these subsections isn't
particularly interesting. This patch removes these test subsets,
instead defining a single "libfdt" test subset for testcases related
to libfdt (and not dtc proper only. Currently that's all of the
testcases, but with any luck we'll have some dtc testcases in the
future.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Flat device trees always have integers in their structure stored as
big-endian. From this point of view, property values are
bags-of-bytes and any endianness is up to users of the device tree to
determine.
The libfdt testcases which use properties with integer values,
currently use native endian format for the architecture on which the
testcases are run. This works ok for now, since both the creation and
checking of the example device trees happen in the same endianness.
This will become a problem, however, for tests of dtc which we want to
add in the nearish future. dtc always uses big-endian format for
'cell' format data in properties; as it needs to in order to produce
powerpc-usable device trees when hosted on a little-endian
architecture.
This patch, therefore, changes the libfdt testsuite to use big-endian
format always for integer format data, in order to interoperate sanely
with future dtc testcases. This also means that the example trees
created by the testsuite should now be byte-for-byte identical
regardless of dtc and libfdt's host platform, which is arguably an
advantage.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds a function to libfdt to locate nodes containing a
property with a specific value.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch turns on optimisation in the Makefile by default. With the
optimizer on, some uninitialized variable warnings (one real, two
bogus) are now generated. This patch also squashes those again.
The bookkeeping for producing the testsuite summary (total number of
tests passed, failed and so forth) is broken. It uses $? across
several tests, but for checks after the first, the value of $? will no
longer contain the original return code, but just that from the
previous test. This patch fixes the problem.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds an fdt_parent_offset() function which returns an
offset to the parent node of a given node. It also adds two helper
functions which are used to implement fdt_parent_offset() but are also
exported: fdt_supernode_atdepth_offset() which returns the ancestor of
a given node at a specified depth from the root of the tree, and
fdt_node_depth() which returns the depth of a given node.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds an fdt_get_path() function to libfdt, which returns
the full path of a given node in a caller supplied buffer.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds a new fdt_get_name() function to libfdt which will
return a node's name string (including unit address, if any).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, fdt_path_offset() returns FDL_ERR_BADOFFSET if given a path
with a trailing '/'. In particular this means that
fdt_path_offset("/") returns FDT_ERR_BADOFFSET rather than 0 as one
would expect.
This patch fixes the function to accept and ignore trailing '/'
characters. As well as allowing fdt_path_offset("/") this means that
fdt_path_offset("/foo/") will return the same as
fdt_path_offset("/foo") which seems in keeping with the principle of
least surprise.
This also adds a testcase to ensure that fdt_path_offset("/") returns
0 as it should.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch substantially revamps the dtc Makefiles, in particular
better integrating the Makefile for dtc proper with the Makefiles
imported from libfdt for libfdt and the shared testsuite. Notable
changes:
- No recursive make calls. Instead subsidiary Makefiles are
included into the top-level Makefile so we get a complete dependency
information.
- Common pattern rules, CFLAGS etc. shared between dtc, libfdt
and testsuite, rather than separate copies.
- Vaguely Kbuild-like non-verbose mode used by default, which
makes warnings more prominent.
- libfdt Makefile consists only of variable definitions and
helper rules, to make it more easily embeddable into other Makefile
systems.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There are quite a lot of testcases in the dtc testsuite (recently
imported from libfdt). It can be easy to miss a stray FAIL result in
the midst of all the rest. To improve this, this patch adds a summary
to the end of the testsuite results giving the total number of tests
along with the number of PASSes FAILs and other results.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
As a read-only functions, which take a const pointer to the fdt, treat
fdt_get_property() and fdt_getprop() as returning const pointers to
within the blob. fdt_get_property_w() and fdt_getprop_w() versions
are supplied which take a non-const fdt pointer and return a non-const
pointer for the benefit of callers wishing to alter the device tree
contents.
Likewise the lower-level fdt_offset_ptr() and _fdt_offset_ptr()
functions are changed to return const pointers, with *_w() versions
supplied.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present fdt.h #includes stdint.h. This makes some sense, because fdt.h
uses the standard fixed-width integer types. However, this can make life
difficult when building in different environments which may not have a
stdint.h. Therefore, this patch removes the #include from fdt.h, instead
requiring that users of fdt.h define the fixed-width integer types before
including fdt.h, either by themselves including stdint.h, or by any other
means.
David Gibson