When the DTS output has no type markers, we have to guess the type. Prior
to commit 32b9c61307 ("Preserve datatype markers when emitting dts
format"), instances of string lists would be delimited. Since then, a
single string with embedded "\0"s are emitted. An embedded "\0" is valid
for DTS files, but that's a rare exception and lists of strings are the
overwhelming majority. Restore the prior behavior.
stringlist.dts is reused for testing this, but needs a couple of tweaks
in order to match the dts output.
Reported-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Rob Herring <robh@kernel.org>
Message-ID: <20231027142901.2536622-1-robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The handling of "type preservation" dts output is based on the idea of
"phandles with arguments" in properties, which isn't really a thing, other
than a fairly common convention about how bindings are written. There's
nothing preventing a binding which freely mixes phandles and other integers
in an array of cells.
Currently write_propval() handles this incorrectly: specifically the case
of a phandle which follows a regular integer in a 32-bit cell array, but
without a new '< >' delimited causing an extra TYPE_UINT32 marker to be
inserted. In this case it omits the necessary space between the integer
and the phandle reference, leading to output which can't be sent back into
dtc and parsed.
Correct this, and update tests to match. I think this is more or less
correct for now, but really write_propval() is a big mess :(.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The dts output will just output phandle integer values, but often the
necessary markers are present with path or label references. Improve the
output and maintain phandle label or path references when present in dts
output.
Signed-off-by: Rob Herring <robh@kernel.org>
Message-Id: <20210727183023.3212077-6-robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We use the .string pseudo-op both in some of our test assembly files
and in our -Oasm output. We expect this to emit a \0 terminated
string into the .o file. However for certain targets (e.g. HP
PA-RISC) it doesn't include the \0. Use .asciz instead, which
explicitly does what we want.
There's also one place we can use .ascii (which explicitly emits a
string *without* \0 termination) instead of multiple .byte directives.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
When using overlays, a target-path property pointing to the root node is
quite common. However, "dtc -O dts" prints it as a byte array:
target-path = [2f 00];
instead of a string:
target-path = "/";
For guess_value_type() to consider a value to be a string, it must
contain less nul bytes than non-nul bytes, thus ruling out strings
containing only a single character. Allow printing such strings by
relaxing the condition slightly.
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Message-Id: <20200623094343.26010-1-geert+renesas@glider.be>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Because of the convention of packed representations in property layouts,
it's not uncommon to have integer values in properties which aren't
naturally aligned. Thus, there are several places in the dtc code where we
cast a potentially unaligned byte pointer into an integer pointer and load
it directly. On a number of architectures (including sparc64 and arm) this
won't work and will cause a fault. In some cases it may be trapped and
emulated by the kernel, but not always.
Therefore, replace such direct unaligned reads with a helper which will
handle unaligned data reads (a variant on the fdtXX_ld() functions already
used in libfdt).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Replace instances of GPLv2 or later boilerplate with SPDX tags.
Signed-off-by: Rob Herring <robh@kernel.org>
Message-Id: <20190620211944.9378-2-robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Provide the new command-line option:
--annotate (abbreviated -T)
--annotate provides one or more filenames and line numbers indicating
the origin of a given line. The filename is expressed relative the the
filename provided on the command line. Nothing is printed for overlays,
etc.
-T can be repeated giving more verbose annotations. These consist of
one or more tuples of: filename, starting line, starting column, ending
line ending column. The full path is given for the file name.
Overlays, etc are annotated with <no-file>:<no-line>.
The verbose annotations may be too verbose for normal use.
There are numerous changes in srcpos.c to provide the relative filenames
(variables initial_path, initial_pathlen and initial_cpp, new functions
set_initial_path and shorten_to_initial_path, and changes in
srcfile_push and srcpos_set_line). The change in srcpos_set_line takes
care of the case where cpp is used as a preprocessor. In that case the
initial file name is not the one provided on the command line but the
one found at the beginnning of the cpp output.
shorten_to_initial_path only returns a string if it has some shortening
to do. Otherwise it returns NULL and relies on the caller to use the
initial string. This simplifies memory management, by making clear to
the caller whether a new string is allocated.
The new functions srcpos_string_comment, srcpos_string_first, and
srcpos_string_last print the annotations. srcpos_string_comment is
recursive to print a list of source file positions.
Various changes are sprinkled throughout treesource.c to print the
annotations.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
If we have a phandle in the middle of a sequence of numbers and
it is not bracketed (e.g. <0x1234 &phandle 0x5678>), the dts output will
be corrupted due to missing a space between the phandle value and the
following number.
Fixes: 8c59a97ce0 ("Fix missing labels when emitting dts format")
Cc: Grant Likely <grant.likely@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Commit 8c59a97ce0 ("Fix missing labels when emitting dts format")
fixed label output, but broke output when there is a REF_PATH marker.
The problem is a REF_PATH marker causes a zero length string to be
emitted. The write_propval_string() function requires a length of at
least 1 (including the terminating '\0'), but that was not being
checked.
For the integer output, a length of 0 is valid as it is possible to have
labels inside the starting '<':
int-prop = < start: 0x1234>;
REF_PHANDLE is another marker that we don't explicitly handle, but it
doesn't cause a problem as it is fundamentally just an int.
Fixes: 8c59a97ce0 ("Fix missing labels when emitting dts format")
Reported-by: Kumar Gala <kumar.gala@linaro.org>
Cc: Grant Likely <grant.likely@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
When there is a label inside a sequence of ints at the end of a
property, an assertion is hit because write_propval() expects all the
labels to be at the very end of the property data. This is clearly wrong
behaviour.
To reproduce run: "dtc -O dts tests/label01.dts". dtc fails on property
/randomnode/blob.
Fix by reworking the write_propval() loop to remove the separate
iterating over label markers. Instead handle the label markers as part
of the main marker iteration loop. This guarantees that each label
marker is handled at the right location, even if all the data markers
have already been handled, and has the added advantage of making the
code simpler.
However, a side effect of this code is that a label at the very end of
an int sequence will be emitted outside the sequence delimiters. For
example:
Input: intprop = < 1 2 L1: >, L2: < 3 4 L3: > L4:;
Output: intprop = < 1 2 >, L1: L2: < 3 4 > L3: L4:;
The two representations are equivalent in the data model, but the
current test case was looking for the former, but needed to be modified
to look for the later. The alternative would be to render labels before
closing the sequence, but that makes less sense syntactically because
labels between sequences are normally to point at the next one, not the
former. For example:
Input: intprop = < 1 2 L1: >, L2: < 3 4 L3: > L4:;
Output: intprop = < 1 2 L1: L2: >, < 3 4 L3: L4: >;
DTC doesn't current have the information to know if the label should be
inside or outside the sequence, but in common usage, it is more likely
that L1 & L2 refer to the second sequence, not the end of the first.
Fixes: 32b9c61307 ("Preserve datatype markers when emitting dts
format")
Reported-by: Łukasz Dobrowolski <lukasz.dobrowolski@nokia.com>
Signed-off-by: Grant Likely <grant.likely@arm.com>
Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Commit 32b9c61307 ("Preserve datatype markers when emitting dts format")
add spaces between <> and [] and the encapsulated numbers. Fix this to
keep the prior formatting and not break some users needlessly.
Fixes: 32b9c61307 ("Preserve datatype markers when emitting dts format")
Reported-by: Stewart Smith <stewart@linux.ibm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Make type_marker_length available to other users of TYPE_* markers.
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Since commit 32b9c61307 "Preserve datatype markers when emitting dts
format", we no longer try to guess the value type. Instead, we reuse
the type of the datatype markers when they are present, if the type
is either TYPE_UINT* or TYPE_STRING.
This causes 'dtc -I fs' to crash:
Starting program: /root/dtc -q -f -O dts -I fs /proc/device-tree
/dts-v1/;
/ {
Program received signal SIGSEGV, Segmentation fault.
__strlen_power8 () at ../sysdeps/powerpc/powerpc64/power8/strlen.S:47
47 ld r12,0(r4) /* Load doubleword from memory. */
(gdb) bt
#0 __strlen_power8 () at ../sysdeps/powerpc/powerpc64/power8/strlen.S:47
#1 0x00007ffff7de3d10 in __GI__IO_fputs (str=<optimized out>,
fp=<optimized out>) at iofputs.c:33
#2 0x000000001000c7a0 in write_propval (prop=0x100525e0,
f=0x7ffff7f718a0 <_IO_2_1_stdout_>) at treesource.c:245
The offending line is:
fprintf(f, "%s", delim_start[emit_type]);
where emit_type is TYPE_BLOB and:
static const char *delim_start[] = {
[TYPE_UINT8] = "[",
[TYPE_UINT16] = "/bits/ 16 <",
[TYPE_UINT32] = "<",
[TYPE_UINT64] = "/bits/ 64 <",
[TYPE_STRING] = "",
};
/* Data blobs */
enum markertype {
TYPE_NONE,
REF_PHANDLE,
REF_PATH,
LABEL,
TYPE_UINT8,
TYPE_UINT16,
TYPE_UINT32,
TYPE_UINT64,
TYPE_BLOB,
TYPE_STRING,
};
Because TYPE_BLOB < TYPE_STRING and delim_start[] is a static array,
delim_start[emit_type] is 0x0. The glibc usually prints out "(null)"
when one passes 0x0 to %s, but it seems to call fputs() internally if
the format is exactly "%s", hence the crash.
TYPE_BLOB basically means the data comes from a file and we don't know
its type. We don't care for the former, and the latter is TYPE_NONE.
So let's drop TYPE_BLOB completely and use TYPE_NONE instead when reading
the file. Then, try to guess the data type at emission time, like the
code already does for refs and labels.
Instead of adding yet another check for TYPE_NONE, an helper is introduced
to check if the data marker has type information, ie, >= TYPE_UINT8.
Fixes: 32b9c61307
Suggested-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
If datatype markers are present in the property value, use them to
output the data in the correct format instead of trying to guess the
datatype. This also will preserve data grouping, such as in an
interrupts list.
This is a step forward for preserving and using datatype information
when processing DTS/DTB files. Schema validation tools can use the
datatype information to make sure a DT is correctly formed and
intepreted.
Signed-off-by: Grant Likely <grant.likely@arm.com>
[robh: rework marker handling and fix label output]
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This fixes a great many sparse warnings on the fdt and libfdt sources.
These are mostly due to incorrect mixing of endian annotated and native
integer types.
This includes fixing a couple of quasi-bugs where we had endian conversions
the wrong way around (this will have the right effect in practice, but is
certainly conceptually incorrect).
This doesn't make the whole tree sparse clean: there are many warnings in
bison and lex generated code, and there are a handful of other remaining
warnings that are (for now) more trouble than they're worth to fix (and
are not genuine bugs).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
struct fdt_reserve_entry is defined in fdt.h to exactly mirror the
in-memory layout of a reserve entry in the flattened tree. Since that is
always big-endian, it uses fdt64_t elements, which have sparse annotations
marking them as not native endian.
However, in dtc, we also use struct fdt_reserve_entry inside struct
reserve_info, and use it with native endian values. This will cause sparse
errors.
This stops this abuse, making struct reserve_info have its own native
endian fields for the same information.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
struct boot_info is named that for historical reasons, and isn't
particularly meaningful. Essentially it contains all the information -
in "live" form from a single dts or dtb file. As we move towards support
for dynamic dt overlays, that name will become increasingly bad.
So, in preparation, rename it to dt_info. At the same time rename the
'the_boot_info' global to 'parser_output' since that's its actual purpose.
Unfortunately we do need the global unless we switch to bison's re-entrant
parser extensions, which would introduce its own complications.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Some platforms (including the Microsoft C compiler) have char defaulting
to signed. write_propval_bytes() in the -O dts code will not behave
correctly in this case, due to sign extension.
Signed-off-by: Andrei Errapart <andrei@errapartengineering.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This has been there for ages, but the assertion makes no sense in the
context of the test immediately preceding it. This caused an abort()
when in -I dts -O dts mode with the right sort of internal labels in a
string property value.
Add a testcase for this and another candidate edge case (though this one
we already get right).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The isdigit(), isprint(), etc. functions take an int, whose value is
required to be in the range of an _unsigned_ char, or EOF. This, horribly,
means that systems which have a signed char by default need casts to pass
a char variable safely to these functions.
We can't do this more nicely by making the variables themselves 'unsigned
char *' because then we'll get warnings passing them to the strchr() etc.
functions.
At least the cygwin version of these functions, are designed to generate
warnings if this isn't done, as explained by this comment from ctype.h:
These macros are intentionally written in a manner that will trigger
a gcc -Wall warning if the user mistakenly passes a 'char' instead
of an int containing an 'unsigned char'.
Signed-off-by: Serge Lamikhov-Center <Serge.Lamikhov@gmail.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We already use the C99 bool type from stdbool.h in a few places. However
there are many other places we represent boolean values as plain ints.
This patch changes that.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch fixes a segmentation fault caused by dereferencing a NULL
pointer (pos->file aka yylloc.file) in srcpos_string when the input
length is 0 (fe 'dtc </dev/null'.) Reason: yylloc.file is initialized
with 0 and the tokenizer, which updates yylloc.file via srcpos_update
doesn't get a chance to run on zero-length input.
Signed-off-by: Horst Kronstorfer <hkronsto@frequentis.com>
At present, both the grammar and our internal data structures mean
that there can be only one label on a node or property. This is a
fairly arbitrary constraint, given that any number of value labels can
appear at the same point, and that in C you can have any number of
labels on the same statement.
This is pretty much a non-issue now, but it may become important with
some of the extensions that Grant and I have in mind. It's not that
hard to change, so this patch does so, allowing an arbitrary number of
labels on any given node or property. As usual a testcase is added
too.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
This patch cleans up our handling of input files, particularly dts
source files, but also (to an extent) other input files such as those
used by /incbin/ and those used in -I dtb and -I fs modes.
We eliminate the current clunky mechanism which combines search paths
(which we don't actually use at present) with the open relative to
current source file behaviour, which we do.
Instead there's a single srcfile_relative_open() entry point for
callers which opens a new input file relative to the current source
file (which the srcpos code tracks internally). It doesn't currently
do search paths, but we can add that later without messing with the
callers, by drawing the search path from a global (which makes sense
anyway, rather than shuffling it around the rest of the processing
code).
That suffices for non-dts input files. For the actual dts files,
srcfile_push() and srcfile_pop() wrappers open the file while also
keeping track of it as the current source file for future opens.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
When in -Odts mode, dtc will not produce correct output for
string-like properties which have more than one \0 character at the
end of the property's bytestring. In fact, it generates output which
is not syntactically correct. This patch fixes the bug, and adds a
testcase for future regressions here.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch marks various functions not shared between c files
'static', as they should be. There are a couple of functions in dtc,
and many in the testsuite.
This is *almost* enough to enable the -Wmissing-prototypes warning.
It's not quite enough, because there's a mess of junk in the flex
generated code which triggers that warning which I'm not yet sure how
to deal with.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Enabling -Wcast-qual warnings in dtc shows up a number of places where
we are incorrectly discarding a const qualification. There are also
some places where we are intentionally discarding the 'const', and we
need an ugly cast through uintptr_t to suppress the warning. However,
most of these are pretty well isolated with the *_w() functions. So
in the interests of maximum safety with const qualifications, this
patch enables the warnings and fixes the existing complaints.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently both libfdt and dtc define a set of endian conversion macros
for accessing the device tree blob which is always big-endian. libfdt
uses names like cpu_to_fdt32() and dtc uses names like cpu_to_be32 (as
the Linux kernel). This patch switches dtc over to using the libfdt
macros (including libfdt_env.h to supply them). This has a couple of
small advantages:
- Removes some code duplication
- Will make conversion a bit easier if we ever need to produce
little-endian device tree blobs.
- dtc no longer needs to pull in netinet/in.h simply for the
ntohs() and ntohl() functions
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, main() tests if it got a valid input tree from whichever
dt_from_*() function it invoked and if not, die()s. For one thing,
this test has, for no good reason, three different ways for those
functions to communicate a failure to provide input (bi NULL, bi->dt
NULL, or bi->error non-zero). For another, in every case save one, if
the dt_from_*() functions are unable to provide input they will
immediately die() (with a more specific error message) rather than
proceeding to the test in main().
Therefore, this patch removes this test, making the one case that
could have triggered it (in dt_from_source()) call die() directly
instead. With this change, the error field in struct boot_info is now
unused, so remove it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, dt_from_source() uses push_input_file() to set up the
initial input file for the lexer. That sounds sensible - put the
outermost input file at the bottom of the stack - until you realise
that what it *actually* does is pushes the current, uninitialized,
lexer input state onto the stack, then sets up the new lexer input.
That necessitates an extra check in pop_input_file(), rather than
signalling termination in the natural way when the include stack is
empty, it has to check when it pops the bogus uninitialized state off
the stack. Ick.
With that fixed, push_input_file(), pop_input_file() and
incl_file_stack itself become local to the lexer, so make them static.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
At present -I dts and -I fs modes both use the fill_fullpaths() helper
function to fill in the fullpath and basenamelen fields of struct
node, which are useful in later parts of the code. -I dtb mode,
however, fills these in itself.
This patch simplifies flattree.c by making -I dtb mode use
fill_fullpaths() like the others.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Previously, only failure to parse caused the reading of the tree to fail;
semantic errors that called yyerror() but not YYERROR only emitted a message,
without signalling make to stop the build.
Signed-off-by: Scott Wood <scottwood@freescale.com>
This adds 'const' qualifiers to many variables and functions. In
particular it's now used for passing names to the tree accesor
functions.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, every 'data' object, used to represent property values, has
two lists of fixup structures - one for labels and one for references.
Sometimes we want to look at them separately, but other times we need
to consider both types of fixup.
I'm planning to implement string references, where a full path rather
than a phandle is substituted into a property value. Adding yet
another list of fixups for that would start to get silly. So, this
patch merges the "refs" and "labels" lists into a single list of
"markers", each of which has a type field indicating if it represents
a label or a phandle reference. String references or any other new
type of in-data marker will then just need a new type value - merging
data blocks and other common manipulations will just work.
While I was at it I made some cleanups to the handling of fixups which
simplify things further.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch alters the -Odts mode output so that it uses dts-v1 format.
This means that dtc -Idts -Odts used on a v0 dts file will convert
that file to v1.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
This patch changes -Odts mode output so that labels within property
values in the input are preserved in the output. Applied on top of
the earlier patch to preserve node and property labels in -Odts mode,
this means that dtc in -Idts -Odts mode will transfer all labels in
the input to the output.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently the main recursive tree printing function,
write_tree_source_node(), calls guess_type() to apply heuristics to
see how to print a property value, then calls the appropriate
write_propval_*() function to print it.
However, future heuristics for handling internal labels and the like
don't work well this way. Therefore, this patch refactors things to
have write_tree_source_node() call a new write_propval() function,
which incorporates the heurstic logic from guess_type() and also calls
the right function to do the actual printing.
No behavioural change.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch changes -Odts mode output so that labels on properties,
nodes and memreserve entries in input source are preserved in the
output.
Preserving labels within property values is trickier - another patch
coming later.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch makes improvements to the way properties are printed when
in dtc is producing dts output.
- Characters which need escaping are now properly handled when
printing properties as strings
- The heuristics for what format to use for a property are
improved so that 'compatible' properties will be displayed as
expected.
- escapes.dts is altered to better demonstrate the changes,
and the string_escapes testcase is adjusted accordingly.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This large patch removes all trailing whitespace from dtc (including
libfdt, the testsuite and documentation). It also removes a handful
of redundant blank lines (at the end of functions, or when there are
two blank lines together for no particular reason).
As well as anything else, this means that quilt won't whinge when I go
to convert the whole of libfdt into a patch to apply to the kernel.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Keeps track of open files in a stack, and assigns
a filenum to source positions for each lexical token.
Modified error reporting to show source file as well.
No policy on file directory basis has been decided.
Still handles stdin.
Tested on all arch/powerpc/boot/dts DTS files
Signed-off-by: Jon Loeliger <jdl@freescale.com>
Rob Herring