dtc does not use the input() function in flex. Apparently on some gcc
versions the unused function will cause warnings. Therefore, this
patch removes the function by using the 'noinput' option to flex.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently we scan the /include/ directive as two tokens, the
"/include/" keyword itself, then the string giving the file name to
include. We use a special scanner state to keep the two linked
together, and use the scanner state stack to keep track of the
original state while we're parsing the two /include/ tokens.
This does mean that we need to enable the 'stack' option in flex,
which results in a not-easily-suppressed warning from the flex
boilerplate code. This is mildly irritating.
However, this two-token scanning of the /include/ directive also has
some extremely strange edge cases, because there are a variety of
tokens recognized in all scanner states, including INCLUDE. For
example the following strange dts file:
/include/ /dts-v1/;
/ {
/* ... */
};
Will be processed successfully with the /include/ being effectively
ignored: the '/dts-v1/' and ';' are recognized even in INCLUDE state,
then the ';' transitions us to PROPNODENAME state, throwing away
INCLUDE, and the previous state is never popped off the stack. Or
for another example this construct:
foo /include/ = "somefile.dts"
will be parsed as though it were:
foo = /include/ "somefile.dts"
Again, the '=' is scanned without leaving INCLUDE state, then the next
string triggers the include logic.
And finally, we use a different regexp for the string with the
included filename than the normal string regexpt, which is also
potentially weird.
This patch, therefore, cleans up the lexical handling of the /include/
directive. Instead of the INCLUDE state, we instead scan the whole
include directive, both keyword and filename as a single token. This
does mean a bit more complexity in extracting the filename out of
yytext, but I think it's worth it to avoid the strageness described
above. It also means it's no longer possible to put a comment between
the /include/ and the filename, but I'm really not very worried about
breaking files using such a strange construct.
On Wed, Jun 04, 2008 at 09:26:23AM -0500, Jon Loeliger wrote:
> David Gibson wrote:
>
>> But as I said that can be dealt with in the future without breaking
>> compatibility. Objection withdrawn.
>>
>
> And on that note, I officially implore Scott to
> re-submit his binary include patch!
Scott's original patch does still have some implementation details I
didn't like. So in the interests of saving time, I've addressed some
of those, added a testcase, and and now resubmitting my revised
version of Scott's patch.
dtc: Add support for binary includes.
A property's data can be populated with a file's contents
as follows:
node {
prop = /incbin/("path/to/data");
};
A subset of a file can be included by passing start and size parameters.
For example, to include bytes 8 through 23:
node {
prop = /incbin/("path/to/data", 8, 16);
};
As with /include/, non-absolute paths are looked for in the directory
of the source file that includes them.
Implementation revised, and a testcase added by David Gibson
Signed-off-by: Scott Wood <scottwood@freescale.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Acked-by: Scott Wood <scottwood@freescale.com>
Currently, dt_from_source() uses push_input_file() to set up the
initial input file for the lexer. That sounds sensible - put the
outermost input file at the bottom of the stack - until you realise
that what it *actually* does is pushes the current, uninitialized,
lexer input state onto the stack, then sets up the new lexer input.
That necessitates an extra check in pop_input_file(), rather than
signalling termination in the natural way when the include stack is
empty, it has to check when it pops the bogus uninitialized state off
the stack. Ick.
With that fixed, push_input_file(), pop_input_file() and
incl_file_stack itself become local to the lexer, so make them static.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
All current callers of dtc_open_file() immediately die() if it returns
an error. In a non-interative tool like dtc, it's hard to see what
you could sensibly do to recover from a failure to open an input file
in any case.
Therefore, make dtc_open_file() itself die() if there's an error
opening the requested file. This removes the need for error checking
at the callsites, and ensures a consistent error message in all cases.
While we're at it, change the rror message from fstree.c when we fail
to open the input directory to match dtc_open_file()'s error message.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Error reporting in push_input_file() is a mess. One error results in
a message and exit(1), others result in a message and return 0 - which
is turned into an exit(1) at one callsite. The other callsite doesn't
check errors, but probably should. One of the error conditions gives
a message, but can only be the result of an internal programming
error, not a user error.
So. Clean that up by making push_input_file() a void function, using
die() to report errors and quit.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This allows /include/s to work when in non-default states,
such as PROPNODECHAR.
We may want to use state stacks to get rid of BEGIN_DEFAULT() altogether...
Signed-off-by: Scott Wood <scottwood@freescale.com>
Looking in the diretory dtc is invoked from is not very useful behavior.
As part of the code reorganization to implement this, I removed the
uniquifying of name storage -- it seemed a rather dubious optimization
given likely usage, and some aspects of it would have been mildly awkward
to integrate with the new code.
Signed-off-by: Scott Wood <scottwood@freescale.com>
Commit 7c44c2f9cb broke backwards
compatibility more badly than I realised. Contrary to what I thought
there are in-kernel, in-use dts files which relied on
references-to-path with paths including a comma, which no longer
compile after that commit.
So, this patch reinstates full support for bare references-to-path in
dts-v0 input. This means there will be some rather surprising lexical
corner cases when using path-expanded references in v0 files. But,
since path-expanded references are new, v0 files shouldn't typically
be using them anyway. If the corner cases cause a problem, you can
always convert to dts-v1 which handles the lexical issues here more
nicely.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch applies a couple of tiny cleanups to the lexer. The
not-very-useful 'WS' named pattern is removed, and the debugging
printf() for single character tokens is moved to the top of the
action, which results in less confusing output when LEXDEBUG is
switched on (because it goes before the printf()s for possible
resulting lexer state changes).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The recent change to the lexer to only recognize property and node
names in the appropriate context removed a number of lexical warts in
our language that would have gotten ugly as we add expression support
and so forth.
But there's one nasty one remaining: references can contain a full
path, including the various problematic node name characters (',', '+'
and '-', for example). This would cause trouble with expressions, and
it also causes trouble with the patch I'm working on to allow
expanding references to paths rather than phandles. This patch
therefore reworks the lexer to mitigate these problems.
- References to labels cause no problems. These are now
recognized separately from references to full paths. No syntax change
here.
- References to full paths, including problematic characters
are allowed by "quoting" the path with braces
e.g. &{/pci@10000/somedevice@3,8000}. The braces protect any internal
problematic characters from being confused with operators or whatever.
- For compatibility with existing dts files, in v0 dts files
we allow bare references to paths as before &/foo/bar/whatever - but
*only* if the path contains no troublesome characters. Specifically
only [a-zA-Z0-9_@/] are allowed.
This is an incompatible change to the dts-v1 format, but since AFAIK
no-one has yet switched to dts-v1 files, I think we can get away with
it. Better to make the transition when people to convert to v1, and
get rid of the problematic old syntax.
Strictly speaking, it's also an incompatible change to the v0 format,
since some path references that were allowed before are no longer
allowed. I suspect no-one has been using the no-longer-supported
forms (certainly none of the kernel dts files will cause trouble).
We might need to think about this harder, though.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
dtc: Switch dtc to C-style literals
This patch introduces a new version of dts file, distinguished from
older files by starting with the special token /dts-v1/. dts files in
the new version take C-style literals instead of the old bare hex or
OF-style base notation. In addition, the "range" for of memreserve entries
(/memreserve/ f0000-fffff) is no longer recognized in the new format.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
The current scheme of having CELLDATA and MEMRESERVE states to
recognize hex literals instead of node or property names is
arse-backwards. The patch switches things around so that literals are
lexed in normal states, and property/node names are only recognized in
the special PROPNODENAME state, which is only entered after a { or a
;, and is left as soon as we scan a property/node name or a keyword.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
dtc supports the use of C-style escapes (\n, \t and so forth) in
string property definitions via the data_copy_escape_string()
function. However, while it supports the most common escape
characters, it doesn't support the full set that C does, which is a
potential gotcha.
Worse, a bug in the lexer means that while data_copy_escape_string()
can handle the \" escape, a string with such an escape won't lex
correctly.
This patch fixes both problems, extending data_copy_escape_string() to
support the missing escapes, and fixing the regex for strings in the
lexer to handle internal escaped quotes.
This also adds a testcase for string escape functionality.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This large patch removes all trailing whitespace from dtc (including
libfdt, the testsuite and documentation). It also removes a handful
of redundant blank lines (at the end of functions, or when there are
two blank lines together for no particular reason).
As well as anything else, this means that quilt won't whinge when I go
to convert the whole of libfdt into a patch to apply to the kernel.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Change the lexer to recognise a label in any context. Place
before other celldata and bytestrings to avoid the initial
characters being stolen by other matches.
A label is a character sequence starting with an alphabetic
or underscore optinally followed by the same plus digits and
terminating in a colon.
The included terminating colon will prevent matching hex numbers.
Signed-off-by: Milton Miller <miltonm@bga.com>
At present, the lexer in dtc recognizes only space, tab and newline as
whitespace characters. This is broken; in particular this means that
dtc will get syntax errors on files with DOS-style (CR-LF) newlines.
This patch fixes the problem, using flex's built-int [:space:]
character class.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
- Change include syntax to: /include/ "filename"
- Move private functions directly into dtc-lexer.l
- Define YYID for some older parser templates
Also fix a #include ordering problem around YYLTPE.
Signed-off-by; Jon Loeliger <jdl@freescale.com>
Acked-by: Haiying Wang <Haiying.Wang@freescale.com>
Keeps track of open files in a stack, and assigns
a filenum to source positions for each lexical token.
Modified error reporting to show source file as well.
No policy on file directory basis has been decided.
Still handles stdin.
Tested on all arch/powerpc/boot/dts DTS files
Signed-off-by: Jon Loeliger <jdl@freescale.com>
New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base. Eg: <d# 123>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
At present each property definition in a dts file must give as the
value either a string ("abc..."), a bytestring ([12abcd...]) or a cell
list (<1 2 3 ...>). This patch allows a property value to be given as
several of these, comma-separated. The final property value is just
the components appended together. So a property could have a list of
cells followed by a string, or a bytestring followed by some cells.
Cells are always aligned, so if cells are given following a string or
bytestring which is not a multiple of 4 bytes long, zero bytes are
inserted to align the following cells.
The primary motivation for this feature, however, is to allow defining
a property as a list of several strings. This is what's needed for
defining OF 'compatible' properties, and is less ugly and fiddly than
using embedded \0s in the strings.
Signed-off-by: David Gibson <dwg@au1.ibm.com>
Signed-off-by: Jon Loeliger <jdl@freescale.com>
lex file has undefined behaviour. In fact it ends up including the stuff
within the definition of the yylex() function, leading to strange warnings
on gcc-3.4 and compile errors with gcc 4.
David Gibson