Skip to content
Snippets Groups Projects
Commit 44a243a9 authored by Mark OLESEN's avatar Mark OLESEN
Browse files

ENH: improve stream handling of expansions (#2095)

* removed internal upper limit on word/string length for parsed input.

  - Although it has not caused many problems, no reason to retain
    these limits.
  - simplify some of the internal logic for reading string-like items.
  - localize parsers for better separation from the header

  - expose new function seekCommentEnd_Cstyle(), as useful
    handler of C-style comments

* exclude imbalanced closing ')' from word/variable

  - previously included this into the word/variable, but makes more
    sense to leave on the parser for the following token.

    Prevents content like 'vector (10 20 $zmax);' from being parsed
    as '$zmax)' instead of as '$zmax' followed by a ')'.
    No conceivable reason that the former would actually be desirable,
    but can still be obtained with brace notation: Eg, '${zmax)}'

* consistent handling of ${{ ... }} expressions

  - within a dictionary content, the following construct was
    incorrectly processed:

        value ${{2*sqrt(0.5)}};

    Complains about no dictionary/env variable "{2*sqrt(0.5)}"

    Now trap expressions directly and assign their own token type
    while reading. Later expansion can then be properly passed to
    the exprDriver (evalEntry) instead of incorrectly trying
    variable expansion.

    Does not alter the use of expressions embedded within other
    expansions. Eg, "file${{10*2}}"

* improve #eval { ... } brace slurping

  - the initial implementation of this was rudimentary and simply
    grabbed everything until the next '}'.  Now continue to grab
    content until braces are properly balanced

    Eg, the content:   value #eval{${radius}*2};

    would have previously terminated prematurely with "${radius" for
    the expression!

NOTE:
    both the ${{ expr }} parsed input and the #eval { ... } input
    discard C/C++ comments during reading to reduce intermediate
    overhead for content that will be discarded before evaluation
    anyhow.

* tighten recognition of verbatim strings and expressions.

  - parser was previously sloppy and would have accepted content such
    as "# { ..." (for example) as an verbatim string introducer.
    Now only accept parse if there are no intermediate characters
    discarded.
parent efd1ac4b
Branches
Tags
1 merge request!452ENH: adjust tokenizing
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment