Skip to content
  • Mark OLESEN's avatar
    ENH: robuster lemon parsing · e6697edb
    Mark OLESEN authored
    - previously simply reused the scan token, which works fine for
      non-nested tokenizations but becomes too fragile with nesting.
    
      Now changed to use tagged unions that can be copied about
      and still retain some rudimentary knowledge of their types,
      which can be manually triggered with a destroy() call.
    
    - provide an 'identifier' non-terminal as an additional catch
      to avoid potential leakage on parsing failure.
    
    - adjust lemon rules and infrastructure:
    
      - use %token to predefine standard tokens.
        Will reduce some noise on the generated headers by retaining the
        order on the initial token names.
    
      - Define BIT_NOT, internal token rename NOT -> LNOT
    
    - handle non-terminal vector values.
      Support vector::x, vector::y and vector::z constants
    
    - permit fieldExpr access to time().
      Probably not usable or useful for an '#eval' expression,
      but useful for a Function1.
    
    - provisioning for hooks into function calls. Establishes token
      names for next commit(s).
    e6697edb