markup for pstream token types is fragile
This seems to be the root of the issue I encountered while adding in dictionary macros. The additional MACRO enumeration value (placed immediately after WORD) means that the indices of the subsequent tokenType enumerations increase by one. This made DOUBLE_SCALAR receive a value of 9.
In UOPstream::write(double) this enumeration value is written as a character, which unfortunately corresponds to Tab and thus gets swallowed as a whitespace character. After this, the receiving end hasn't much chance.
I think that we should be invoking writeToBuffer(char) directly instead of using things like write(char(token::DOUBLE_SCALAR)) - should be more reliable.