- the tokenType was being send via write(char), which includes
  isspace filtering. If the tokenType enumeration coincides
  with a whitespace character, it would be suppressed.

  Now add character directly to the buffer.

STYLE: some additional minor private methods to help refactoring

- readStringFromBuffer, writeStringToBuffer for common string
  operation.