Improved delimter stop condition

This patch improves the delimiter stop condition to detect any characters that are not part of a normal token. As usual if you want you can always test your own delimiters.