v0.0-503-gc1dbc80: Improve handling of unexpected EOFs in the lexer.

The important thing is to not call yylex() after an EOF was encountered.

This allows unterminated comments (and similar) to gracefully return
accumulated text, and possible signal returning a different token type.
For comments, we permissively treat a missing \n as if it were implicitly there.

This also fixes the case where a macro definition body doesn't properly end
with a \n.

#PermissivelyNotPOSIX

fixes #242

PiperOrigin-RevId: 324287279