Unicode Issue

[seanzamora]

It appears luaparse is having trouble parsing non-English characters such as í. I implemented a temporary fix but I thought I should let you know. If I have time to prepare a permanent solution ill initiate a pull request.

Error:

SyntaxError: [1:240] unexpected symbol 'í' near 'Palad'

[fstirlitz]

What is the input?

Do you happen to be using the command line tool?

[seanzamora]

Input: ["Class"] = Paladín

No, library in NodeJS application.

[fstirlitz]

Would have appreciated a fuller sample, not one line stripped of any context.

But anyway, this is expected; identifiers in PUC-Rio Lua can only consist of ASCII characters. Latest git master adds the extendedIdentifiers option that also allows Unicode characters outside the Basic Latin block, to cover LuaJIT. Exact semantics are yet to be decided upon and stabilised.

Closing this as a duplicate of #53. Please follow that issue for more information.

[seanzamora]

Hey thanks for the info. The sample i provided was complete. : D

[fstirlitz]

In that case it still shouldn't parse, because it's not a valid Lua statement. I assumed it was part of a table constructor expression; there it would at least have made some sense.

[seanzamora]

Oh yeah i apologize for the confusion my script auto appends the appropriate table structure:

DB= {
["Class"] = "Paladín",
}

Thanks for the above solution appreciate it.

[seanzamora]

Hey fstirlitz,

Thanks for suggesting turning on extendedIdentifiers, it did help resolve my issue. I did identify a possible problem. Not sure if it is.

fixupHighCharacters() method seems to be obscuring Cyrillic characters (Russian characters). I read the section "Note on character encodings" and it was very informative and was related to the problem I was running into. I ended up overriding the method and simply returned the original string.

Can you elaborate on this I would appreciate it?

Thank in advance.

[fstirlitz]

I'm not sure what you mean, but follow #68 for future directions. Semantics are liable to change before the release.