@amaury: Ok, I added your long comment in tokenizer.c. You're also 
right about the strange code in the test. I reused ocean-city's 
test. "sys.exc_info()[2].tb_lineno" raises an additional (useless) 
error. So I simplified the code to use only "raise RuntimeError(...)" 
with the try/except/else.

Since tokenizer.c is hard to understand, I don't wnat to change the 
code of tok_nextc().