@brett.cannon: I found it: ast.c used a hack for iso-8859-1! Since 
this hack introduces a bug (your compile(...) example), I prefer to 
remove it to simplify to code. The new patch just removes the hack in 
tokenizer.c and ast.c. It does also document encoding and enc 
attributes of tok_state in tokenizer.h.

Using tokenizer_iso-8859-1-patch2.patch, all tests (test_pep3120.py, 
iso.py, test_shlex.py, etc.) are OK.