This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
标题: Delayed exception using non-text encodings with TextIOWrapper
类型: behavior Stage: resolved
Components: IO Versions: Python 3.4
process
状态: closed Resolution: fixed
Dependencies: 后续:
分配给: ncoghlan 抄送列表: georg.brandl, larry, ncoghlan, pitrou, python-dev, serhiy.storchaka, vstinner
优先级: release blocker 关键字: patch

Created on 2014-01-27 05:01 by ncoghlan, last changed 2022-04-11 14:57 by admin. This issue is now closed.

文件
文件名 上传时间 Description 编辑
issue20404_check_valid_textio_codec.diff ncoghlan, 2014-01-27 12:29 Check codec validity in TextIOWrapper.__init__ review
issue20404_check_valid_textio_codec_v2.diff ncoghlan, 2014-02-02 05:08 Rationalise codec lookups during TextIOWrapper creation review
issue20404_check_valid_textio_codec_v3.diff ncoghlan, 2014-02-04 10:53 Updated for Serhiy's comments review
issue20404_check_valid_textio_codec-3.3.patch serhiy.storchaka, 2014-02-25 18:38 review
Messages (12)
msg209396 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) 日期: 2014-01-27 05:01
TextIOWrapper doesn't check the codec type early the way the convenience methods now do, so the open() method currently still suffers from the problems Victor described in issue 19619 for str.encode() and bytes.decode():

>>> with open("hex.txt", 'w') as f:
...     f.write("aabbccddeeff")
... 
12
>>> print(open('hex.txt').read())
aabbccddeeff
>>> print(open('hex.txt', encoding='hex').read())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: decoder should return a string result, not 'bytes'

These codecs are never going to work correctly with TextIOWrapper, so we should add the appropriate compatibility check in the constructor.
msg209399 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) 日期: 2014-01-27 05:25
I also created issue 20405 as an RFE for 3.5, since it seems there is a possible gap in capability relative to Python 2 here.
msg209438 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) 日期: 2014-01-27 12:29
Attached patch checks the requested encoding is a valid text encoding in TextIOWrapper.__init__.

Two additional test changes were needed:

- the one that checks handling of non-text-encodings at runtime now tweaks quopri to lie about being a text encoding when creating the stream

- the one that checks the interpreter doesn't crash at shutdown needed to be adjusted to handle the fact that _pyio now also fails in that situation, but with a different error (it can't find the ascii codec because the codec machinery is mostly gone)

Currently, this adds a third lookup of the encoding name to the process of creating a TextIOWrapper instance. This could be reduced to just one by changing the retrieval of the encoder and decoder to look in the retrieved codec info tuple, rather than doing the lookup by name again.
msg209948 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) 日期: 2014-02-02 05:08
Revised patch that avoids doing multiple lookups of the same codec name while creating the stream.

Absent any comments, I'll commit this version with appropriate NEWS and What's New updates tomorrow.
msg209949 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) 日期: 2014-02-02 05:26
Ah, just noticed the test case is still using the overly specific check for the exception wording. I'll fix that, too.
msg209970 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) 日期: 2014-02-02 11:17
The _PyCodec_GetIncrementalDecoder name looks too similar to PyCodec_IncrementalDecoder. It would be better to use more different name.

And please note my minor comments on Rietveld.
msg210197 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) 日期: 2014-02-04 10:53
v3:

- prefix for internal helper APIs is now _PyCodecInfo_ to better distinguish them from the ones that take an encoding name
- error check is now just for "is not a text encoding"
- tweaked the name and comment in the error test to be clear that it is codecs that are not marked as text encodings that are rejected, not just binary transforms
- fixed indentation of multi-line expression in _pyio.py
msg210198 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) 日期: 2014-02-04 11:00
LGTM.
msg210217 - (view) Author: Roundup Robot (python-dev) (Python triager) 日期: 2014-02-04 12:14
New changeset f3ec00d2b75e by Nick Coghlan in branch 'default':
Close #20404: blacklist non-text encodings in io.TextIOWrapper
http://hg.python.org/cpython/rev/f3ec00d2b75e
msg212195 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) 日期: 2014-02-25 18:38
Here is backported to 3.3 patch.
msg212539 - (view) Author: Roundup Robot (python-dev) (Python triager) 日期: 2014-03-02 08:22
New changeset 140a69d950eb by Georg Brandl in branch '3.3':
Issue #20404: reject non-text encodings early in TextIOWrapper.
http://hg.python.org/cpython/rev/140a69d950eb
msg244549 - (view) Author: Roundup Robot (python-dev) (Python triager) 日期: 2015-05-31 17:21
New changeset cf6e782a7f94 by Serhiy Storchaka in branch '2.7':
Issue #19543: Emit deprecation warning for known non-text encodings.
https://hg.python.org/cpython/rev/cf6e782a7f94
历史
日期 用户 动作 参数
2022-04-11 14:57:57admin修改github: 64603
2015-05-31 17:21:42python-dev修改消息: + msg244549
2014-03-02 08:22:51python-dev修改消息: + msg212539
2014-02-25 18:38:23serhiy.storchaka修改文件: + issue20404_check_valid_textio_codec-3.3.patch
抄送: + georg.brandl
消息: + msg212195

2014-02-04 12:14:10python-dev修改状态: open -> closed

抄送: + python-dev
消息: + msg210217

resolution: fixed
stage: commit review -> resolved
2014-02-04 11:00:20serhiy.storchaka修改消息: + msg210198
stage: test needed -> commit review
2014-02-04 10:53:39ncoghlan修改文件: + issue20404_check_valid_textio_codec_v3.diff

消息: + msg210197
2014-02-02 11:17:18serhiy.storchaka修改消息: + msg209970
2014-02-02 05:26:57ncoghlan修改消息: + msg209949
2014-02-02 05:08:59ncoghlan修改文件: + issue20404_check_valid_textio_codec_v2.diff

消息: + msg209948
2014-01-27 12:29:17ncoghlan修改文件: + issue20404_check_valid_textio_codec.diff

抄送: + pitrou
消息: + msg209438

components: + IO
keywords: + patch
2014-01-27 05:25:43ncoghlan修改消息: + msg209399
2014-01-27 05:01:16ncoghlan创建