Sublime Forum

Bug reading file with BOM on OSX

#1

Hey, I accidentally saved the py files in my plugin with a byte order mark. (Notepad and VS write one by default.) I then found OSX would not read them – see below, “import hashlib” is the first characters of the file after the BOM. Removing the BOM made the error disappear. Also, it works fine on Windows.

According to python.org/dev/peps/pep-0263/ the BOM should be treated as if the first line was “# coding: utf-8” so it should work fine.

Error spew with a BOM on OSX:
reloading plugin Web Inspector.swi
Traceback (most recent call last):
File “/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py”, line 74, in reload_plugin
m = importlib.import_module(modulename)
File “./importlib/init.py”, line 90, in import_module
File “”, line 1584, in _gcd_import
File “”, line 1565, in _find_and_load
File “”, line 1532, in _find_and_load_unlocked
File “/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py”, line 678, in load_module
exec(compile(source, source_path, ‘exec’), mod.dict)
File “swi in /Users/montogeek/Library/Application Support/Sublime Text 3/Installed Packages/Web Inspector.sublime-package”, line 1
import hashlib

0 Likes

#2

I talked to the developer of the VS Python tools, and he said:
“It’s possible that Sublime uses custom code to load Python files from disk, and they don’t handle BOM there correctly (i.e. they don’t strip it, just read the file as is and pass it on to Python as a string). As I recall, the BOM detection and removal logic in the interpreter itself will only work when reading from files.
It might be worth filing on them as a bug, as well, because the PEP clearly defines how they should behave here, even if, for some mysterious reason, they really need to be in the business of reading files themselves.”

If this is correct, then Sublime should perhaps strip any BOM and replace it with “# coding: utf-8” (although, that might mess up line numbers for debugging and exceptions), or possibly let Python read the file itself. Or, maybe Python’s interpreter needs to strip the BOM when passed python as a string.

0 Likes

#3

BTW, I’m not arguing BOM’s on UTF8 are a good thing. But they should work, according to the Python spec. And it would be good if they did, for VS users.

0 Likes

#4

Is this the right place to “file” this bug report, or should it go elsewhere?

0 Likes