Home Download Buy Blog Forum Support

Bug: Non UTF-8 character causes file truncation

Bug: Non UTF-8 character causes file truncation

Postby GreyWyvern on Tue Oct 27, 2009 5:56 pm

When Sublime Text loads a file and encounters a multi-byte UTF-8 character, I assume it "switches on" UTF-8 display. If, later on in the file, a non UTF-8 byte is encountered, Sublime Text stops loading the rest of the file and the file appears truncated in the edit window.

If you open a notepad window (low-level editing), paste in the following text, and save the file, then load the file in Sublime Text the bug will appear. The first special character on line 2 is a multi-byte UTF-8 a-e ligature (æ) while the one on line 4 is a non UTF-8 double low-9 quotation mark.

Code: Select all
This is a new file.
æ
This text will be displayed.

Sublime Text will not display this line, nor any line beneath it.
Lorem ipsum dolor sit amet.


My fallbackEncoding hasn't been changed from the default.

Expected behaviour: Load the entire file and display at least a question mark or broken character box instead of the non UTF-8 character.
GreyWyvern
 
Posts: 106
Joined: Fri Oct 23, 2009 7:08 pm
Location: Canadia

Re: Bug: Non UTF-8 character causes file truncation

Postby jps on Wed Oct 28, 2009 10:37 am

Thanks for the bug report, this will be fixed in the next beta
jps
Site Admin
 
Posts: 3071
Joined: Wed Mar 19, 2008 12:33 pm

Re: Bug: Non UTF-8 character causes file truncation

Postby jps on Thu Oct 29, 2009 1:14 pm

This in fixed in 20091029.

If a file containing an invalid UTF-8 sequence is loaded, it'll now be correctly identified as invalid UTF-8, and the fallback encoding will be used. If it's explicitly chosen to be UTF-8 via File/Open with Encoding, then invalid sequences will be substituted with a replacement character.
jps
Site Admin
 
Posts: 3071
Joined: Wed Mar 19, 2008 12:33 pm

Re: Bug: Non UTF-8 character causes file truncation

Postby GreyWyvern on Thu Oct 29, 2009 2:26 pm

Fixed. Thank you :)
GreyWyvern
 
Posts: 106
Joined: Fri Oct 23, 2009 7:08 pm
Location: Canadia


Return to Technical Support

Who is online

Users browsing this forum: Exabot [Bot], Google [Bot] and 25 guests