Home Download Buy Blog Forum Support

Hex Viewer (a hex viewer and editor)

Hex Viewer (a hex viewer and editor)

Postby facelessuser on Sat Nov 12, 2011 10:14 pm

About
Hex Viewer is a plugin for Sublime Text 2 that allows the toggling of a file into a hex viewing mode. Hex Viewer also supports hex editing.

Image

Installation
  • Download is available in Package Control or you can download directly and drop into your Sublime Text 2 packages directory (plugin folder must be named HexViewer)
  • You may need to restart Sublime Text 2 after installation

Features
  • Toggling a file into hex view mode
  • Search bytes by address
  • Customizable highlight of byte and corresponding ASCII representation
  • Customizable byte grouping and bytes per line
  • Hex inspector to view current selected bytes as different unit types (endianness is configurable.)
  • Display total selected number of bytes and addresses of first group of consecutive bytes in status bar
  • Hex editing
  • Checksumming of files
  • Generate a hash from a string
  • Auto open binary files in Hex Viewer (disabled by default)
Source Code and More Documentation
https://github.com/facelessuser/HexViewer

Version 1.6
- Report Tiger-192 as big endian as most hashing programs do

Version 1.5
- Allow the hashing of current selection(s). Multiselect regions's content will be combined and evaluated together
- Update hash progress quicker
- Chunk hashes entered in the input panel in case very large amounts of data are pasted in

Version 1.4
- Fix issue where some selections will select an extra hex char that is not really selected
- Update missing item from changelog

Version 1.3
- Decrease debounce threads to one

Version 1.2
- Auto open specified binary files in Hex Viewer (disabled by default)

Version 1.1
- Add MD2 so Windows OS can find it (Mac and Linux do not have this one; Mac uses MDC2 for some reason)

Version 1.0
- Allow unicode text in "Generate Hash"

Version 0.9
- String to hash not always working now fixed
- Fixes for when certain hashes are not found

Version 0.8
- New hash algorithms: Whirlpool and Tiger-192
- Checksums are now threaded
- Exposed new command to tap into hashes called "Hex Viewer: Generate Hash"
- Rework logic to determine which hashes are available

Version 0.7
- New hash algorithms from openssl: sha, mdc2, md4, ripemd160 (dynamcially added if system supports them)
- Rename "Get Checksum" command to "Run Checksum"
- When invoking shortcut or command for checksumming, show quick panel with all valid hash algorithms to choose from

Version 0.6
- Commands accessible from menu Tools > Packages > Hex Viewer
- Fix checksum not responding to settings change
- Add CRC32 and ADLER32 hash algorithms for checksums

Version 0.5
- New progress animation in status bar when reading bin file
- Setting to enable or disable highlight throttling (default is disabled)
- Don't display incorrect byte count when highlight is throttled
- Get hex char range only once
- Better highlighting of edits
- A little cleanup

Version 0.4
- Hide Inspector tmLanguage file
- Chunk checksum and export for faster, more linear performance with large files
- Threaded binary file reading and show progress of current bytes parsed in status bar
- If hex view command is issued while a file is being converted, abort the conversion
- Simplify logic with hex view; always close original tab and load in new one opposed to conditional logic (easier with threading)
- Add setting to throttle highlighting selected bytes
- Add divider in settings menu

Version 0.3
- On export, rename tab to new export name
- Fix checksum failing when passing certain data through Sublime Text run_command API
- Account for NaNs in floats and doubles in Hex Inspector[/list]

Version 0.2
- Fix issue where large blocks get highlighted when the end of line is selected
- Add checksum command (also runs when export is made)

Version 0.1
- Initial release
Last edited by facelessuser on Wed Feb 08, 2012 4:53 am, edited 10 times in total.
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Mon Nov 14, 2011 7:24 pm

If you would like to have upper or lower nibble stand out, you can add this to your theme file and configure the nibbles' color to compliment each other.

nibble_accent.jpg
nibble_accent.jpg (14.95 KiB) Viewed 44748 times


Code: Select all
        <dict>
            <key>name</key>
            <string>Upper Byte Nibble</string>
            <key>scope</key>
            <string>raw.nibble.upper</string>
            <key>settings</key>
            <dict>
                <key>foreground</key>
                <string>#66D9EF</string>
            </dict>
        </dict>
        <dict>
            <key>name</key>
            <string>Lower Byte Nibble</string>
            <key>scope</key>
            <string>raw.nibble.lower</string>
            <key>settings</key>
            <dict>
                <key>foreground</key>
                <string>#FFFFFF</string>
            </dict>
        </dict>


I am purposely not scoping this by default because certain themes might not have good contrast colors for whatever I scope them as. This way, you can add it if you want it and have complete control to make sure they stand out how you like them.
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Tue Nov 15, 2011 5:21 pm

FYI, there is a small issue with checksumming from Hex Viewer. I already have a fix that I will commit tonight.

Basically I cannot pass hex data through the run_command api, at least not how I am doing it. If any of the hex data is not a printable ASCII, it will fail. This is easily remedied by postponing the conversion of the file data content to hex until right before checksum is ran.

I just had this nail me at work today .
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Wed Nov 16, 2011 12:34 am

New version posted:

#Version 0.3
- On export, rename tab to new export name
- Fix checksum failing when passing certain data through Sublime Text run_command API
- Account for NaNs in floats and doubles in Hex Inspector
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Wed Nov 16, 2011 3:38 pm

I am in optimization mode now. If you have noticed the speed of checksumming to be abysmal on very large files, don't worry, I have noticed as well.

I was able to improve the checksum speed by orders of magnitude on large files and probably exporting as well. I am cutting down anywhere we are applying code to extremely large buffers. This will allow python not to bog down trying to manage such large buffers. So in the next little bit, I will try to optimize all big processes (or at least attempt to further optimize). Loading is pretty good right now, I will see if I can push it a bit more. Checksumming and exporting should be much better. I will do some tests on restoring a tab speeds on large files, and also see if I can throttle highlighting on large selections.

Whether it happens before next commit or not, I will also be threading heavy hitting processes, and hopefully provide a cancel option in case you were silly enough to try and edit a 100MB binary file. ;)

I will at least have the optimized checksumming/exporting in a commit tonight. I just need to get some more real world testing in.
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Thu Nov 17, 2011 5:34 am

#Version 0.4
-Hide Inspector tmLanguage file
-Chunk checksum and export for faster, more linear performance with large files
-Threaded binary file reading and show progress of current bytes parsed in status bar
-If hex view command is issued while a file is being converted, abort the conversion
-Simplify logic with hex view; always close original tab and load in new one opposed to conditional logic (easier with threading)
-Add setting to throttle highlighting selected bytes
-Add divider in settings menu
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Thu Nov 17, 2011 4:30 pm

A couple of notes on the last commit:

I am only threading the reading of the binary file currently, so the editor still hangs on large files when loading the formatted hex data into the view buffer (I am using the replace API command which is the process that takes the longest). I will probably thread this portion as well when I am sure I am accounting properly for all things that could happen when this is handed off to the thread. I may even break up the loading of the data into the view buffer by chunking the data and using the insert API command instead. This might add a little overhead to the loading, but it would allow for a load to be canceled during the buffer load process as well, and also allow for some visual indication of how far along the buffer writing is.

I have throttled the highlighting of selected bytes, mainly because highlighting them requires me to parse each selection and then find it in the ASCII table (time consuming). You will notice that the byte count and displayed addresses in the status bar are tied to the bytes processed and highlighted, so the range of selected bytes and byte count will only represent currently selected bytes. In the next commit, I will probably just display the first address and not display the byte count (until I come up with a quicker way to count the bytes independent of byte highliting) if the byte highlight threshold has been reached.

I am currently not threading writing the binary file and check-summing the binary file (these are very quick compared to loading the binary file, but the time they take to finish is linearly proportional to the size of the file). I may thread these in the future when I finish addressing the loading of the file threading.
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Fri Nov 18, 2011 6:23 pm

New version posted:

A couple of notes:
I tried to thread the loading of the content to the buffer, but that is not reasonably possible. Edits to the view buffer must be on the main thread. So basically, the parsing of the bin file is threaded allowing you to cancel during that time, but as soon as the content is being written to the buffer, you just have to wait.

I also have decided against threading any other processes. The bin file loading is by far the weightiest process, and I have done about all I can there. Checksumming and bin file exporting is much much faster than loading. If you can wait for the file to load, checksumming and export will be no problem.

A couple of frivolous things made it in: new progress animation in status bar when parsing binary file, and better highlighting of edited bytes.

I also added another setting to turn off highlight throttling; it is now off by default. I originally added it when I thought the highlighting was causing the editor to lock up if you selected very large blocks (1500+ bytes). Though, while it will pause to highlight the bytes, the locking up is actually caused by multi-select. I turned off my plugin and tried to do a large multi-select in a normal code file, and the whole editor froze up and never gave me back control, so it is not the plugin. So if you are doing large multi-select blocks, you can enable throttling if you choose, but by default I have it disabled (I don't really do large multi-select blocks often, I mainly do large single selection blocks which are no problem at all; moderate multi-select blocks are also fine).
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Sat Nov 19, 2011 2:32 am

#Version 0.6
- Commands accessible from menu Tools > Packages > Hex Viewer (thanks to jsadeli)
- Fix checksum not responding to settings change
- Add CRC32 and ADLER32 hash algorithms for checksums

Somewhere along the lines I broke checksums responding to the settings file for the algorithm; this is now fixed. Also, I added some obvious checksumming algorithms that were not there before: CRC32 and ADLER32.

That should be all of the major things I wanted to get done. Let me know if any bugs turn up or you have any feature ideas.
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Re: Hex Viewer (a hex viewer and editor)

Postby facelessuser on Sun Nov 20, 2011 1:30 am

Got some new checksum hashes in: sha, mdc2, md4, ripemd160.

I already had the ones from hashlib, and zlib, but I pulled in the ones that are included with openssl. These may differ on some systems, so I currently check if they are available and only include them if they are. If I find some other ones that are available that I am not including, I will add those.

I was so close to getting whirlpool algorithm in here. The algorithm is public domain, and I had it working, but I could not get it working yet when the data is chunked; the hash comes out wrong. So, to get the right hash currently, the entire file's data is loaded into one big buffer and the hash is applied on that, and since it is 100% python, it takes a long time on big files...but it worked. Due to this, I have not included it. If I can deconstruct the algorithm enough to figure out how the get chunked data to evaluate the same, I will include it later.

Since the current supported hashes could vary on certain systems, when the checksum command or shortcut is invoked, I will now show a quickpanel populated with the valid hashes that can be used for selection.
facelessuser
 
Posts: 1571
Joined: Tue Apr 05, 2011 7:38 pm

Next

Return to Plugin Announcements

Who is online

Users browsing this forum: No registered users and 9 guests