Home Download Buy Blog Forum Support

Preprocessing files on load

Preprocessing files on load

Postby aksdb on Fri Mar 07, 2014 9:52 am

I'm currently trying to workaround a problem of ST2/3 with huuuuuge text files. And I'm not talking about a 10000 lines of code file, I'm talking about hundreds of megabyte of ascii, without any linebreaks. Such files a pretty usual on mainframes and I currently need to work with them.

Some are structured (EDIFACT for example) some are record based (fixed intervals of fixed length text).

Currently I preprocess these files for viewing externally (i.e. split EDIFACT by specific patterns and split the record based files into separate lines according to the record length).

Now it would be awesome if I could do that as a ST plugin so it happens magically. I imagine something like when opening a file beyond a certain size (>= 10 MB maybe) a popup asks me how to proceed and if split is intended, how to do it (by length or by pattern). Popup and splitting aren't the problem, however I can't find a way to intercept the file loading _before_ ST loads it completely - because that takes ages. So I have to "fix"/preprocess the buffer before ST tries to do anything with it (parsing, or whatever else it does while loading a file).

Any ideas?

BTW. just as a side note: vim, scite, notepad++, gedit, kate, ... all struggle with these kind of files. The only editor I have so far that loads them instantly is UltraEdit. So I don't blame ST for not handling them well, I just want to fix it :D
aksdb
 
Posts: 6
Joined: Wed Aug 22, 2012 1:23 pm

Re: Preprocessing files on load

Postby FichteFoll on Tue Mar 11, 2014 3:43 am

You won't get anything before "on_load" I'm afraid. The only thing you could possibly do is listen on "open_file" command calls in the "on_window_command" callback, but most files will be opened using the `window.open_file()` API or some other internal magic so you won't get to use that in many cases.
FichteFoll
 
Posts: 388
Joined: Fri Mar 16, 2012 11:49 pm
Location: Germany

Re: Preprocessing files on load

Postby duy on Wed Mar 12, 2014 2:02 am

Currently I preprocess these files for viewing externally (i.e. split EDIFACT by specific patterns and split the record based files into separate lines according to the record length).


You could create a bash script and pre-process the file in your prefered language before calling subl xyz. if you want to do it in sublime, you could create a new view, parse the file async and fill the view buffer by yourself (of course that would only be a shallow view of your file)
duy
 
Posts: 12
Joined: Mon Mar 10, 2014 12:28 am

Re: Preprocessing files on load

Postby aksdb on Fri Mar 14, 2014 9:16 am

Loading it completely by myself sounds like the only way currently. Too bad. I had hoped to get it "generic" so it also works with plugins like SFTP.
Then again, I probably won't open 2gb files on the fly via SFTP plugin.

I will have to think about this a bit more.

Thanks for your ideas :-)
aksdb
 
Posts: 6
Joined: Wed Aug 22, 2012 1:23 pm


Return to Plugin Development

Who is online

Users browsing this forum: aksdb and 4 guests