Home Download Buy Blog Forum Support

RegReplace Plugin

Re: RegReplace Plugin

Postby facelessuser on Tue Jan 17, 2012 3:12 am

# Version 0.5
- Make replace an optional parameter defaulted to "" (empty string)
- Allow override actions to use instead of replace: fold and unfold

Currently all finds are regex with option scope qualifiers. I plan on implementing the opposite as well: scope searches with regex qualifiers. That way you can find all regions of any scope and apply an optional regex on the region and determine if you want to do something to it.
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Tue Jan 17, 2012 6:48 pm

If I have enough time today, I plan on pushing out the option to do multi-passes using a regex sequence.

There are times when you have a regex that can be fairly complicated and can only find one instance of a particular target on a line. The nature of regex is that it can only find a finite number of matches. So if the complexity is so great that it is not feasible to target all in a single pass, multi-pass can help.

For example (I got this from another post today):
Somebody wanted to find instances of un-escaped dollar symbols between a certain tag. It is easy enough to target one, but to target an unknown number is quite a bit more difficult without perform regex on the regex result returned.
So this example will find one instance for every content tag.
Code: Select all
((?=<content)(?:.*))([^\\])\$((?:.*)(?<=</content>))


But I plan on adding a feature that will keep sweeping the file until no instances are found (or if you hit some threshold; I want to protect against infinite loops due to bad regex defines).

I think this will be a pretty good feature, and it shouldn't be to difficult to implement; I have most of it done already; I just need to restrict it to replace actions only and maybe update how I do feedback results to work with multi-pass.
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Wed Jan 18, 2012 3:48 am

Version 0.6
- Add multi-pass sweeps
- Report bad actions

Enjoy.
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Wed Jan 18, 2012 11:57 pm

Just a heads up; I will be commenting out all of the default commands except the "Reg Replace: Regex Input Sequencer" and "Reg Replace: Clear Highlights". I originally added the others as examples, so that is why I am going to have them commented out and remain, but I realized, I didn't want all of them, and they would have to be removed after every update.

So, moving forward, I will leave the default replace definitions in place because they hurt nothing and you can override them in your User folder if you don't want them. And I will leave the two commands mentioned above in place and comment out the rest so they can still be used as examples. I you like any of them, you can simply copy them to your Packages/User/Default.sublime-commands file, and those will persist with updates.

Figured I needed to mention it though in case some people used any of them and they disappeared on them.
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Thu Jan 19, 2012 3:57 am

# Version 0.7
- Replace command examples now commented out by default
- RegReplace Commands and Settings now available via preference menu and command palette

Very small update. So default replace commands are commented out, you can define your own in Default.sublime-commands in your User directory. Now you don't have to delete the default ones every time if you didn't want them.
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Thu Jan 19, 2012 7:31 pm

I have some fairly useful replacements.

These are definitions and commands for comments. Some C/C++ style, CSS (c multi-line), python style, and batch. They also come in multi line (where applicable), single line, and consecutive single line (good for code folding).

What is nice about these, is that they seem to work pretty good with folding without collapsing the next line into the first line. They seem to be working pretty good for me.

Code: Select all
      "batch_comments": {
         "find": "((?:^)?[^\\S\\r\\n]*)(::[^\\r\\n]*)",
         "replace": "",
         "scope_filter": ["comment", "-string"]
      },
      "batch_consecutive_single_comments": {
         "find": "((?:[^\\S\\r\\n]*::[^\\r\\n]*(?:[\\r\\n])?)*)([^\\S\\r\\n]*::[^\\r\\n]*)",
         "replace": "",
         "scope_filter": ["comment", "-string"]
      },
      "c_comments": {
         "find": "((?:^)?[^\\S\\r\\n]*)(//[^\\r\\n]*)",
         "replace": "",
         "scope_filter": ["comment", "-string"]
      },
      "c_consecutive_single_comments": {
         "find": "((?:[^\\S\\r\\n]*//[^\\r\\n]*(?:[\\r\\n])?)*)([^\\S\\r\\n]*//[^\\r\\n]*)",
         "replace": "",
         "scope_filter": ["comment", "-string"]
      },
      "c_multi_line_comments": {
         "find": "((?:^)?[^\\S\\r\\n]*)/(\\*([\\w\\W]*?)\\*/)([^\\S\\r\\n]*)",
         "replace": " ",
         "scope_filter": ["comment", "-string"]
      },
      "python_comments": {
         "find": "((?:^)?[^\\S\\r\\n]*)(#[^\\r\\n]*)",
         "replace": "",
         "scope_filter": ["comment", "-string"]
      },
      "python_consecutive_single_comments": {
         "find": "((?:[^\\S\\r\\n]*#[^\\r\\n]*(?:[\\r\\n])?)*)([^\\S\\r\\n]*#[^\\r\\n]*)",
         "replace": "",
         "scope_filter": ["comment", "-!string"]
      },
      "python_multi_line_double_block_strings": {
         "find": "((?:^)?[^\\S\\r\\n]*)(\"\"\"([\\w\\W]*?)\"\"\")([^\\S\\r\\n]*)",
         "replace": " ",
         "scope_filter": ["-comment", "string.quoted.double.block.python"]
      },
      "python_multi_line_single_block_strings": {
         "find": "((?:^)?[^\\S\\r\\n]*)('''([\\w\\W]*?)''')([^\\S\\r\\n]*)",
         "replace": " ",
         "scope_filter": ["-comment", "string.quoted.single.block.python"]
      }


Here are some remove and folding commands for comments.
Code: Select all
   {
      "caption": "Remove: Python Style Comments",
      "command": "reg_replace",
      "args": {"replacements": ["python_comments", "remove_trailing_spaces"], "find_only": true}
   },
   {
      "caption": "Remove: Python Block Strings",
      "command": "reg_replace",
      "args": {"replacements": [
         "python_multi_line_double_block_strings",
         "python_multi_line_single_block_strings",
         "remove_trailing_spaces"
      ], "find_only": true}
   },
   {
      "caption": "Remove: C Style Comments",
      "command": "reg_replace",
      "args": {"replacements": ["c_comments", "c_multi_line_comments", "remove_trailing_spaces"], "find_only": true}
   },
   {
      "caption": "Remove: CSS Comments",
      "command": "reg_replace",
      "args": {"replacements": ["c_multi_line_comments", "remove_trailing_spaces"], "find_only": true}
   },
   {
      "caption": "Remove: Batch Comments",
      "command": "reg_replace",
      "args": {"replacements": ["batch_comments", "batch_multi_line_comments", "remove_trailing_spaces"], "find_only": true}
   },
   // Fold Comments
   {
      "caption": "Code Folding: Fold Python Style Comments",
      "command": "reg_replace",
      "args": {"replacements": ["python_consecutive_single_comments"], "action": "fold"}
   },
   {
      "caption": "Code Folding: Fold Python Block Strings",
      "command": "reg_replace",
      "args": {"replacements": [
         "python_multi_line_double_block_strings",
         "python_multi_line_single_block_strings"
      ], "action": "fold"}
   },
   {
      "caption": "Code Folding: Fold C Style Comments",
      "command": "reg_replace",
      "args": {"replacements": ["c_consecutive_single_comments", "c_multi_line_comments"], "action": "fold"}
   },
   {
      "caption": "Code Folding: Fold CSS Comments",
      "command": "reg_replace",
      "args": {"replacements": ["c_multi_line_comments"], "action": "fold"}
   },
   {
      "caption": "Code Folding: Fold Batch Comments",
      "command": "reg_replace",
      "args": {"replacements": ["batch_consecutive_single_comments", "batch_multi_line_comments"], "action": "fold"}
   }


Unfold commands would be the same as the fold, except you change the caption to say "unfold" and change the action to "unfold".

If you have any improvements, let me know. If you know of some other comment styles, let me know. I may add some of these definitions by default to the plugin; I will have to see.

Edit: Typo fixed on Python block strings.
Edit: Scope fix on all comments
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Thu Jan 19, 2012 10:25 pm

They can get a little tripped up with some symbols in strings. Like if you have a comment symbol in a string and comment at the end of a line, but it isn't too bad.

This is quite common for regex though. You pretty much have to parse the entire line to determine where comments really start and stop. And in some languages it is easier than others. Currently if you parse the entire line, you may be able to target comment exactly, but currently "fold" actions etc, are performed on the entire regex selection. I do not allow fine targeting of regions within a regex at this time, maybe in the future.

Eventually, I will get the scope searching done, with regex qualifiers, and then comments can be targeted very accurately.
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Fri Jan 20, 2012 5:32 am

Okay, so I now have scope searching working, but I am not committing it yet; I only have parts of it working.

I have it now where you can greedily find scopes and apply an optional find and replace with regex on those scopes (greedy or non-greedy).

I do not have non-greedy scope search done yet. I do not have literal search and replace on a scope done yet either. But it does work great on greedy scope searches with regex. I have been playing around with, and it just made all of my regex for comments obsolete. :) .

The rules for defining a scope find and replace will be different.
Code: Select all
      "scope_test": {
         "scope": "comment",
         "find": "SomeRegex",
         "replace": "PrependText\\0",
         "greedy_replace": false,
         "greedy_scope": true,
         "literal": false,
         "case": true
      }
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Fri Jan 20, 2012 10:49 pm

Should have a new release out tonight hopefully; if not, tomorrow for sure.

I have the "searching by scope and then applying regex to the scope region" feature pretty much done (I am skipping literal find and replace for now). Something to note though, due to the nature of Python's subn function, you need to be careful with optional captured groups. Normally you will expect something like this to return an empty string for group 1 if this or that is not found.

Code: Select all
(this|that)?(SomethingElse)


But subn, which I am using for applying regex to the the scope region, will return "unmatched group error" if you try and access group 1 and group 2 when this or that is not found.

You can still use the "?" in all sorts of other instances, but it might give you issues if you try and use it on a group. There are probably other scenarios you might need to be mindful of.

So to fix this, you would have to provide an empty option. This probably makes the "?" mark unneeded now, but I just want to illustrate what you have to do to fix the error.
Code: Select all
(this|that|)?(SomethingElse)


This only applies to regex applied to scope regions. Regex with scope qualifiers goes through ST2's api which does not suffer from this.
I could use Python's findall function, but then I have to perform some coding gymnastics which I am not anxious to do right now, maybe down the road; we will see.


I also modified the folding command to not fold the trailing newline at the end of a selection. This makes folding regex much better, especially when targeting scopes like comments.

I also added a "mark" and "unmark" command where you can permanently highlight regions based on regex and also clear them; not sure how much use one might get out of this one, but it was easy to add, so I went ahead. It might come in handy.
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

Re: RegReplace Plugin

Postby facelessuser on Sat Jan 21, 2012 6:28 am

# Version 0.8
- New "mark" and "unmark" actions
- Return error dialog showing regex issue
- Add support for scope search with regex find and replace in scope region
- Smarter folding of regex regions for "fold" action
- Small tweak to non-greedy algorithm
- Change default of optional replace parameter to "\\0"; do not delete by default, leave unchanged by default.
- Allow spaces in the "Regex Input Sequencer"

One big thing to stress is the change of the default value of replace. If you have definitions that relied on the old default of "", the new default is "\\0". I decided it was better to not replace anything by default instead of deleting everything by default.

The other thing to note, is now when you create a malformed regex expression, you should get an error dialog giving you the internal error: things like unmatched brackets etc.

So now that scope selection is in, it is super easy to target difficult things like comments in every language. Here is a simple definition to delete all comments.

This is included in the default settings. It simply removes the comments leaving the line in tact
Code: Select all
      "remove_comments": {
         "scope": "comment",
         "find" : "([^\\n\\r]+)",
         "replace": "",
         "greedy_replace": true
      }


Here is the actual command:
Code: Select all
   {
      "caption": "Remove: All Comments",
      "command": "reg_replace",
      "args": {"replacements": ["remove_comments", "remove_trailing_spaces"], "find_only": true}
   },


Folding is much better now. Folding now will automatically adjust regions that have trailing newlines so as not to fold lines into each other.

Screen Shot 2012-01-20 at 11.12.33 PM.png
Screen Shot 2012-01-20 at 11.12.33 PM.png (42.52 KiB) Viewed 3063 times


Here are the fold/unfold commands:
Code: Select all
   // Folding
   {
      "caption": "Code Folding: Fold All Comments",
      "command": "reg_replace",
      "args": {"replacements": ["remove_comments"], "action": "fold"}
   },
   {
      "caption": "Code Folding: Unfold All Comments",
      "command": "reg_replace",
      "args": {"replacements": ["remove_comments"], "action": "unfold"}
   }


The mark and unmark actions are pretty self explanatory. It simply allows you to highlight specific searches by whatever scope you choose. You can have multiple different regex highlights on the screen at the same time as long as you give them unique keys. How to use them is documented in the readme. The highlights will persist until you delete the region, or clear them with the unmark command, etc. I guess it might be good if you have things you are always looking for in certain files. You can make them stand out a lot. It might not be that useful, it is kind of similar to the "find_only" option; the only difference is it doesn't disappear until you clear it. But it was easy to to add...so there it is.
facelessuser
 
Posts: 1563
Joined: Tue Apr 05, 2011 7:38 pm

PreviousNext

Return to Plugin Announcements

Who is online

Users browsing this forum: Yahoo [Bot] and 9 guests