Home Download Buy Blog Forum Support

Oniguruma syntax definition help.

Re: Oniguruma syntax definition help.

Postby askeeve on Tue May 15, 2012 7:49 pm

atomi wrote:If you don't want to match the starting ; and ending :Code you can use non-capturing groups ie: (?:;)

I appreciate the tip, there are lots of ways I could deal with the bounds on it. I'm struggling more with the super specific first ';'. I've gotten an implementation of matching outer-most brackets working with the Boost regex in ST2's search, but I can't seem to get it to translate to the Oniguruma regex for the syntax file. And even then, I'm not 100% sure I'll be able to match the first ';' not in braces. I think I know how it will work but I haven't been able to test it. Any implementation to find that ';' and use it as the leftmost bound of my match would be seriously welcome.

Also, just curious about the "across one pattern" part. I know that once I've matched comments or strings, any "special" syntax within them doesn't get highlighted. Is that because they've been matched already or because they've been matched and assigned a scope-name?
askeeve
 
Posts: 51
Joined: Wed Apr 18, 2012 1:33 pm

Re: Oniguruma syntax definition help.

Postby atomi on Tue May 15, 2012 9:30 pm

You can still hilight syntax nested in an already matched string like comments.

What you do is nest a new pattern within that comment syntax definition and scope it.

I was always thinking something simple like:

Code: Select all
<dict>
    <key>begin</key>
    <string>(;)(?!})</string>
    <key>captures</key>
    <dict>
        <key>1</key>
        <dict>
            <key>name</key>
            <string>punctuation.definition.something.begin.code</string>
        </dict>
    </dict>
    <key>end</key>
    <string>(?i:\:code)</string>
    <key>name</key>
    <string>orphaned.blah.something.code</string>
    <key>patterns</key>
    <array>
        <dict>
            <key>match</key>
            <string>{(.*)}</string>
            <key>name</key>
            <string>entity.some.nested.curly.code</string>
        </dict>
    </array>
</dict>


But yea, the scope names should be changed to whatever is appropriate for your case.
EDit: Not tested...
atomi
 
Posts: 342
Joined: Thu Jan 20, 2011 5:06 pm
Location: Los Angeles CA US

Re: Oniguruma syntax definition help.

Postby nick. on Wed May 16, 2012 2:30 am

I'm not saying this is what you should do, it's just a 5-minute example to get you started:
Capture.PNG
Capture.PNG (17.1 KiB) Viewed 858 times
Code: Select all
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>fileTypes</key>
   <array>
      <string>pl</string>
   </array>

   <key>firstLineMatch</key>
   <string>^#!.*\bperl\b</string>

   <key>foldingStartMarker</key>
   <string>[</string>

   <key>foldingStopMarker</key>
   <string>]</string>

   <key>name</key>
   <string>Foo language</string>

   <key>scopeName</key>
   <string>source.foo</string>

   <key>uuid</key>
   <string>491AB614-A9F1-4733-ADC1-2F33213E87F7</string>





   /* Syntax Patterns
    * ================================== */
   <key>patterns</key>
   <array>
      <dict>
         <key>contentName</key>
         <string>comment</string>
         <key>begin</key>
         <string>;</string>
         <key>end</key>
         <string>\:Code</string>
      </dict>

      <dict>
         <key>name</key>
         <string>three</string>
         <key>match</key>
         <string>[^{]*(?={)</string>
      </dict>

      <dict>
         <key>name</key>
         <string>one</string>
         <key>begin</key>
         <string>{</string>
         <key>beginCaptures</key>
         <dict>
            <key>0</key>
            <dict>
               <key>name</key>
               <string>begin</string>
            </dict>
         </dict>
         <key>end</key>
         <string>}</string>
         <key>endCaptures</key>
         <dict>
            <key>0</key>
            <dict>
               <key>name</key>
               <string>end</string>
            </dict>
         </dict>
         <key>patterns</key>
         <array>
            <dict>
               <key>include</key>
               <string>#function</string>
            </dict>
            <dict>
               <key>include</key>
               <string>#string</string>
            </dict>
            <dict>
               <key>include</key>
               <string>#nested_braces</string>
            </dict>
         </array>
      </dict>
   </array>





   /* Repository
   * ================================== */

   <key>repository</key>
   <dict>

      <key>function</key>
      <dict>
         <key>name</key>
         <string>markup.deleted.diff</string>
         <key>match</key>
         <string>@\w+</string>
      </dict>

      <key>string</key>
      <dict>
         <key>name</key>
         <string>string</string>
         <key>match</key>
         <string>"\w+"</string>
      </dict>

      <key>nested_braces</key>
      <dict>
         <key>begin</key>
         <string>{</string>
         <key>beginCaptures</key>
         <dict>
            <key>0</key>
            <dict>
               <key>name</key>
               <string>two</string>
            </dict>
         </dict>
         <key>end</key>
         <string>}</string>
         <key>endCaptures</key>
         <dict>
            <key>0</key>
            <dict>
               <key>name</key>
               <string>two</string>
            </dict>
         </dict>
         <key>patterns</key>
         <array>
            <dict>
               <key>include</key>
               <string>#function</string>
            </dict>
            <dict>
               <key>include</key>
               <string>#string</string>
            </dict>
            <dict>
               <key>include</key>
               <string>#nested_braces</string>
            </dict>
         </array>
      </dict>

   </dict>

</dict>
</plist>
nick.
 
Posts: 266
Joined: Wed Jan 18, 2012 3:45 am

Re: Oniguruma syntax definition help.

Postby askeeve on Wed May 16, 2012 12:16 pm

nick. wrote:I'm not saying this is what you should do, it's just a 5-minute example to get you started:

This looks super helpful! Thank you so much! If I could just ask you a couple of questions about what's happening here to make sure I understand as I fine tune it:
  • 1) I haven't used the repository keyword before. Is that just where you define patterns that can be used with an include? Or can any pattern be used as an include and is the repository just a way of visually separating patterns that are explicitly for that purpose? Does that mean that your "function" and "string" matches will match the entire file like any other match?
  • 2) Do you have to match the function/string inside the nested braces explicitly (as you're doing with the includes)? Technically any code is valid inside braces and I'm already doing a lot of matching for that. Will the nested brace matching negate all that other stuff or do I just have to match the nested braces after (or before?) all the other matches?
I didn't quite understand the patterns keyword inside of a dict before seeing your example. Also it seems like doing an include of a match inside itself is by far the best way to implement recursive matching in a pattern. Certainly the easiest to read. Would you agree?
Once again thank you so much!
askeeve
 
Posts: 51
Joined: Wed Apr 18, 2012 1:33 pm

Re: Oniguruma syntax definition help.

Postby nick. on Wed May 16, 2012 1:35 pm

1. The repository is like a library that you reference using the include key. You can use include anywhere a key is valid, but you can only include from the repository.

2. As I said previously, nothing can be matched twice. So if you match some content as a nested brace, it will be only scoped as a nested brace. You can implement subpatterns of the nested braces (as I have done in the example) to add specific scopes within the braces.

It's also possible to include $self, but that's pretty tricky to get right.

See here for the full documentation.
nick.
 
Posts: 266
Joined: Wed Jan 18, 2012 3:45 am

Re: Oniguruma syntax definition help.

Postby askeeve on Wed May 16, 2012 2:48 pm

So anything in the repository isn't matched unless it's included somewhere? Ergo, if I want to match all the same things inside the nested braces as I am outside the nested braces, I'll essentially have to move all my matches to the repository, and then then include them in my patterns as well as in the "begin": "\{", "end": "\}"'s patterns? Obviously I could play with how I group it to be a little more readable/clean but that's the gist of it?
askeeve
 
Posts: 51
Joined: Wed Apr 18, 2012 1:33 pm

Re: Oniguruma syntax definition help.

Postby atomi on Wed May 16, 2012 2:59 pm

askeeve wrote:So anything in the repository isn't matched unless it's included somewhere?


Yea, the repository is used so you can easily group, include and nest multiple patterns. It prevents copy paste code. Anything in the repository is not matched unless it's included in "patterns"

Great questions by the way. This is going to be a nice thread to point to folks asking about tmLanguage development.
atomi
 
Posts: 342
Joined: Thu Jan 20, 2011 5:06 pm
Location: Los Angeles CA US

Re: Oniguruma syntax definition help.

Postby askeeve on Wed May 16, 2012 3:06 pm

Thanks :) I think writing a syntax file is one of the best ways to teach yourself regex. I've used it many times in the past but this experience has taught me so many tricks, like zero-width assertions (Awesome!). I'm a little disappointed by how complicated my syntax file will be now but that's really the fault of this screwed up language I'm being forced to work with.
askeeve
 
Posts: 51
Joined: Wed Apr 18, 2012 1:33 pm

Re: Oniguruma syntax definition help.

Postby askeeve on Wed May 16, 2012 3:53 pm

nick. wrote:I'm not saying this is what you should do, it's just a 5-minute example to get you started:

Could I simplify this
Code: Select all
   /* Syntax Patterns
    * ================================== */
   <key>patterns</key>
   <array>
...
      <dict>
         <key>name</key>
         <string>one</string>
         <key>begin</key>
         <string>{</string>
         <key>beginCaptures</key>
         <dict>
            <key>0</key>
            <dict>
               <key>name</key>
               <string>begin</string>
            </dict>
         </dict>
         <key>end</key>
         <string>}</string>
         <key>endCaptures</key>
         <dict>
            <key>0</key>
            <dict>
               <key>name</key>
               <string>end</string>
            </dict>
         </dict>
         <key>patterns</key>
         <array>
            <dict>
               <key>include</key>
               <string>#function</string>
            </dict>
            <dict>
               <key>include</key>
               <string>#string</string>
            </dict>
            <dict>
               <key>include</key>
               <string>#nested_braces</string>
            </dict>
         </array>
      </dict>
   </array>
to this:
Code: Select all
   /* Syntax Patterns
    * ================================== */
   <key>patterns</key>
   <array>
...
      <dict>
         <key>include</key>
         <string>#nested_braces</string>
      </dict>
   </array>
?

I'm thinking since it's already in the repository and it's already doing the recursion there I should just be able to use it right?
askeeve
 
Posts: 51
Joined: Wed Apr 18, 2012 1:33 pm

Re: Oniguruma syntax definition help.

Postby askeeve on Wed May 16, 2012 4:00 pm

Also, (sorry for the multi-posting) I see that you're matching the ;-:Code bit before the nested braces. I'm confused about how the order of evaluation plays out here. Wouldn't the ; match to something inside the braces (if it existed) because the braces match hasn't happened yet?
askeeve
 
Posts: 51
Joined: Wed Apr 18, 2012 1:33 pm

PreviousNext

Return to Technical Support

Who is online

Users browsing this forum: Exabot [Bot], Yahoo [Bot] and 11 guests