Home Download Buy Blog Forum Support

Specializing C++ grammar for custom patterns

Specializing C++ grammar for custom patterns

Postby coreload on Wed Jan 23, 2013 8:03 pm

I am doing unit testing in C++ using a some simple macros that I've written myself. I would like create a new grammar which specializes the default C++ grammar to add scopes for my test macros. The reason is that I want to provide different syntax highlighting for the test macros which emphasizes the content of the assertion rather than the "ASSERT_EQUALS" text (for example I might dull the color of the "ASSERT_EQUALS" text or mark it with a special color).

The test files follow a simple shape:

Code: Select all
// Example.test.cpp
// Other C++ code
TEST(MyTestFunction) {
    // Other C++ code
    ASSERT_EQUALS(1,2); // will fail
}

Where the macro "TEST" expands to function definition (among other things), and the macro ASSERT_EQUALS is self explanatory.

The issue I'm having is that I can't extend the C/C++ grammar definition for a function block to override the meaning of ASSERT_EQUALS in the context of TEST methods but still support all the normal C++ syntax highlighting for everything else in the test method. Is there a way to do this?

Here is what I have tried so far (which hopefully illustrates what I'm trying to do). It works for TEST and ASSERT_EQUALS and for normal C++ syntax outside of TEST functions, but does not work for normal C++ syntax inside TEST functions.
Code: Select all
{
   "name": "SimpleTesting test file",
   "scopeName": "source.c++.tests",
   "fileTypes": ["tests.cpp"],
   "uuid": "bbfa5657-536d-4c19-83cd-2f0213dfe703",
    "patterns": [
        {
            "name": "meta.block.c++.test",
            "begin": "TEST\\s*\\(\\s*[A-Za-z_][A-Za-z0-9_]*\\s*\\)\\s*\\{",
            "beginCaptures": {
                "0": { "name": "meta.function.test.head.c++" }
            },
            "end": "\\}",
            "patterns": [
                {
                    "match": "\\bASSERT[A-Za-z0-9_]*\\s*\\(.*?\\)",
                    "name": "meta.function-call.assert.c++"
                },
                {
                    "comment": "I'm trying to import all the normal c stuff here, but it doesn't work",
                    "include": "meta.function.c"
                },
                {
                    "comment": "I'm trying to import all the normal c stuff here, but it doesn't work",
                    "include": "meta.block.c"
                }
            ]
        },
        {
            "include": "source.c++"
        }
    ]
}
coreload
 
Posts: 5
Joined: Wed Jan 23, 2013 6:36 pm

Re: Specializing C++ grammar for custom patterns

Postby coreload on Thu Jan 24, 2013 5:41 pm

For those who are interested, I solved the problem by copying the C and C++ grammars into the User folder as an override to the existing C and C++ grammars. I then made direct modifications to those grammars to support the special syntax I have in my testing framework. For example I wrote a pattern to capture ASSERT_EQUALS and inserted it directly above the existing C pattern for scope "meta.function-call.c" so it takes higher precedence (and called it "meta.function-call.assert.c").

And of course I copied my favorite color scheme into the user folder and made the corresponding modifications to highlight the new syntax.
coreload
 
Posts: 5
Joined: Wed Jan 23, 2013 6:36 pm

Re: Specializing C++ grammar for custom patterns

Postby FichteFoll on Thu Jan 24, 2013 10:12 pm

This is apparently the way to do things like this. Except that precedence isn't based on the items' positions but on the positions of their matches. Hard to explain but if you for instance define a syntax with a comment "//" and a string '".*?"' the syntax coloring won't be affected by which item would be first in the list.
FichteFoll
 
Posts: 388
Joined: Fri Mar 16, 2012 11:49 pm
Location: Germany

Re: Specializing C++ grammar for custom patterns

Postby coreload on Fri Jan 25, 2013 7:23 pm

I'm not satisfied with this method at all. It means I'm copy-and-pasting somebody else's work into my files. Apart from the licensing and versioning issues this might cause (if the original file gets updated it won't update the copy-and-pasted code), it is also just a pain and an impediment to modular development. It would be much better if there was a way to augment existing packages with small modifications, in the same way the default settings can be overridden with user settings.
coreload
 
Posts: 5
Joined: Wed Jan 23, 2013 6:36 pm

Re: Specializing C++ grammar for custom patterns

Postby FichteFoll on Sat Jan 26, 2013 12:54 am

I can see a way of doing this by implementing some "new filetype" which allows deleting some entries and adding some others. However, implementing this would be fairly complex because of how language definitions work and you would either have to use a numerical index for the entry to remove or specify a match and a name parameter to somewhat accurately find the correct entry in the original file and delete it. (Of course, this is using/writing a plugin.)

And btw, including "random" scopes in a language definition does not work. You can only specify "#repository-items" which you define locally in the repository root key or the "global" scopeName of other files which is "source.c++.tests" in your case; "include": "meta.function.c" does not work.
FichteFoll
 
Posts: 388
Joined: Fri Mar 16, 2012 11:49 pm
Location: Germany

Re: Specializing C++ grammar for custom patterns

Postby coreload on Tue Jan 29, 2013 6:05 pm

@FichteFoll

I understand why including "random" scopes doesn't work. I tried it because including "source.c++" worked to include the existing C++ grammar, identified by the scope. So I was hoping that since the whole grammar is associated with a scope that the individual grammar patterns would also be.

Can you elaborate on your idea of a "new filetype"? Would this be a pluggin that generates a new grammar file based on an old one (for example a built-in grammar file) and a "patch" file of insertions and deletions? Or do you know some way of generating this information on the fly (as Sublime reads the grammar) so that the "new" file need not exist in the file system?

I wouldn't need to actually provide a mechanism for deletion, since sublime already has the pattern prioritization mechanism. I would just be looking for a way to add new patterns to the beginning of a given pattern list. Since patterns have names identifying their scope it would make sense to have a file that can extend existing scopes (which would implicitly "hide" old behavior with the same match). Or a mechanism to include grammar patterns from the old file by naming their scope.
coreload
 
Posts: 5
Joined: Wed Jan 23, 2013 6:36 pm

Re: Specializing C++ grammar for custom patterns

Postby FichteFoll on Wed Jan 30, 2013 5:57 pm

coreload wrote:Can you elaborate on your idea of a "new filetype"? Would this be a pluggin that generates a new grammar file based on an old one (for example a built-in grammar file) and a "patch" file of insertions and deletions? Or do you know some way of generating this information on the fly (as Sublime reads the grammar) so that the "new" file need not exist in the file system?


Yes, mostly. I like the patch approach, such as you define a diff file or something and the plugin automatically applies this patch to the default file (e.g. using a build system or hooking into "on_post_save") while backing up the default file.
FichteFoll
 
Posts: 388
Joined: Fri Mar 16, 2012 11:49 pm
Location: Germany

Re: Specializing C++ grammar for custom patterns

Postby fjl on Thu Jan 31, 2013 12:28 pm

TextMate 2 solved this problem using injection grammars, which allow you to define overlays of all sorts.
fjl
 
Posts: 36
Joined: Wed Dec 28, 2011 12:41 am


Return to Plugin Development

Who is online

Users browsing this forum: Majestic-12 [Bot] and 8 guests