Sublime Forum

Syntax colour for ST API

#15

@facelessuser. Okay. Go play with the kids and their ‘gamestations’ :smiley:

When you get back… I’m sooo… close to what I want with this expression:

<string>(?:\.)?(size|substr|begin_edit|end_edit|insert|erase|replace|sel|line|full_line|lines|split_by_newlines|word|find|find_all| rowcol|text_point)\s*(?=\()</string>

It works if I follow the dot with a space ‘. begin_edit()’ but not (currently) without the space?!

0 Likes

#16

No worries. I needed a more complex expression - following your lead… That is, I looked through some of your additional regex in .tmLanguage.

<string>(?:[a-zA-Z_][a-zA-Z0-9_]*\.)?(size|substr|begin_edit|end_edit|insert|erase|replace|sel|line|full_line|lines|split_by_newlines| word|find|find_all|rowcol|text_point)\s*(?=\()</string>

This is in place of the keyword ‘global’ which is very near the top of the tmLanguage file. I should then be able to modify the theme colour (that refers to the scope for ‘global’), and if I want to format the word ‘global’ I can* bung it in* with something appropriate.

I might go for something yellow-ish or orange :smile:

0 Likes

#17

Using the my Python modifications that I pointed you to earlier (it needs the other changes I made in the github version to work), I inserted a ST2 API check before checking for normal functions.


And I defined the API section by copying the function section and placing it under the Repository part in the XML with the same name you see above. Don’t replace the original function section, but make sure to add it before. You still want to make sure non API calls get scoped as functions.

<key>st2_api</key> <dict> <key>begin</key> <string>(?:\.)?(size|substr|begin_edit|end_edit|insert|erase|replace|sel|line|full_line|lines|split_by_newlines|word|find|find_all|rowcol|text_point)\s*(?=\()</string> <key>beginCaptures</key> <dict> <key>1</key> <dict> <key>name</key> <string>meta.function-call.st2-api.python</string> </dict> </dict> <key>end</key> <string>(\))</string> <key>endCaptures</key> <dict> <key>1</key> <dict> <key>name</key> <string>punctuation.definition.arguments.end.python</string> </dict> </dict> <key>name</key> <string>meta.function-call.python</string> <key>patterns</key> <array> <dict> <key>begin</key> <string>(?=[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*\s*\()</string> <key>end</key> <string>(?=\s*\()</string> <key>patterns</key> <array> <dict> <key>include</key> <string>#dotted_name</string> </dict> </array> </dict> <dict> <key>begin</key> <string>(\()</string> <key>beginCaptures</key> <dict> <key>1</key> <dict> <key>name</key> <string>punctuation.definition.arguments.begin.python</string> </dict> </dict> <key>contentName</key> <string>meta.function-call.arguments.python</string> <key>end</key> <string>(?=\))</string> <key>patterns</key> <array> <dict> <key>include</key> <string>#keyword_arguments</string> </dict> <dict> <key>include</key> <string>$self</string> </dict> </array> </dict> </array> </dict>

I attached the modified python language. Diff it against the version I pointed to on github if you need to.
Python.tmLanguage.zip (6.07 KB)

0 Likes

#18

@facelessuser. Thank you, I shall study at length :smile:

I’m guessing that your (?:.)? worked because it was registered within a beginCapture.

I thought my task was complete, but not of the theme files contain ‘storage.modifier.global.python’, or even the word ‘global’ at all. Am I able to just create a new dict entry, following an existing structure:

<dict> <key>name</key> <string>Keyword</string> <key>scope</key> <string>keyword</string> <key>settings</key> <dict> <key>fontStyle</key> <string></string> <key>foreground</key> <string>#FF3854</string> </dict> </dict>
and making up a name (Global?)? I’m optimistic that I can.

I’m not so concerned about other function calls at the moment - at least everything will not be the same colour :wink:

0 Likes

#19

There are a couple of advantages to what my version does over yours. There were a number of changes to make sure self and cls still get scoped proper in lines like this:

self.object.api_call()

self and cls don’t get colored in the theme, but they are getting scoped, and my requirement was not to break any other scoping. The one thing I did change was that if you had this:

self.object.another_object.function()

object.another_object.function() would all get scoped as meta.function-call. I found this to be overzealous, so I reduced it to only scope function() as meta.function-call. After that, all I did was add generic to the function call so that all was pretty much the same meta.function-call.generic, but you add an extra specifier you could target it in your theme, but you don’t have to and it would then be unhighlighted and benign.

By inserting the api check right before the normal function check, you get two kinds of function calls to target (keep in mind, built in functions are not touched by these, so they still get scoped like normal; I think I might have made some adjustments to ensure this).

meta.function-call.generic.python and meta.function-call.st2-api.python

I highlight like this if all I want to highlight is the api:

<dict> <key>name</key> <string>Function Call</string> <key>scope</key> <string>meta.function-call.st2-api</string> <key>settings</key> <dict> <key>fontStyle</key> <string></string> <key>foreground</key> <string>#A6E22E</string> </dict> </dict>

0 Likes

#20

Cool, so I can copy a dictionary entry, use my scope and make up a name - and colour.

I’m not concerned about any other functions, variables, etc., at the moment - I just want(ed) the code a little less monochrome. I’m still learning the API at the moment, so I was particularly interested in these methods.

The ST API documentation is quite limited, and I’m sure it’s only mentioning a fraction of what’s available. It doesn’t mention on_query_completions, extract_completions, match_selector, or describe arguments in enough detail. So it’s been mainly a question of trawling through, and deciphering, any code samples I can find.

I’m quite keen to pursue ‘on_query_completions’ and might build on this, together with a completions (or other source) file. Python and/or, more specifically, the API would be a sensible choice for this (as I can learn the language at the same time :smile: ). Besides, CodeIntel seems to have fallen by the wayside. *

I’ll modify the theme file tomorrow… and possibly come back if I can’t get a nice orange effect :laughing:. Andy.

Off-topic: What’s a straight-forward way (with the API) to position the cursor? I know I could use run_command, but…*

0 Likes

#21

Glad I could help. I just happened to have done this stuff about a week or two ago. Had I not, I would have been pretty clueless.

What do you mean position the cursor…move the cursor to postion x on line y?

This is assuming you know how to calculate your new position and create a region.
You usually need to account for multiselect so you would clear the current selections and then add each new selection region:

view.sel().clear() map(lambda x: view.sel().add(x), new_sel_array)

or for one:

view.sel().clear() view.sel().add(new_sel)

Basically you can parse the view.sel keeping the regions you want or just delete them all without checking and add the new one/or ones you want.

0 Likes

#22

Thank you @facelessuser

So something like:

sel = view.sel()[0]
view.sel().clear()
view.sel().add(sel)

would clear all selections and re-instate the cursor to the latest position :wink:

0 Likes

#23

Well this works, thank you @facelessuser. Although I’m not entirely happy with my choice of colour (a sort of lilac…) but at list I achieved my aim :smile:

I’m not sure I want to pursue this any further just at the moment. I can imagine spending hours (days…) on this :astonished:. I would learn a little more about regex, ST themes, and have control over colours I suppose.

Regards, Andy.

Edited: I also modified the scope from ‘…global.python’ to ‘storage.type.function.api.python’ so that it should behave better, and chucked the word ‘global’ in with def|lambda.

0 Likes

#24

@facelessuser. Sorry to bore :smile: but I’ve got *control *now without too much effort, whe’hay! I copied my new category/dictionary entry ‘storage.type.function.api.python’ to create ‘storage.type.function.general.python’, and added a few standard, and commonly used library, functions to it - see the yellow text in the screenshot. These two categories are near the front of the tmLanguage file, so conflict seems unlikely.

I was under the impression that I would need to study the whole language file to be able to slip these in :laughing:. So now I can add functions as I go along, and create more categories if I so chose, whe’hay! Job done.

I might try and tackle objects like window, view… This might prove a little trickier though, as I haven’t got an opening brace ‘(’ to latch on to :question:

Regards, Andy.

0 Likes

#25

Window and view should be targetable. In the method I am using, I catch all of the object members (even the ones without (); I just don’t scope them with anything; like I said I changed it a lot of things to help me out in the future).

But yeah, there are always more ways than one to do things.

0 Likes

#26

I see now what you were indicating with self, cls :frowning: . In a line such as

self.window.focus_view(OrderedFilesCommand.file_views[index][1])

focus_view is my chosen colour, but self and index are no longer the colour they should be. My expression

<string>(?:[a-zA-Z_][a-zA-Z0-9_]*\.)*(append|lower|sort|split|itemgetter)\s*(?=\()</string>

is not sufficient to discount everything else on the line.

Can you elaborate on “I just don’t scope them with anything” please? This should help me…

**

0 Likes

#27

[quote=“agibsonsw”]I see now what you were indicating with self, cls :frowning: . In a line such as

self.window.focus_view(OrderedFilesCommand.file_views[index][1])

focus_view is my chosen colour, but self and index are no longer the colour they should be. My expression

<string>(?:[a-zA-Z_][a-zA-Z0-9_]*\.)*(append|lower|sort|split|itemgetter)\s*(?=\()</string>

is not sufficient to discount everything else on the line.

Can you elaborate on “I just don’t scope them with anything” please? This should help me…
[/quote]

This is what I was trying to say earlier. I already accounted for all of the stuff, so I was suggesting that you leverage the work I already did.
Basically in my version I don’t do this (?:[a-zA-Z_][a-zA-Z0-9_].). This will greedily include any parent object that happens to come before your target. That is the way it was doing things before I touched it. If an object had a function in it, all preceding object members got pulled into the scope which I though overzealous. So what I did is change the default function check to not greedily include those members, and then I catch them with this and give them no additional scope (I should probably check for lists here as well):

<key>generic_object_names</key> <dict> <key>match</key> <string>(\.\b([A-Za-z_][A-Za-z0-9_]*)\b(?!\()|\b([A-Za-z_][A-Za-z0-9_]*)\b\.)</string> </dict>

So you can see here I look for built in things first, and if I don’t find those, I throw out all the dot members that aren’t function calls etc, then I check for API and then functions, and then lists etc.:

<dict> <key>include</key> <string>#builtin_functions</string> </dict> <dict> <key>include</key> <string>#builtin_types</string> </dict> <dict> <key>include</key> <string>#magic_function_names</string> </dict> <dict> <key>include</key> <string>#language_variables</string> </dict> <dict> <key>include</key> <string>#generic_object_names</string> </dict> <dict> <key>include</key> <string>#st2_api</string> </dict> <dict> <key>begin</key> <string>(?:\.)?([a-zA-Z_][a-zA-Z_0-9]*)\s*(?=\()</string> <key>beginCaptures</key>

I think I also made some minor adjustments to ensure that scope and built in globals get highlighted correctly too, but I would have to diff my changes to remember what I did.

[quote=“agibsonsw”]**
Not so silly now eh? :smile:[/quote]

0 Likes

#28

@facelessuser. What seems to be relatively straight-forward is getting complicated :wink:. Part of my confusion was/is that even though I use a non-capturing group (?: ), it seems that the more general term ‘function’ later on collects this text.

It also seems odd that only the word ‘self’ gets caught out, although I think ‘key’ has also lost its colour, and (possibly) index.

So changing the key ‘captures’ to ‘match’ is not likely to help me? And you wouldn’t recommend that I ditch the name-part ‘function’ all-together? Edited: Ignore this remark about ‘function’ - I realise this wouldn’t help (it IS a function, regardless).

I apologise :wink: . I should be studying all the content you have supplied me with in great depth, but I’m still taking pigeon steps at the mo. I could just “borrow” your language file, but I’m a bit obstinate :smiling_imp:

0 Likes

#29

It is what makes us programmers. Early on with javascript, I wrote my own simple framework (not because I thought I could do better than Jquery; I think Jquery is great and way better) but I just wanted to learn as much as I could about javascript…it was both fun and painful, but I ended up with a pretty usable framework. In the end, I think I learned more about web browsers and javascript than I ever cared to learn.

0 Likes

#30

@facelessuser. Ditto. I wrote my own JavaScript library :laughing:. Admittedly, some methods such as $() and getElementsByClassName() I pretty much copied letter for letter, but most methods I adapted (improved upon…), including creating my own sort procedure and date parser. I wanted to look at jQuery, but I didn’t fancy reading another book. So I went on to the jQuery forum and answered tons of questions (before I’d even typed a line of jQuery code :astonished: ).

Anyway, I think I’ve short-circuited my labours. I realised that the function greediness is how the language file behaves. I wasn’t looking to correct this behaviour, merely to add a number of specific methods/terms to the language. And it’s mainly the term ‘self’ that was an issue (the word ‘key’ isn’t even in the file!).

For the moment I’ve added the terms self and cls in with global, very near the top of the file. The screenshot shows that self, and my api calls, are colour-coded :smile:. If the function scope gobbles up other terms then that’s SEP! [Actually, placing ‘self’ near the top of the file may prevent ‘function’ from gobbling up text anyway…]

All I need to do now (hopefully) is decide what words/methods/additional categories I want and flatten out the colours a bit.

Regards, and thanks again, Andy.


0 Likes

#31

… in fact, I might just make *everything * orange and be done with it :laughing: :wink: - it will remind me of a simpler time. Andy.

0 Likes

#32

:smile:

I have to admit, I understand what you mean. Digging around in these tmLanguage files isn’t that fun…with all of the regex, and ensuring the proper order of definitions, and the recursion…

0 Likes

#33

I realise that I’m opening myself up to mockery, but I welcome criticism of the attached screenshot :smile:

My view (excuse the pun…) on this topic is:

Every character should be legible;
There shouldn’t be too harsh a contrast between any of the colours;
Important *marker *words, like ‘class’ and ‘def’, should be more pronounced;
Comments should seep into the background, but remain legible.

Andy.


0 Likes

#34

:open_mouth:

Not my taste, but it looks like you are getting the hang of the tmLanguage files. The important thing is that it works for you :smile: .

0 Likes