Sublime Forum

Syntax colour for ST API

#23

Well this works, thank you @facelessuser. Although I’m not entirely happy with my choice of colour (a sort of lilac…) but at list I achieved my aim :smile:

I’m not sure I want to pursue this any further just at the moment. I can imagine spending hours (days…) on this :astonished:. I would learn a little more about regex, ST themes, and have control over colours I suppose.

Regards, Andy.

Edited: I also modified the scope from ‘…global.python’ to ‘storage.type.function.api.python’ so that it should behave better, and chucked the word ‘global’ in with def|lambda.

0 Likes

#24

@facelessuser. Sorry to bore :smile: but I’ve got *control *now without too much effort, whe’hay! I copied my new category/dictionary entry ‘storage.type.function.api.python’ to create ‘storage.type.function.general.python’, and added a few standard, and commonly used library, functions to it - see the yellow text in the screenshot. These two categories are near the front of the tmLanguage file, so conflict seems unlikely.

I was under the impression that I would need to study the whole language file to be able to slip these in :laughing:. So now I can add functions as I go along, and create more categories if I so chose, whe’hay! Job done.

I might try and tackle objects like window, view… This might prove a little trickier though, as I haven’t got an opening brace ‘(’ to latch on to :question:

Regards, Andy.

0 Likes

#25

Window and view should be targetable. In the method I am using, I catch all of the object members (even the ones without (); I just don’t scope them with anything; like I said I changed it a lot of things to help me out in the future).

But yeah, there are always more ways than one to do things.

0 Likes

#26

I see now what you were indicating with self, cls :frowning: . In a line such as

self.window.focus_view(OrderedFilesCommand.file_views[index][1])

focus_view is my chosen colour, but self and index are no longer the colour they should be. My expression

<string>(?:[a-zA-Z_][a-zA-Z0-9_]*\.)*(append|lower|sort|split|itemgetter)\s*(?=\()</string>

is not sufficient to discount everything else on the line.

Can you elaborate on “I just don’t scope them with anything” please? This should help me…

**

0 Likes

#27

[quote=“agibsonsw”]I see now what you were indicating with self, cls :frowning: . In a line such as

self.window.focus_view(OrderedFilesCommand.file_views[index][1])

focus_view is my chosen colour, but self and index are no longer the colour they should be. My expression

<string>(?:[a-zA-Z_][a-zA-Z0-9_]*\.)*(append|lower|sort|split|itemgetter)\s*(?=\()</string>

is not sufficient to discount everything else on the line.

Can you elaborate on “I just don’t scope them with anything” please? This should help me…
[/quote]

This is what I was trying to say earlier. I already accounted for all of the stuff, so I was suggesting that you leverage the work I already did.
Basically in my version I don’t do this (?:[a-zA-Z_][a-zA-Z0-9_].). This will greedily include any parent object that happens to come before your target. That is the way it was doing things before I touched it. If an object had a function in it, all preceding object members got pulled into the scope which I though overzealous. So what I did is change the default function check to not greedily include those members, and then I catch them with this and give them no additional scope (I should probably check for lists here as well):

<key>generic_object_names</key> <dict> <key>match</key> <string>(\.\b([A-Za-z_][A-Za-z0-9_]*)\b(?!\()|\b([A-Za-z_][A-Za-z0-9_]*)\b\.)</string> </dict>

So you can see here I look for built in things first, and if I don’t find those, I throw out all the dot members that aren’t function calls etc, then I check for API and then functions, and then lists etc.:

<dict> <key>include</key> <string>#builtin_functions</string> </dict> <dict> <key>include</key> <string>#builtin_types</string> </dict> <dict> <key>include</key> <string>#magic_function_names</string> </dict> <dict> <key>include</key> <string>#language_variables</string> </dict> <dict> <key>include</key> <string>#generic_object_names</string> </dict> <dict> <key>include</key> <string>#st2_api</string> </dict> <dict> <key>begin</key> <string>(?:\.)?([a-zA-Z_][a-zA-Z_0-9]*)\s*(?=\()</string> <key>beginCaptures</key>

I think I also made some minor adjustments to ensure that scope and built in globals get highlighted correctly too, but I would have to diff my changes to remember what I did.

[quote=“agibsonsw”]**
Not so silly now eh? :smile:[/quote]

0 Likes

#28

@facelessuser. What seems to be relatively straight-forward is getting complicated :wink:. Part of my confusion was/is that even though I use a non-capturing group (?: ), it seems that the more general term ‘function’ later on collects this text.

It also seems odd that only the word ‘self’ gets caught out, although I think ‘key’ has also lost its colour, and (possibly) index.

So changing the key ‘captures’ to ‘match’ is not likely to help me? And you wouldn’t recommend that I ditch the name-part ‘function’ all-together? Edited: Ignore this remark about ‘function’ - I realise this wouldn’t help (it IS a function, regardless).

I apologise :wink: . I should be studying all the content you have supplied me with in great depth, but I’m still taking pigeon steps at the mo. I could just “borrow” your language file, but I’m a bit obstinate :smiling_imp:

0 Likes

#29

It is what makes us programmers. Early on with javascript, I wrote my own simple framework (not because I thought I could do better than Jquery; I think Jquery is great and way better) but I just wanted to learn as much as I could about javascript…it was both fun and painful, but I ended up with a pretty usable framework. In the end, I think I learned more about web browsers and javascript than I ever cared to learn.

0 Likes

#30

@facelessuser. Ditto. I wrote my own JavaScript library :laughing:. Admittedly, some methods such as $() and getElementsByClassName() I pretty much copied letter for letter, but most methods I adapted (improved upon…), including creating my own sort procedure and date parser. I wanted to look at jQuery, but I didn’t fancy reading another book. So I went on to the jQuery forum and answered tons of questions (before I’d even typed a line of jQuery code :astonished: ).

Anyway, I think I’ve short-circuited my labours. I realised that the function greediness is how the language file behaves. I wasn’t looking to correct this behaviour, merely to add a number of specific methods/terms to the language. And it’s mainly the term ‘self’ that was an issue (the word ‘key’ isn’t even in the file!).

For the moment I’ve added the terms self and cls in with global, very near the top of the file. The screenshot shows that self, and my api calls, are colour-coded :smile:. If the function scope gobbles up other terms then that’s SEP! [Actually, placing ‘self’ near the top of the file may prevent ‘function’ from gobbling up text anyway…]

All I need to do now (hopefully) is decide what words/methods/additional categories I want and flatten out the colours a bit.

Regards, and thanks again, Andy.


0 Likes

#31

… in fact, I might just make *everything * orange and be done with it :laughing: :wink: - it will remind me of a simpler time. Andy.

0 Likes

#32

:smile:

I have to admit, I understand what you mean. Digging around in these tmLanguage files isn’t that fun…with all of the regex, and ensuring the proper order of definitions, and the recursion…

0 Likes

#33

I realise that I’m opening myself up to mockery, but I welcome criticism of the attached screenshot :smile:

My view (excuse the pun…) on this topic is:

Every character should be legible;
There shouldn’t be too harsh a contrast between any of the colours;
Important *marker *words, like ‘class’ and ‘def’, should be more pronounced;
Comments should seep into the background, but remain legible.

Andy.


0 Likes

#34

:open_mouth:

Not my taste, but it looks like you are getting the hang of the tmLanguage files. The important thing is that it works for you :smile: .

0 Likes

#35

Not my taste, but it looks like you are getting the hang of the tmLanguage files. The important thing is that it works for you :smile: .[/quote]

:laughing: Yes, it is significantly different to your earlier screenshot!

For my further education, could you/someone describe in brief the difference between ‘match’ and ‘captures’ in tmLanguage? And what is the purpose of ‘includes’ please?

Regards, Andy.

0 Likes

#36

The whole file is parsed by the regex definitions. As soon as a chunk of code is processed with a definition, the syntax parser moves on and that chunk of code will not be touched again.

Matches: matches your regex and assigns it your scope to group0 only (if one is provided) and moves on.

Captures: allow you to capture the specific groups and assign scopes to each group and also the entire match if desired. This example from python assigns group 1 a scope, but the entire match (group 0) has no no scope assigned to it.

<dict> <key>captures</key> <dict> <key>1</key> <dict> <key>name</key> <string>storage.modifier.global.python</string> </dict> </dict> <key>match</key> <string>\b(global)\b</string> </dict>

Begin/End Captures: Allow you to give a start capture and and end capture (as I understand, the end capture and reference the begin capture with lookaheads etc. and I think vice versa), but then it also allows you to then provide regex for the content between the Begin and End. You can even do recursion as I understand.

0 Likes

#37

Thanks. ‘includes’ ?

Am I mistaken, or does ST not recognise regex-lookbehinds? I tried and it seemed to fail.

BTW I’m tempted to try and colour braces ( ), but I think I would be in for a world of pain :astonished:

Andy.

0 Likes

#38

Oh yeah, includes. You can usually have a repository section in your tmLanguage and you give the definition key names. Then you can include the definition by its key name. Also, languages included other sources. You can see in things like the HTML language they will include “source.js” to pull in javascript tmLanguage file. It might be more powerful than that, but that is what I understand at this point.

0 Likes

#39

Mmmm lot to learn :wink:

The more I look at ‘Python.tmLanguage’ the messier it appears. There is a large collection of standard functions not listed. Also, a word like ‘set’ is used in a number of contexts, so it’s difficult to distinguish the ST-API version. And datetime is both a module(object) and an object, etc.

I think I’ll try and move on to something else for a while. Regards, Andy :wink:

0 Likes

#40

If you still haven’t worked this out, you need to escape the “less than” sign. E.g., (?<=\W) should read (?<=\W)

Alex

0 Likes

#41

[quote=“quodlibet”]

If you still haven’t worked this out, you need to escape the “less than” sign. E.g., (?<=\W) should read (?<=\W)

Alex[/quote]

Thanks Alex. I caught that :wink:

0 Likes

#42

(Just an update). I believe I’ve finally got this sussed with the code below, placed before ‘meta.function-call.python’ :smiley:

What I wanted to do was scope specific method-names, but only if they are preceded by a dot. ‘meta.function-call’ kept interfering, grabbing any remaining text on the line and, therefore, applying one colour. My previous attempt was working, but the brackets and arguments (following my named-methods) remained as ‘source.python’.

My code captures general methods, but only if preceded by a dot. It colours the method-name, ignoring any preceding text. It also works with nested function/method calls. [Part of the solution is to name the method’s arguments ‘meta.method.call.arguments.python’, but do nothing with them - but perhaps I can colour these as well.]

I copied this code and substituted all the ST-API method calls, and also gave a different name to the brackets ( ) - so now I can colour-code all bracket pairs, whe’hay :smiley:

I believe I should be able to modify ‘meta.function-call’ with code similar to this, so that it won’t misbehave at all.

<dict> <key>begin</key> <string>(?:[a-zA-Z_][a-zA-Z0-9_]*\.)*(?&lt;=\.)([a-zA-z_][a-zA-Z0-9_]*)\s*(\()</string> <key>beginCaptures</key> <dict> <key>1</key> <dict> <key>name</key> <string>meta.method.call.python</string> </dict> <key>2</key> <dict> <key>name</key> <string>punctuation.definition.arguments.begin.python</string> </dict> </dict> <key>end</key> <string>(\))</string> <key>endCaptures</key> <dict> <key>1</key> <dict> <key>name</key> <string>punctuation.definition.arguments.end.python</string> </dict> </dict> <key>contentName</key> <string>meta.method.call.arguments.python</string> <key>patterns</key> <array> <dict> <key>include</key> <string>#keyword_arguments</string> </dict> <dict> <key>include</key> <string>$self</string> </dict> </array> </dict>

0 Likes