Home Download Buy Blog Forum Support

Python Improved - a better Python language definition

Re: Python Improved - a better Python language definition

Postby quodlibet on Tue Nov 05, 2013 6:00 pm

@facelessuser If you were to do this, why not _add it_ to PlistJsonConverter? I mean apart from the name absurdity?

None of my business, how you run your plugin empire, of course ;)
quodlibet
 
Posts: 498
Joined: Sat Jun 18, 2011 5:53 pm

Re: Python Improved - a better Python language definition

Postby facelessuser on Tue Nov 05, 2013 6:36 pm

quodlibet wrote:@facelessuser If you were to do this, why not _add it_ to PlistJsonConverter? I mean apart from the name absurdity?

None of my business, how you run your plugin empire, of course ;)

:) The thought occurred to me, and it might actually be done that way in the end.

I don't think it will be too tough to do. The whole thing is built around PyYaml. It seems pretty easy to get running, and they provide a python 3 and 2 version. There are still some issues I see. Apparently when converting from plist to a python object that can be fed to the yaml converter, plistlib doesn't use straight up dictionaries, but it uses some class derived from dictionary. I will probably have to iterate through the python object and convert them all over to dictionary classes, but this isn't a big deal. I am confident I can get a plugin going, you can see it is on its way (at least in a one way conversion form plist to yaml). I literally threw this together by dropping the pyyaml lib into PlistJsonConverter, and deriving a couple new classes from the the language convert class, but you can see it is starting to take shape:

              begin: ^((["']?)(repository)(\2))(:)
beginCaptures: !!python/object/new:plistlib._InternalDict
dictitems:
'1': !!python/object/new:plistlib._InternalDict
dictitems: {name: string.other.quoted-or-unquoted.yaml-tmlanguage}
'2': !!python/object/new:plistlib._InternalDict
dictitems: {name: punctuation.definition.string.yaml-tmlanguage}
'3': !!python/object/new:plistlib._InternalDict
dictitems: {name: entity.other.repository.yaml-tmlanguage}
'4': !!python/object/new:plistlib._InternalDict
dictitems: {name: punctuation.definition.string.yaml-tmlanguage}
'5': !!python/object/new:plistlib._InternalDict
dictitems: {name: keyword.control.definition.yaml-tmlanguage}
end: ^(?!\s*($|#))(?=\S)
name: meta.repository-block.yaml-tmlanguage
patterns:
- !!python/object/new:plistlib._InternalDict
dictitems: {include: '#comment'}
- !!python/object/new:plistlib._InternalDict
dictitems: {include: '#repository-key'}
- !!python/object/new:plistlib._InternalDict
dictitems: {include: '#patterns'}


I am pretty busy, so don't expect it in the next couple of days...but keep your eyes out.
facelessuser
 
Posts: 1544
Joined: Tue Apr 05, 2011 7:38 pm

Re: Python Improved - a better Python language definition

Postby FichteFoll on Wed Nov 06, 2013 10:28 pm

Well, you can guess I stumbled with the same problem. I fixed it by mapping the internal Plist dict (no idea why it exists in the first place, it does nothing) to the "yaml.mapping" to the default dict-dumping function (https://github.com/SublimeText/AAAPacka ... #L385-L388).

I also have quite some patching going on and did some work on better dumping of syntax definitions the way I prefer them (multiline dicts only when necessary, use single quotes when useful, specific order of keys ...). I even added auto-completion to it. It feels weird to me to read people working on things I already did, and which are open source.


PackageDev doesn't really have much overhead imo, it defines some useful language definitions for sublime-specific files (settings, keymaps ...) and some commands and snippets for these. I'd really like to hear your reasons for avoiding it. If I understand you, I can maybe chat up with guillermoo so that we split the repo into the general utility package it already is and the language def only package.
FichteFoll
 
Posts: 377
Joined: Fri Mar 16, 2012 11:49 pm
Location: Germany

Re: Python Improved - a better Python language definition

Postby facelessuser on Wed Nov 06, 2013 11:41 pm

I have it all working, but I didn't really look dig that deep or look at what AAAPackageDev did. I mainly looked to see what library was used and ran from there. But I will look into adding your patch which is a bit cleaner...mine was just a quick a dirty fix to keep moving.

The PackageDev yaml syntax is very nice for sublime stuff, but still has some things to be hammered out. None of these things are complaints really, just noted stuff and opinions. First off, you have done amazing considering how much of a pain it is to parse YAML...parsing YAML with regex sucks. I don't think there can ever be a perfect YAML highlighter in sublime just because regex alone just doesn't cut it for YAML. Anyways, PackageDev is way too strict on keys (but I get what you are trying to do); repository keys all get invalid highlighting. Oniguruma regex syntax seems to cause highlight failures on occasion; falling back to a simpler regex highlighter gave overall better results. Obviously there are more issues than that, but like I said, YAML sucks to highlight. Sublime Text's default YAML is horrible.

I got the plugin all working (needs cleanup), and I threw together a more simple stripped down YAML language file which is better than sublimes, but not as sublime specific or as exhaustive as yours. But as a compromise for YAML in general, I saw the ajax.org ace project had some decent rules, so I grabbed those and have been building on that...oh and it has its own issues as well though :). I think if I can just get the comments to be little better, I will be happy enough for now.

I will be opening a beta thread soon so I can leave this thread alone and stop getting it off topic :).

YAML generated from PLIST:
fileTypes:
- yaml
name: YAML (simple)
patterns:
- match: \t+
name: invalid.illegal.tab.yaml
- match: '#.*$'
name: comment.yaml
- captures:
'1':
name: markup.list.yaml
match: ^(-{3}|\.{3})\s*(?=#|$)
- captures:
'1':
name: markup.list.yaml
match: ^\s*([\-?])(?:$|\s)
- match: '!![\w//]+'
name: constant.yaml
- match: '[&\*][a-zA-Z0-9-_]+'
name: constant.language.yaml
- captures:
'1':
name: meta.tag.yaml
match: ^\s*(\w.*?)(\:(?:\s+|$))
- captures:
'1':
name: meta.tag.yaml
match: (\w+?)(\s*\:(?:\s+|$))
- captures:
'1':
name: meta.tag.yaml
'2':
name: string.yaml
match: '\s*(["]((?:(?:\\.)|(?:[^"\\]|\\"))*?)["])(\:(?:\s+|$))'
- captures:
'1':
name: meta.tag.yaml
'2':
name: string.yaml
match: '\s*(['']((?:(?:\\.)|(?:[^''\\]|''{2}))*?)[''])(\:(?:\s+|$))'
- match: <<\w*:\w*
name: keyword.operator.yaml
- match: -\s*(?=[\{])
name: keyword.operator.yaml
- match: '["](?:(?:\\.)|(?:[^"\\]|\\"))*?["]$'
facelessuser
 
Posts: 1544
Joined: Tue Apr 05, 2011 7:38 pm

Re: Python Improved - a better Python language definition

Postby FichteFoll on Thu Nov 07, 2013 1:04 am

facelessuser wrote:The PackageDev yaml syntax is very nice for sublime stuff, but still has some things to be hammered out. None of these things are complaints really, just noted stuff and opinions. First off, you have done amazing considering how much of a pain it is to parse YAML...parsing YAML with regex sucks. I don't think there can ever be a perfect YAML highlighter in sublime just because regex alone just doesn't cut it for YAML. Anyways, PackageDev is way too strict on keys (but I get what you are trying to do); repository keys all get invalid highlighting. Oniguruma regex syntax seems to cause highlight failures on occasion; falling back to a simpler regex highlighter gave overall better results. Obviously there are more issues than that, but like I said, YAML sucks to highlight. Sublime Text's default YAML is horrible.


I beg you, please report any issues you have with the PackageDev YAML language (preferrably on the repo's issues page) with some sample code. I built the syntax definition completely on my own, from my own examples. Since I whitelisted the keys (in order to highlight rules in a certain way) I just used what I got (I also forced an indent of at least 2 spaces, which is recommended anyway). Whenever a definition broke highlighting, e.g. when I converted a different language into YAML, I adjusted it, but I can't fix something I don't know about or that never happened to me. Especially the oniruguma regex thing is interesting because this should work rather well.

If github supported some kind of messaging system, that would be cool. We could also use PMs on this forum, but I fear that I might not notice an unread PM. I think continuing this on issue(s) is an overall better idea than to keep polluting this thread with OT. If everything fails, you are free to send me an email (to my github's address).
FichteFoll
 
Posts: 377
Joined: Fri Mar 16, 2012 11:49 pm
Location: Germany

Re: Python Improved - a better Python language definition

Postby facelessuser on Thu Nov 07, 2013 3:40 am

FichteFoll wrote:I beg you, please report any issues you have with the PackageDev YAML language (preferrably on the repo's issues page) with some sample code. I built the syntax definition completely on my own, from my own examples. Since I whitelisted the keys (in order to highlight rules in a certain way) I just used what I got (I also forced an indent of at least 2 spaces, which is recommended anyway). Whenever a definition broke highlighting, e.g. when I converted a different language into YAML, I adjusted it, but I can't fix something I don't know about or that never happened to me. Especially the oniruguma regex thing is interesting because this should work rather well.

Yeah, I will post some issues sometimes soon.
facelessuser
 
Posts: 1544
Joined: Tue Apr 05, 2011 7:38 pm

Previous

Return to Plugin Announcements

Who is online

Users browsing this forum: No registered users and 9 guests