Home Download Buy Blog Forum Support

How to create a new language?

How to create a new language?

Postby SteveCooperOrg on Wed Apr 02, 2008 8:48 am

Jon, can you give me some guidance on creating a new language?

Basically, I want to make a variant of Markdown which has it's own settings and file extension, and later it'll have it's own language file. I've tried creating a package, but it doesn't add anything to the list of file types in the bottom-right corner. Is this master-list of languages extensible?

Steve
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Re: How to create a new language?

Postby jps on Wed Apr 02, 2008 1:00 pm

These days Sublime Text uses TextMate style syntax definitions in *.tmLanguage files. There's an introduction at http://macromates.com/textmate/manual/language_grammars.

You can download existing ones from http://code.google.com/p/tmbundles4win/, the only caveat being that the files must have a .tmLanguage extension. Some already do, others have to be renamed from .plist to .tmLanguage to be recognised.

It's worth noting that sublime uses Apple's byzantine propertly-list-in-xml format, which is equivalent to what you see in the above documentation, but more verbose.

Any syntax definition that has file extensions associated with it appears as a selectable option, so if you make sure you have:

Code: Select all
...
<key>fileTypes</key>
<array>
   <string>some-extension</string>
</array>
...


then it'll be an available choice.

Note that tmLanguage files are _not_ presently auto reloaded, which I realise isn't ideal.

wrt the settings, just make a *.sublime-options file with the same base name as the .tmLanguage file, and it'll get picked up.

Below is a simple example of a definition that marks up *.sublime-options files, which will be included in the next beta:

Code: Select all
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>fileTypes</key>
   <array>
      <string>sublime-options</string>
      <string>sublime-build</string>
   </array>
   <key>name</key>
   <string>Sublime Options</string>
   <key>patterns</key>
   <array>
      <dict>
         <key>captures</key>
         <dict>
            <key>1</key>
            <dict>
               <key>name</key>
               <string>punctuation.definition.comment.sublime-options</string>
            </dict>
         </dict>
         <key>match</key>
         <string>(#).*$\n?</string>
         <key>name</key>
         <string>comment.line.number-sign.sublime-options</string>
      </dict>
      <dict>
         <key>match</key>
         <string>\b(:?[1-9]+[0-9]*|0)\b</string>
         <key>name</key>
         <string>constant.numeric.integer.decimal.sublime-options</string>
      </dict>
      <dict>
         <key>match</key>
         <string>\b(:?true|false)\b</string>
         <key>name</key>
         <string>constant.language.sublime-options</string>
      </dict>
   </array>
   <key>scopeName</key>
   <string>source.sublime-options</string>
</dict>
</plist>


Ctrl+Alt+P is a handy key binding to keep in mind as you're developing a syntax definition, it shows you the name assigned to the character that's just to the right of the cursor.
jps
Site Admin
 
Posts: 3077
Joined: Wed Mar 19, 2008 12:33 pm

Re: How to create a new language?

Postby SteveCooperOrg on Wed Apr 02, 2008 4:15 pm

OK, great! I've got a project I'm working on with a fiction author, and I should be able to develop, basically, a custom fiction-writing IDE for him entirely in Sublime. Sweet.

(BTW, I noticed that the .package-menu files aren't registered as being XML files, so they don't syntax highlight yet.)
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Re: How to create a new language?

Postby SteveCooperOrg on Wed Apr 02, 2008 9:38 pm

I've added this info to the wiki at http://www.sublimetextwiki.com/cgi-bin/ ... wnLanguage
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Re: How to create a new language?

Postby SteveCooperOrg on Wed Apr 02, 2008 10:26 pm

Anyone out there trying this, you'll find yourself restarting sublime A LOT. If you have python installed, you can run this script; it'll create a new instance of Sublime every time you close. Very useful for reloading those .tmLanguage files.

Code: Select all
# endlessly sublime
#
# reloads sublime text over and over and over...
# useful if you need to restart A LOT, for example
# while developing languages

import os, sys, subprocess

sublimePath = "c:\\program files\\sublime text\\sublimetext.exe"

i = 0
while i < 100: # limited just in case it starts spawning forever...
  os.spawnl(os.P_WAIT, sublimePath)
  i = i + 1
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Re: How to create a new language?

Postby SteveCooperOrg on Wed Apr 02, 2008 11:32 pm

Well, that's it -- I've got a package written with a very basic markup language, and it's all highlighing beautifully. Thanks, Jon.
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Re: How to create a new language?

Postby jps on Wed Apr 02, 2008 11:35 pm

Excellent! I'll take a look at getting tmLanguage files auto reloading, it's unduly painful atm.
jps
Site Admin
 
Posts: 3077
Joined: Wed Mar 19, 2008 12:33 pm

Re: How to create a new language?

Postby SteveCooperOrg on Thu Apr 03, 2008 1:58 pm

Heh. You knew I couldn't go ten minutes without requesting an API feature...

So, I'm liking the way that the lexer is giving me syntax highlighting. I've invented a mini-markup language. Neat!

Now, of course, I want to write a program that parses the markup using the same rules as the syntax highlighter.

Ie, if I'm using this syntax for creating emphasis;

here is an *important* message

I'd love to get a hierarchical version of the output, something like;

{
doc =
{
text = "here is an ",
markup.italic.script = "*important*",
text = "message"
}
}

If, by some stroke of luck, such a thing is available in ST, is it possible to get hold of it?
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Re: How to create a new language?

Postby jps on Thu Apr 03, 2008 10:19 pm

There's the API function extractScope, which will return the extents of a given region. However, in terms of extracting an AST of the buffer, it's not great - it doesn't descend into children. e.g, if you give it a point in a string, it'll give you the extents of the string, but it won't notify you of any escaped characters, interpolated code etc contained within the string.

There's no reason the full tree structure of the buffer couldn't be exposed though.

There's also a stand alone TM syntax parser, which may be of interest:

http://ultraviolet.rubyforge.org/
jps
Site Admin
 
Posts: 3077
Joined: Wed Mar 19, 2008 12:33 pm

Re: How to create a new language?

Postby SteveCooperOrg on Fri Apr 04, 2008 8:26 am

Hi, Jon. Cheers. I'm getting the regions at start-and-end tuples. Is there any way to get the text that you'd see if you press ctrl-alt-p? I think with that, I can probably create the AST.

Also, the markup language isn't that complex, so I can recreate the lexer; it's not a biggie.
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Next

Return to Plugin Announcements

Who is online

Users browsing this forum: No registered users and 2 guests