Thursday, April 5, 2007

Ruby Screenshot of the Week #8: Live Code Templates and TextMate Snippets

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

I've received several requests for support for TextMate-style snippets. TextMate is a very popular editor among Ruby developers, and a lot of its power comes from its code snippet functionality. Snippets are basically identical to what NetBeans calls "Live Code Templates". The code templates are live in the sense that when you insert the template, you're in a mode where you can jump from one logical section of the template to another, and edits in one section can automatically be updated in other sections.

I've just implemented an import mechanism for TextMate bundles. Point at your own bundles (or even the ones distributed with TextMate, although I don't know what the legal issues are with that), and choose import.
This will convert all* the snippets into NetBeans Live Code Templates (*=for some values of all).

There are two features in TextMate snippets that are not supported:

  • Command execution. A snippet can contain a `shell command` (in backquotes), and at template evaluation time, the command will be executed and the result inserted. In the snippet bundles I looked at, this was used only for a single thing: inserting parentheses on a conditional basis. So I just made the import handle parentheses directly instead.

  • Regular expression replacement. A template can perform regular expression replacement on different instances of the parameters. I can't even tell exactly how this is supposed to work, but for now, I skip snippets that do this.

TextMate is essentially a text editor; it doesn't (as far as I can tell, having never used it but having read their snippets and macros in detail) have any real semantic information about the code. Therefore, doing complicated text processing with regular expressions is the Way of TextMate to achieve logical code operations. Live Code Templates on the other hand can take advantage of all the information the IDE has about the code.
I've added a bunch of logical code template variables you can use in templates. Rather than try to reproduce the regular expression operations
of TextMate's snippets engine, I would like to create logical Ruby operations for the code templates and use those instead.

Here are some examples of what is possible with this mechanism. None of the imported snippets use these variables, but the imported snippets are useful in their own right - especially for those of you with muscle memory.

${unusedlocal defaults="i,j,k,x,y,foo,bar"}

This will evaluate to an unused local variable in the current scope. It will try to use the provided suggested names, in order. It checks whether each one is already in scope, and if so, tries the next candidate. (If none succeed, it starts appending numbers to find a unique name. If there are more Rubyish ways of generating unique variables, please let me know.)

First, we create the template, naming it "dob":

Then, in some method I want a do-block. I just type "dob" and hit Tab, and voila:

It inserts a do-end block. When it comes to the block variable, it has looked at the context and discovered that the first two preferred alternatives, i and j, are both taken - so it chooses k. This shows the advantage of using semantically aware code templates - you don't accidentally alias an existing variable. This wouldn't be a problem in Java, where it would complain that your new variable already exists in scope, but in Ruby, this is valid (and sometimes, but not usually, what you want.)

As another example, here's a silly template which spits out a hash that defines a bunch of local context:

Let's say I have the following silly file, and type "ctx":

If I now hit tab, I get this:

I will now be able to walk through (with the tab key) each of the variable sections and edit them. For an example like this, it's likely that we don't want these things to be edited by the user - so just add editable=false on each such variable (e.g. ${class editable=false}) and they will be processed but not edited by the user.

One of the things that makes this really powerful, is that it uses a lot of the context information the IDE has about your program. In this example, computing the super class of the current class isn't that hard - and a fancy Ruby shell script executed might be able to pull it off. But if I was simply redefining an existing class, let's say Integer, and there is no local reference to Integer's parent class, the live code template will spit out Numeric. Yes, it looks in the index.

Call for Help

As I've shown, the basic machinery is there now, but what we really need is a set of code templates to be bundled with the IDE. Some will be
simple substitutions or linked variables, but we should be able to come up with some much stronger ones. Please let me know what kinds of
logical parameters you can take advantage of. For example, perhaps a parameter which evaluates to the require statements necessary for
the classes referenced in the file? Or a parameter which looks for database table names in the database? Perhaps some simple transformations which
produce a constant name or a method name (upcasing and downcasing) from some other symbol? There are a lot of possibilities here,
but since I'm not a real Ruby programmer I'm not the best person to dream this stuff up. If you can look at snippets you have written, import them into NetBeans, optionally edit them to add more semantic analysis if appropriate, and contribute them, I'd be very thankful!

Also, I have never used TextMate, so I would appreciate feedback on whether the templates work the way they do in TextMate, or if I misunderstood something from just reading the snippets themselves. Bug reports (preferably via the issue tracker) are much appreciated.

So, where can you get this stuff? I just integrated everything, but the Live Code Template management is not in stock NetBeans; it's in an optional download called "CodeTemplateTools" (in contrib/CodeTemplateTools), written by Sandip Chitale. More info about this module is
here. I have built an NBM you can install directly - just download the module bundle file, then install it via the IDE via the Update Center (in the first panel choose manually downloaded NBM). Note however that you must have the other IDE fixes as well; they should be live on soon but probably won't hit the Update Center for a couple of days. (I'll try to make the slim IDE bundle the tools). As always, see the wiki for installation instructions. Once you have installed the Code template tools, look for "Show Templates..." in the Edit menu. The import UI is still a bit rough - no progress bar etc - I'll worry about that later. There's one more rough spot: the "|" character must be escaped (by typing it twice) because historically, NetBeans used "|" to indicate the cursor position. There is the logical variable ${cursor} for that now, but there's still some issues we need to resolve in this area.


  1. Thanks Tor

    Looking forward to trying this out! Can you confirm where you would like us to put comments noting there is your blog, netbeans ruby wiki, netbeans mailing list, netbeans bug tracker (I've not use this before).

    Which of these areas would you prefer to receive:

    a) feeback on your questions from the blog

    b) bugs we find in the latest release drops

    c) ideas for improvements re ruby/rails support

    Cheers Greg

  2. Very Cool!! Looking forward to playing with it...

  3. I'm having trouble trying to import the textmate rails & ruby bundles I checked out (using svn).

    I go Edit/Show Templates/Import, but then Netbeans is looking for a "Abbreviations" file, for which I can't find a file in the checked out Textmate bundle files that match?

    Can anyone see where I'm going wrong?

    Cheers Greg

  4. One more thing regarding indexing speed: Maybe I spoke to soon. On my year-old mac, indexing takes 6 minutes. On my three years old Ultra 20 running Linux, it takes 8 minutes. I've never seen it take 3 hours, and somebody earlier reported 8 (eight!) hours. So there might be something going on in some Ruby distributions that I'm not handling properly - or perhaps there are some really large gems I don't have stalled (large either in terms of number of files or lines of code). If after this fix you still find indexing to be slow, can you tell me what it seems to be doing? (The status bar should at least list the root of what it's indexing - that should tell me which gem it's in etc.).

  5. It's quite cool now !
    But I really want to know how to set the rails's encoding in Netbeans?
    For instance ,if I want to use UTF-8 as my default encoding,how to do that??

  6. Could you please place a link for up to date builds on the side for handy reference?

  7. Ok I got the link to issue tracker, sorry for that.

  8. Just now I tried the Netbeans M8. In this version, the Java-Project's encoding could be setted in its project properties, while Rails project could not.
    And I have tried the method in "", but it doesn't work well in my system. I used simplified chinese Windows-XP, JDK6.

  9. Hudson build seems to be failing, regardless of success status. Certainly no artifacts are being produced:
    /hudson/workdir/jobs/ruby/workspace/scripting/ruby/build.xml:26: The following error occurred while executing this line:
    /hudson/workdir/jobs/ruby/workspace/nbbuild/templates/projectized.xml:53: Cannot compile against a module: /hudson/workdir/jobs/ruby/workspace/nbbuild/netbeans/ruby1/modules/org-netbeans-api-gsf.jar because of dependency: org.netbeans.api.gsf/1 > 0.20.0

  10. Thanks Tor, I was missing my cluster updates :)

    WRT "ugly problems" you mention in your post, the bug that worst affects me by far happens during rhtml editing. Would this fall under scripting/ruby, or is it handled elsewhere currently? (scripting/www?).

    It seems to be document structure related. For example swapping repeated p tags to li tags, for a better semantic match, I get Assertion errors popping up every keystroke. Portions of rhtml start dropping from the editor view, needing the file to be reloaded in the editor.

  11. Thanks Si. This does fall in the scripting/ruby category. I hacked together the RHTML support pretty quickly a while back and haven't touched it since, but the good news is that another engineer (Marek Fukala) is implementing it properly it in a branch right now and it's going to get a lot better pretty soon. I'll post something here (as a separate blog entry) when it gets integrated.