NetBeans Screenshot of the Week #38: E4X Support

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

As part of the JavaScript 1.7 work,
I also beefed up the E4X support. E4X allows you to embed XML objects
directly in your JavaScript source. Here's some simple E4X JavaScript:



We've had E4X support in NetBeans in both NetBeans 6.1 and 6.5. But now, in NetBeans 7.0, semantic highlighting is E4X aware, such that the source code looks like this instead:



The whole XML document declaration has a faint purple background, element names are blue and document content is bolded.
(The green here, as for all JavaScript code in NetBeans, indicates variables in global scope.)

Another new E4X feature is formatting/pretty printing. If you press Alt-Shift-F to format the above source, the E4X is
reformatted and indented:



In addition, the navigator window shows the logical structure of the XML document as well. Clicking in the source
will expand and select the corresponding item in the navigator, and conversely double clicking in the navigator
warps the source to the corresponding location in the XML declaration:



Finally, there's some primitive support in code completion for the XML document:

NetBeans Screenshot of the Week #37: JavaScript 1.7 Support

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

I just checked in support for JavaScript 1.7. This means that NetBeans will no longer give you syntax errors if you try to use the new language. Here are some screenshots. First, let's open a file using JavaScript 1.7 constructs:



Obviously the file is using yield which is not recognized as a keyword, and in particular, this is just a syntax error - the parser was expecting a semicolon. But there's a little lightbulb... let's see what it says (type Alt-Enter or click on the lightbulb):



Let's select the option to change the active language to 1.7. I could also open the JavaScript options dialog directly and set it to 1.7 as shown here:



After applying the language change, the editor immediately re-lexes and reparses the file according to the new language. The errors are gone, and yield is now a proper keyword:



Similarly, the let keyword can be used, as well as destructuring assignments, generators and iterators, etc. (See New In JavaScript 1.7 for more information).

If you place the caret on function definitions, it will highlight all exit points, and this includes yields now.

This work isn't done; I'd like it to be smarter about generators, and there may be some issues with scope handling. But at least the editor doesn't get in your way with false error messages now - you can start writing JavaScript 1.7 clean code. (This is with NetBeans 7.0 dev).

In related news, I just heard from Tom Enebo that a lot of Ruby 1.9 language features are now supported by the JRuby trunk parser, so hopefully we can soon start fully supporting Ruby 1.9 features in NetBeans as well.

NetBeans Screenshot of the Week #36: Python Code Completion

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

Here are some screenshots of the code completion working for Python.
First, let's try it from an import statement:



Note that the documentation for the module is shown. Next, let's
try completing on imported symbols - again we've got the documentation:



I'm writing a unit test, extending TestCase. Now let's try overriding
some methods - completion after typing def:



Obviously, invoking methods on self. will show the inherited
methods - and their documentation:



Here's completion in my function (not on self) showing
the local variables, imported symbols and builtins. Notice that deprecated
methods are marked with strikethrough (and deprecated modules are in imports
as well, though I didn't show that above).



Notice also that the documentation here looks better - it has links,
colors, etc. This is because NetBeans supports the reStructuredText
Docstring Format (see PEP 287).
System libraries are being documented in this way now, so more of the
documentation will start looking like this.

When NetBeans finds embedded code, it will also syntax highlight the
code fragments in the documentation:



You can use this in your own documentation as well obviously.
Here's a file where I have restructured text. As I'm writing it,
I can -immediately- preview the text myself by just pressing
Ctrl-Shift-Space (Show Documentation - for those who don't know, you
can do this on regular code as well if you want to just get the documentation
for the code at the caret, rather than completion alternatives up to
the caret prefix.)



Here's a final example - completion after raise (or
exception) will show completion for all Error
classes:

I'll be quiet or about a week now -- I'm going on vacation! Not great
timing given the release date, but this has been planned since February...

NetBeans Screenshot of the Week #35: Support for Python's __all__ Variable

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

Let's start with a pretty simple module - an import, a function, and a couple of classes:



(Notice by the way the semantic highlighting here - the unused parameters and variables are marked with a slight grey underline,
method names in bold, parameters in orange. More on that another day.)

Now let's go to a different file and import this module. Code completion is context dependent and shows us
which symbols are being exported by the given module. We see not only our functions and classes, the imported symbol sys
is also being defined by module, meaning that if you do from compl4 import * you would have sys defined as well:



Python has a way to deal with this: You can specify a variable named __all__, which lets you define exactly what the public API for
this module should be (more details).

Let's go in and define the names we want exported by our module. I will add __all__ = [ "Bar " ] somewhere in the file, to specify that only Bar should be exported:



Look at the file outline / navigator on the left hand side. As soon as I typed in the __all__ line,
the icons changed to show me that only the Bar class and its method are now "public"; there are little
lock icons on everything else! Let's see what code completion offers when we try to import this module again:



Yep, only the public API is exported now. Let's append to the __all__ list to export the func function:



You don't have to rely on icons to tell you if something is wrong though. Let's say I've forgotten all about this, and I decided
to rename my Bar class:



NetBeans checks that all the symbols you declare in your __all list actually exist, and points out any discrepancies. Here, the public Bar class no longer exists since we renamed it.

Here's another little bug I came across related to this. I was checking the __all__ handling on various library files,
in this case posixpath.py. Notice how in the navigator, all the functions are public, except one...



Sure enough, the code looks like it's registering this conditionally added method... Do you spot the problem? There's a dot missing!
As soon as we add it, we get this:



Of course, this last bug was already detected by the
unresolved symbol detector...

Ruby Screenshot of the Week #34: Detecting Duplicates Hash Keys

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

You didn't think we had forgotten Ruby, did you? Here's a new Ruby editor feature: Detecting duplicate hashkeys. As "Fjan" (who filed the
request) says, when some Rails calls allow dozens or more keys, you can accidentally add a hash key twice, with subtle results. NetBeans now tracks these down and points them out for you.

This is going into 7.0, but you can use this 6.5-compatible module as well.

Martin and Erno have also been hard at work. There are many new features in 6.5 (which I'll cover when 6.5 ships in a week or two). But in the 7.0 trunk, the debugger can attach to arbitrary and remote processes - see
Martin's blog entry for more info. And Erno has added support for Shoulda testing and more test runner improvements - details here.

NetBeans Screenshot of the Week #33: Finding Unresolved Symbols

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

In my last entry I showed that the Fix Imports action will alert you to an unresolved symbol. This is really helpful since it can help you track down runtime problems.

Obviously, you shouldn't have to go run Fix Imports on every file, periodically, to look for unresolved symbols!

NetBeans has a special semantic checker for this. It is off by default, for two reasons:

1. I added it just a couple of days ago - during the stabilization period for EA, so I didn't dare enable it in case it's seriously broken.
   
2. There are cases where it doesn't yet work. I'll describe that later.
   

To enable it, open up the Editor options dialog, and under Hints, select Python. You should now see the current batch of Python hints. Enable the "Unresolved Symbol" hint:



Now let's open up the datetime.py file again:



As you can see, the file is marked erroneous (by the red square above the scrollbar), and the scrollbar indicates where in the file the error is. Clicking on it to expose the error location you can see an underline under the unresolved symbol, and a tooltip explaining what the problem is. The error is also shown in the tasklist, so you don't have to open each file to look for problems - just let the tasklist scan your project for problems.

Here's another example. I opened up the Django 1.0 sources, and NetBeans pointed out this problem to me, in django.utils.simplejson.tool:



This shows that the import is unused, and that the attempts to call methods on the import do not work. The call would need to be changed to

obj = django.utils.simplejson.load(infile)

or the import changed to be

import django.utils.simplejson as simplejson

Here's one final one... Perhaps helpful to people having done a lot of Java development recently:

Finding unresolved symbols means that NetBeans must be able to find all the libraries you are importing. This means you can't just point NetBeans at a plain Python file - your project has to contain all the libraries you are using, and, you have to configure your source path correctly such that NetBeans computes the correct package names for your python modules. (I accidentally pointed to the "django" folder instead of its parent folder at first, which meant django. wasn't part of the package prefix and that meant that all kinds of symbols couldn't be resolved.)

Second, this won't work for dynamically generated symbols (eval etc). I haven't run into this a lot yet - but I'm sure there are areas where the unresolved detection can't find symbols that really do exist.

P.S. Is there a canonical Python term for what I've called "unresolved" here that we should use instead?

NetBeans Screenshot of the Week #32: Python Import Management

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

We're about to release an early access release of Python support for NetBeans. We have ambitious goals for the Python support: NetBeans should be the best IDE for Python development, bar none!

Over the next couple of weeks I'll post screenshots of a bunch of editor related features for Python. Today I'll start with a feature near and dear to my heart: Import Management.

I used to be a die hard XEmacs user. When I started coding in Java, I continued using emacs for a couple of years. And the feature which finally made me abandon emacs for a Java IDE? You guessed it -- import management. If I wanted to use for example the List class, I could just press a keystroke to choose which List class I had in mind, and the import statement was added for it at the top of the file while I could continue working in the middle of the file somewhere.

To be sure, this is not rocket science, and Emacs support for Java has improved a lot over the years. But at the time, this was a huge productivity boost and coupled with other code-intelligence and navigation features was enough to lure my away from my comfort zone in the IDE to what was a less powerful plain text editor, but a more powerful Java coding tool.

Anyway, Python shares some similarities with Java. There are lots of libraries, and you need to import modules before you can use them. And this is where NetBeans' Python support can help.

Let's say I've tried calling the cmp method to compare files. I ran the code and realized I need to import cmp before I can use it. To get the IDE to import the symbol for me, I can just pull up the context menu and select Fast Import (or more realistically, just use the keybinding Alt-Shift-I):



This brings up an import dialog like this:



This tells us there are two definitions of a function named cmp - one in filecmp and one in functions. I'm interested in the first one. The label below the list tells me that I can hold the Alt key or the Shift key when I select the list entry to change the import statement. And below that, there is a preview of the import statement that will be generate. As soon as I press the Alt key, the window changes to this:



Here, the bottom line is showing that we will be doing a wildcard import of all symbols instead of just the target symbol. And finally, let's press Shift:



This will not just import the cmp symbol, it will instead import the whole module and change my current call to use the imported module symbol. When I press Enter my editor looks like this:

This is the import facility I ditched Emacs for ten years ago. But it's not how I manage imports for Java anymore. Instead, there is the Fix Imports action (it's also there in the menu -- Ctrl-Shift-I for short). Unlike Fast Import, which looks at the identifier under the editor caret, Fix Imports will look at all your imports in the file, and, well, fix them! This means sorting them, removing duplicates, even removing unused imports. And obviously, resolving imports automatically using the machinery I showed above. Here's how this works.

Here I've opened the SocketServer.py file, and then I've commented out all the imports:



The file is now broken - it has various references to functions that are not defined/imported anymore. Let's see what happens if I press Ctrl-Shift-I:



Tadaaa! It has figured out the missing imports and has inserted them. This is confirmed by a message in the lower left hand corner:



This just happened automatically because NetBeans could figure out, unambiguously, what the missing imports must be. Let's make it less clear-cut. I'm going to insert a call to get(). Now let's try again:



This time we get a dialog. You can see that it has figured out the sys and os imports - but it is not certain to do about the call to get(). There are two modules defining this function - unicodedata and webbrowser:



The corresponding function signatures are listed to help me decide which one I'm interested in. I can choose one of them, and when I press Ok, the imports are applied.

In some cases it may not find the symbol at all. Let's put in a bogus call:



Here the import manager can't figure out what to do - there is no known matching function. Only when NetBeans is certain about the imports does it proceed to clean up the imports without any user intervention.

This facility can also help find bugs! When I was testing this feature, I happened to invoke it in the datetime.py file which ships with Jython. Here's what it said:



That's right - the code is referencing nonexistent class ValuError. This clearly should be ValueError!
NetBeans has additional facilities for tracking down these types of bugs which I'll cover in another post.

So far I've focused on using the import facility to import new symbols. If you're writing code using the editor you may not need it, since when you use code completion to insert calls, it also automatically inserts import statements as needed.
However, another useful aspect of Fix Imports is cleaning up your imports: removing duplicates, sorting alphabetically (configurable), and even deleting unused imports.

If you open up a file with unused imports, NetBeans will show you that immediately:



There are quickfixes here, so if you click on the lightbulbs or press Alt-Enter, you can see the import actions:



The integration with quickfixes means that if you look in the tasklist, it will show files containing unused imports. But when you're editing, you can just press Ctrl-Shift-I to clean up the imports - it will do everything we've covered so far: Add missing imports, remove duplicates, remove unused, and sort the remaining imports.

Import management is also tied into source formatting. If you format your source, you can optionally have it clean up formats. Here are the formatting options for Python:



You can play with the checkboxes and the preview will show you the effect. Here's what happens if we enable sorting system libraries to the top. Note also that removing unused imports is not done by default during formatting (whereas Fix Imports will perform it automatically). You can change that in this formatting preference panel:

As I mentioned, we'll have an early access version out very soon. If you're feeling adventurous, you can always download builds from the continuous build server - http://deadlock.netbeans.org/hudson/job/python/. Please file bugs and ideas here.

Champagne!

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

I'm extremely proud and relieved. If it hadn't been for the embarrassing passage of Prop 8, it would have been a perfect ending to the last eight years.