Wednesday, December 31, 2008

Happy New Year!

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

My music teacher in middle school had a great tradition. Every New Years Eve, at midnight, he would lift his stereo speakers up to his open windows, and on full blast, play ABBA's "Happy New Year" for the neighborhood.

I'm going to do the same thing here, on my little corner of the internet. Enjoy!

My kids have recently discovered ABBA through the Mamma Mia movie. Now they want to listen
to ABBA when we're driving around. No, don't pity me - it's a lot better than what they usually
want to listen to (I'm looking at you, Hannah Montana!). And besides, having grown up
in Norway I'm an ABBA fan at heart. When I was in 4th grade, all the girls in my class
liked ABBA, all the boys in my class liked Kiss, and then there was me - I sided with
the girls. And I'm still not ashamed of it. Despite some of their ridiculous outfits,
ABBA created some timeless melodies and had great sound production values that hold
up to this day.

Tuesday, December 23, 2008

VirtualBox Rocks!

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

In order to get code coverage to work seamlessly in the presence of the Rake build tool, I had to do some pretty ugly tricks - a lot of path manipulation that surely won't work on Windows. Rather than just getting bug reports, I'll need to debug through it to find the troublespots and fix them.

I don't have a Windows machine. I have a Solaris desktop box, and a Mac laptop. And since I don't have any short term plans to go in to the office (I work from home most of the time) I don't want to wait to look into this until my next chance to stop by a computer lab with Windows machines.

So, I decided to dust off the VirtualBox instance I installed and tested a while back. Back then, it worked fine - but was a bit painful to use. Actually, it was my fault. I never went and "finished" my installation. Turns out, after installing Windows on the virtual machine, there is some additional software shipped with VirtualBox that can be installed into Windows to make things work more smoothly.
For example, a special mouse driver makes the Mac mouse also work as the Windows mouse. There's support to share file systems. This is a huge benefit. I simply mapped my working source tree that I have already checked out, edited and built, as a new Windows drive! I didn't have to go and install Mercurial, Python, check out the whole NeBeans source tree, build it, etc. I could simply open up drive Z:, and run the NetBeans IDE from within Windows!

Another cool thing these special tools add is "seamless mode". I'm running the Windows applications running as if they are regular Mac applications on my Mac desktop! So I can debug through my Windows instance of NetBeans while at the same time having my Mac mail and instant messaging programs part of the screen. Look at the below screenshot (click for full size) -- I have the Mac doc on the left, and the Windows doc on the bottom - as well as a mixture of Mac and Windows (uhm) windows on the desktop!

There's my mailtool again... Last year I posted a screenshot showing how I had 13,000 unread mails, but plans to finally catch up. I guess I didn't put that high enough up on my priority list. The above screenshot shows how I'm up to 56,000 unread mails now. Ugh. Oh well, at least the number of unread mails addressed directly to me is decreasing.

Wow. Installation was flawless. It's working beautifully. And it's free. And it's open source. Wow. Oh, and it runs on many other platforms as well.

As I wrote this blog entry I looked up the VirtualBox home page to I could link to it, and I discovered a news blurb:

New Dec 17, 2008

VirtualBox 2.1.0 released!

Sun today released VirtualBox 2.1.0, a major update with exciting new features: among them better 64-bit support, hardware virtualization on the Mac, 3D acceleration, easier networking on Windows and Linux plus full VMDK/VHD support including snapshots. See the ChangeLog for a detailed list of changes.

I just installed it and took it for a quick spin, and things are looking great.

P.S. One little tip: Installing the Guest Additions into Windows is easy - there's a menu item in the Device menu, "Install Guest Additions...". I tried it, and it popped up an installer inside Windows. But since I was in the middle of applying service packs to Windows, I dismissed it so I could deal with it after my service pack upgrade reboot. But when I tried it later, it didn't work! Uh oh, is this incompatible with service pack 3 I wonder? Nope, it's much simpler than that. What "Install Guest Additions..." does, is mount a new CD-ROM into Windows. I'm guessing Service Pack 3 turns off auto-run! Probably a security thing. So all I had to do was open the computer folder, and click on the CD-ROM there to run the installer.

Oh, one other tip: The default RAM allocated to the virtual machine is pretty small: 512Mb. In the Settings for the machine you can crank it up bigger - I set it to 2 Gb. That makes the machine run quite a bit faster under load since it's not continually swapping to disk for virtual memory.

Thursday, December 18, 2008

JavaOne Submissions Due

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

The deadline for submitting talks for JavaOne 2009 is almost here. I was on one of the review committees last year, and saw many common patterns for rejected talks. I wrote these up in a blog entry last year - "Why Your JavaOne Submission Was Rejected. You might want to skim through it again while reviewing your submissions one final time!

Also, there's a new book about develping Ruby and Rails code with NetBeans, written by Chris Kutler and Brian Leonard. I reviewed some of the content and it looked very good. There are a lot of books on Ruby and Rails, but this book really shows you how to do things effectively with NetBeans. Mastering your tools is one of the best gifts you can give yourself as a developer (or as a craftsman of any craft, I think). I gave this advice a while back when I was interviewed for the JavaOne daily newspaper. Highlights from these interviews were recently compiled in this article, containing advice for students from a variety of developers. When this article was discussed on DZone I was surprised to see some people still talk about IDEs as something to be avoided until you know what you're doing. I think precisely when you are new to Java should you use an IDE to help you explore IDEs, to pinpoint errors are you're typing code rather when you try to run javac from the command line etc. The same goes for Ruby, Python, PHP, and so on. The IDEs let you explore APIs using code completion and go to declaration, they present important operations in project and execution menus, quickfixes and semantic warnings pinpoint errors in your code, and so on. If you're still writing code with Notepad, you're missing out.

Tuesday, December 16, 2008

NetBeans Screenshot of the Week #39: Ruby Code Coverage, Python Code Coverage

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

Here's a new feature for NetBeans 7.0: Code Coverage support!

First a screenshot:

There are a couple of key points about the code coverage support:

  • It is fully integrated into the IDE, and especially the editor. You don't get read-only views of your files with coverage information; the files themselves include coverage highlights. You can view and edit the code directly, use your normal IDE navigation commands to jump around, and so on.
  • Enabling and disabling code coverage is trivial. There is no need to modify your build files, run special rake targets etc. Just select the menu checkbox to enable coverage in the new Code Coverage project menu, and then continue as before - run unit tests, run files, run the tests in your project - and the results are updated live as soon as the executed process finishes.

I didn't think a screenshot really described this feature well, so I recorded a couple of quick sessions where I use the feature:

Here's a couple more images if you have problems loading the videos:

The Python version is already in the 7.0 builds. The Ruby support isn't integrated yet; it probably needs some tweaking to make it work on Windows.

Friday, November 28, 2008

NetBeans Screenshot of the Week #38: E4X Support

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

As part of the JavaScript 1.7 work,
I also beefed up the E4X support. E4X allows you to embed XML objects
directly in your JavaScript source. Here's some simple E4X JavaScript:

We've had E4X support in NetBeans in both NetBeans 6.1 and 6.5. But now, in NetBeans 7.0, semantic highlighting is E4X aware, such that the source code looks like this instead:

The whole XML document declaration has a faint purple background, element names are blue and document content is bolded.
(The green here, as for all JavaScript code in NetBeans, indicates variables in global scope.)

Another new E4X feature is formatting/pretty printing. If you press Alt-Shift-F to format the above source, the E4X is
reformatted and indented:

In addition, the navigator window shows the logical structure of the XML document as well. Clicking in the source
will expand and select the corresponding item in the navigator, and conversely double clicking in the navigator
warps the source to the corresponding location in the XML declaration:

Finally, there's some primitive support in code completion for the XML document:

Tuesday, November 25, 2008

NetBeans Screenshot of the Week #37: JavaScript 1.7 Support

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

I just checked in support for JavaScript 1.7. This means that NetBeans will no longer give you syntax errors if you try to use the new language. Here are some screenshots. First, let's open a file using JavaScript 1.7 constructs:

Obviously the file is using yield which is not recognized as a keyword, and in particular, this is just a syntax error - the parser was expecting a semicolon. But there's a little lightbulb... let's see what it says (type Alt-Enter or click on the lightbulb):

Let's select the option to change the active language to 1.7. I could also open the JavaScript options dialog directly and set it to 1.7 as shown here:

After applying the language change, the editor immediately re-lexes and reparses the file according to the new language. The errors are gone, and yield is now a proper keyword:

Similarly, the let keyword can be used, as well as destructuring assignments, generators and iterators, etc. (See New In JavaScript 1.7 for more information).

If you place the caret on function definitions, it will highlight all exit points, and this includes yields now.

This work isn't done; I'd like it to be smarter about generators, and there may be some issues with scope handling. But at least the editor doesn't get in your way with false error messages now - you can start writing JavaScript 1.7 clean code. (This is with NetBeans 7.0 dev).

In related news, I just heard from Tom Enebo that a lot of Ruby 1.9 language features are now supported by the JRuby trunk parser, so hopefully we can soon start fully supporting Ruby 1.9 features in NetBeans as well.

Friday, November 14, 2008

NetBeans Screenshot of the Week #36: Python Code Completion

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

Here are some screenshots of the code completion working for Python.
First, let's try it from an import statement:

Note that the documentation for the module is shown. Next, let's
try completing on imported symbols - again we've got the documentation:

I'm writing a unit test, extending TestCase. Now let's try overriding
some methods - completion after typing def:

Obviously, invoking methods on self. will show the inherited
methods - and their documentation:

Here's completion in my function (not on self) showing
the local variables, imported symbols and builtins. Notice that deprecated
methods are marked with strikethrough (and deprecated modules are in imports
as well, though I didn't show that above).

Notice also that the documentation here looks better - it has links,
colors, etc. This is because NetBeans supports the reStructuredText
Docstring Format (see PEP 287).
System libraries are being documented in this way now, so more of the
documentation will start looking like this.

When NetBeans finds embedded code, it will also syntax highlight the
code fragments in the documentation:

You can use this in your own documentation as well obviously.
Here's a file where I have restructured text. As I'm writing it,
I can -immediately- preview the text myself by just pressing
Ctrl-Shift-Space (Show Documentation - for those who don't know, you
can do this on regular code as well if you want to just get the documentation
for the code at the caret, rather than completion alternatives up to
the caret prefix.)

Here's a final example - completion after raise (or
exception) will show completion for all Error

I'll be quiet or about a week now -- I'm going on vacation! Not great
timing given the release date, but this has been planned since February...

Thursday, November 13, 2008

NetBeans Screenshot of the Week #35: Support for Python's __all__ Variable

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

Let's start with a pretty simple module - an import, a function, and a couple of classes:

(Notice by the way the semantic highlighting here - the unused parameters and variables are marked with a slight grey underline,
method names in bold, parameters in orange. More on that another day.)

Now let's go to a different file and import this module. Code completion is context dependent and shows us
which symbols are being exported by the given module. We see not only our functions and classes, the imported symbol sys
is also being defined by module, meaning that if you do from compl4 import * you would have sys defined as well:

Python has a way to deal with this: You can specify a variable named __all__, which lets you define exactly what the public API for
this module should be (more details).

Let's go in and define the names we want exported by our module. I will add __all__ = [ "Bar " ] somewhere in the file, to specify that only Bar should be exported:

Look at the file outline / navigator on the left hand side. As soon as I typed in the __all__ line,
the icons changed to show me that only the Bar class and its method are now "public"; there are little
lock icons on everything else! Let's see what code completion offers when we try to import this module again:

Yep, only the public API is exported now. Let's append to the __all__ list to export the func function:

You don't have to rely on icons to tell you if something is wrong though. Let's say I've forgotten all about this, and I decided
to rename my Bar class:

NetBeans checks that all the symbols you declare in your __all list actually exist, and points out any discrepancies. Here, the public Bar class no longer exists since we renamed it.

Here's another little bug I came across related to this. I was checking the __all__ handling on various library files,
in this case Notice how in the navigator, all the functions are public, except one...

Sure enough, the code looks like it's registering this conditionally added method... Do you spot the problem? There's a dot missing!
As soon as we add it, we get this:

Of course, this last bug was already detected by the
unresolved symbol detector...

Wednesday, November 12, 2008

Ruby Screenshot of the Week #34: Detecting Duplicates Hash Keys

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

You didn't think we had forgotten Ruby, did you? Here's a new Ruby editor feature: Detecting duplicate hashkeys. As "Fjan" (who filed the
request) says, when some Rails calls allow dozens or more keys, you can accidentally add a hash key twice, with subtle results. NetBeans now tracks these down and points them out for you.

This is going into 7.0, but you can use this 6.5-compatible module as well.

Martin and Erno have also been hard at work. There are many new features in 6.5 (which I'll cover when 6.5 ships in a week or two). But in the 7.0 trunk, the debugger can attach to arbitrary and remote processes - see
Martin's blog entry for more info. And Erno has added support for Shoulda testing and more test runner improvements - details here.

Tuesday, November 11, 2008

NetBeans Screenshot of the Week #33: Finding Unresolved Symbols

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

In my last entry I showed that the Fix Imports action will alert you to an unresolved symbol. This is really helpful since it can help you track down runtime problems.

Obviously, you shouldn't have to go run Fix Imports on every file, periodically, to look for unresolved symbols!

NetBeans has a special semantic checker for this. It is off by default, for two reasons:

  1. I added it just a couple of days ago - during the stabilization period for EA, so I didn't dare enable it in case it's seriously broken.
  2. There are cases where it doesn't yet work. I'll describe that later.

To enable it, open up the Editor options dialog, and under Hints, select Python. You should now see the current batch of Python hints. Enable the "Unresolved Symbol" hint:

Now let's open up the file again:

As you can see, the file is marked erroneous (by the red square above the scrollbar), and the scrollbar indicates where in the file the error is. Clicking on it to expose the error location you can see an underline under the unresolved symbol, and a tooltip explaining what the problem is. The error is also shown in the tasklist, so you don't have to open each file to look for problems - just let the tasklist scan your project for problems.

Here's another example. I opened up the Django 1.0 sources, and NetBeans pointed out this problem to me, in django.utils.simplejson.tool:

This shows that the import is unused, and that the attempts to call methods on the import do not work. The call would need to be changed to

obj = django.utils.simplejson.load(infile)

or the import changed to be

import django.utils.simplejson as simplejson

Here's one final one... Perhaps helpful to people having done a lot of Java development recently:

Finding unresolved symbols means that NetBeans must be able to find all the libraries you are importing. This means you can't just point NetBeans at a plain Python file - your project has to contain all the libraries you are using, and, you have to configure your source path correctly such that NetBeans computes the correct package names for your python modules. (I accidentally pointed to the "django" folder instead of its parent folder at first, which meant django. wasn't part of the package prefix and that meant that all kinds of symbols couldn't be resolved.)

Second, this won't work for dynamically generated symbols (eval etc). I haven't run into this a lot yet - but I'm sure there are areas where the unresolved detection can't find symbols that really do exist.

P.S. Is there a canonical Python term for what I've called "unresolved" here that we should use instead?

Monday, November 10, 2008

NetBeans Screenshot of the Week #32: Python Import Management

WARNING: This blog entry was imported from my old blog on (which used different blogging software), so formatting and links may not be correct.

We're about to release an early access release of Python support for NetBeans. We have ambitious goals for the Python support: NetBeans should be the best IDE for Python development, bar none!

Over the next couple of weeks I'll post screenshots of a bunch of editor related features for Python. Today I'll start with a feature near and dear to my heart: Import Management.

I used to be a die hard XEmacs user. When I started coding in Java, I continued using emacs for a couple of years. And the feature which finally made me abandon emacs for a Java IDE? You guessed it -- import management. If I wanted to use for example the List class, I could just press a keystroke to choose which List class I had in mind, and the import statement was added for it at the top of the file while I could continue working in the middle of the file somewhere.

To be sure, this is not rocket science, and Emacs support for Java has improved a lot over the years. But at the time, this was a huge productivity boost and coupled with other code-intelligence and navigation features was enough to lure my away from my comfort zone in the IDE to what was a less powerful plain text editor, but a more powerful Java coding tool.

Anyway, Python shares some similarities with Java. There are lots of libraries, and you need to import modules before you can use them. And this is where NetBeans' Python support can help.

Let's say I've tried calling the cmp method to compare files. I ran the code and realized I need to import cmp before I can use it. To get the IDE to import the symbol for me, I can just pull up the context menu and select Fast Import (or more realistically, just use the keybinding Alt-Shift-I):

This brings up an import dialog like this:

This tells us there are two definitions of a function named cmp - one in filecmp and one in functions. I'm interested in the first one. The label below the list tells me that I can hold the Alt key or the Shift key when I select the list entry to change the import statement. And below that, there is a preview of the import statement that will be generate. As soon as I press the Alt key, the window changes to this:

Here, the bottom line is showing that we will be doing a wildcard import of all symbols instead of just the target symbol. And finally, let's press Shift:

This will not just import the cmp symbol, it will instead import the whole module and change my current call to use the imported module symbol. When I press Enter my editor looks like this:

This is the import facility I ditched Emacs for ten years ago. But it's not how I manage imports for Java anymore. Instead, there is the Fix Imports action (it's also there in the menu -- Ctrl-Shift-I for short). Unlike Fast Import, which looks at the identifier under the editor caret, Fix Imports will look at all your imports in the file, and, well, fix them! This means sorting them, removing duplicates, even removing unused imports. And obviously, resolving imports automatically using the machinery I showed above. Here's how this works.

Here I've opened the file, and then I've commented out all the imports:

The file is now broken - it has various references to functions that are not defined/imported anymore. Let's see what happens if I press Ctrl-Shift-I:

Tadaaa! It has figured out the missing imports and has inserted them. This is confirmed by a message in the lower left hand corner:

This just happened automatically because NetBeans could figure out, unambiguously, what the missing imports must be. Let's make it less clear-cut. I'm going to insert a call to get(). Now let's try again:

This time we get a dialog. You can see that it has figured out the sys and os imports - but it is not certain to do about the call to get(). There are two modules defining this function - unicodedata and webbrowser:

The corresponding function signatures are listed to help me decide which one I'm interested in. I can choose one of them, and when I press Ok, the imports are applied.

In some cases it may not find the symbol at all. Let's put in a bogus call:

Here the import manager can't figure out what to do - there is no known matching function. Only when NetBeans is certain about the imports does it proceed to clean up the imports without any user intervention.

This facility can also help find bugs! When I was testing this feature, I happened to invoke it in the file which ships with Jython. Here's what it said:

That's right - the code is referencing nonexistent class ValuError. This clearly should be ValueError!
NetBeans has additional facilities for tracking down these types of bugs which I'll cover in another post.

So far I've focused on using the import facility to import new symbols. If you're writing code using the editor you may not need it, since when you use code completion to insert calls, it also automatically inserts import statements as needed.
However, another useful aspect of Fix Imports is cleaning up your imports: removing duplicates, sorting alphabetically (configurable), and even deleting unused imports.

If you open up a file with unused imports, NetBeans will show you that immediately:

There are quickfixes here, so if you click on the lightbulbs or press Alt-Enter, you can see the import actions:

The integration with quickfixes means that if you look in the tasklist, it will show files containing unused imports. But when you're editing, you can just press Ctrl-Shift-I to clean up the imports - it will do everything we've covered so far: Add missing imports, remove duplicates, remove unused, and sort the remaining imports.

Import management is also tied into source formatting. If you format your source, you can optionally have it clean up formats. Here are the formatting options for Python:

You can play with the checkboxes and the preview will show you the effect. Here's what happens if we enable sorting system libraries to the top. Note also that removing unused imports is not done by default during formatting (whereas Fix Imports will perform it automatically). You can change that in this formatting preference panel:

As I mentioned, we'll have an early access version out very soon. If you're feeling adventurous, you can always download builds from the continuous build server - Please file bugs and ideas here.