Building the Android IDE Tools

Do you want to try bleeding edge builds of the Android IDE tooling? No problem. We do all the tool development directly in a public git repository. Just build it, run it, and you can see all daily progress in our review tool.

Andy Rubin recently tweeted how to build Android - and as you can see, it's easy and the instructions fit within the 140 twitter character limit. However, if you want to work on the IDE tooling itself, there are many other tips and tricks to know about. The purpose of this blog entry is to document how you can set up an environment to build the Android SDK, including the IDE support, and have everything configured such that you can run and debug the Android support directly from within the IDE.

My hope is that some of you, once you have the Android tooling up and running, will decide to look under the covers, tinker with the bits, and hopefully find and fix bugs, or even contribute features! I'll talk more about contributions in a later blog post; today the focus is on building and running the bits even if you have no intention of contributing. Even bug reports from you on newly developed features would be useful.

Setup

First, you'll need to install the prerequisites for your platform. This means you'll have git, a C compiler, etc. There are quite a few steps here, and it's all platform dependent, so I'll just point you to the official build instructions where we have detailed instructions:  http://source.android.com/source/download.html .  If you're on a Mac, pay particular attention to the instructions on creating a case sensitive filesystem image. When I started I glossed over that part and hurried down to the git checkout command, only to discover later that I had to start over.

One other Mac specific tip, which isn't in the main Android page yet, is that Apple's recent JDK update "broke" the Android build on Snow Leopard since the build uses JNI. The location of the JNI header file changed or disappeared with the JDK update, so you now need to go to Apple and get the developer SDK, JavaDeveloper.pkg from http://connect.apple.com.

NOTE NOTE NOTE: Xavier tells me that building on Windows is more difficult because the SDK itself doesn't build on Windows, so you may have to download a system image and then build the tools components themselves manually. Hopefully I'll be able to provide more details on this soon.

Once you have everything configured, check out the source code using the first half of Andy Rubin's tweet:

$ mkdir android
$ cd android
$ repo init -u git://android.git.kernel.org/platform/manifest.git
$ repo sync

During the repo init command, it will ask you for your name and e-mail address; these will be used later if you decide to check in changesets and upload them for review.

Build

Now you are ready to build. While you could just type "make", this is where our SDK build instructions diverge a bit.

In a bash shell, run these commands

$ . build/envsetup.sh
$ lunch sdk-eng
$ make sdk

$ ./sdk/eclipse/scripts/create_all_symlinks.sh
That will run for quite a while and build everything. It will also create a lot of symlinks to help you edit and compile files within Eclipse.

Next, configure Eclipse. We support Eclipse 3.4 and up, but we're currently using Eclipse 3.5.2 for development. Make sure you get a version with RCP support, such as the "Eclipse for RCP and RAP Developers" distribution. I'm using eclipse-rcp-galileo-SR2-macosx-cocoa. (You can use Eclipse 3.6.1 - I just tried it. However, the test project won't compile. You won't need that project to run the Eclipse support. If you're going to be making changes you'll want to have it, but for now, if you're just wanting to try the new support and don't want to install an older Eclipse, go right ahead, and skip the "tests" project in the projects-to-import list below.)

To configure Eclipse, right click in the Workspace and choose "Import...", then select "Existing Projects into Workspace".  You want to import precisely these projects:

• sdk/eclipse/plugins/com.android.ide.eclipse.adt
• sdk/eclipse/plugins/com.android.ide.eclipse.tests
• sdk/eclipse/plugins/com.android.ide.eclipse.ddms
• sdk/ddms (import all 3 projects - ddmlib, ddms-plugin, ddmuilib)
• sdk/ide_common
• sdk/androidprefs
• sdk/sdkstats
• sdk/layoutlib_api

Things won't compile yet, until we finish the configuration.

Go to the Preferences panel, search for Classpath Variables, and define ANDROID_SRC pointing to the top of your source tree (e.g. the "android" dir in the tweet instructions).

Then right click on the ddmuilib project and open its properties. Go to Java > Build Path, and open the User Libraries tab. You should see ANDROID_JFREECHART and ANDROID_SWT as libraries there. These will need to be configured. For ANDROID_JFREECHART, select the library and click on Add Jars, then point to prebuilt/common/jfreechart and add all 3 jars. For ANDROID_SWT, add the 3 jars under prebuilt/common/eclipse, as well as the architecture specific version - for my Mac it's prebuilt/darwin-x86/swt/swt.jar but this will depend on your OS and architecture. (When I used Eclipse 3.6 I couldn't just add the jars from this dialog, I had to go and open the User Libraries panel and there I could define libraries named ANDROID_JFREECHART and ANDROID_SWT.)

Voila -- now all the errors in the Problems view should disappear! (There are still quite a few warnings.)

Run and Debug

The next step is to add a run configuration so you can run and debug the ADT. Hit the Run button. When it asks you what to run it as, choose "Eclipse Application".

Important: Open the Configurations panel and edit the new configuration. In particular, go to the "Plug-ins" panel, and uncheck the third or so item,  "com.android.ide.eclipse.tests". If you enable it you can run into some very subtle bugs where jars from the test module's classpath are accidentally conflicting with different versions of jars in the tools plugin. I wasted nearly a day on that issue...

If you're on Mac and you get "java.lang.UnsatisfiedLinkError: Cannot load 32-bit SWT libraries on 64-bit JVM", you need to add "-d32" as an argument to the VM.

You should now be able to run the ADT - it will invoke a second instance of Eclipse, where all the Android tooling is baked in. The first thing you need to do is point the tool to a version of the SDK. Open the Preferences panel, and in the Android panel go to the text field where you can point to an SDK root. If you have downloaded versions of Android in the past, you can point to the root containing all of the different platforms here. However, you don't have to -- when you ran "make sdk" earlier you built a trunk version of the Android SDK (from here on referred to as "AOSP", which I think stands for Android Open Source Project).  Therefore, you can point to your build output here -- and as you continue pulling down new changes and rebuilding the SDK, your Eclipse target will also use the latest and greatest.

To point to it, use <your tree>/out/host/darwin-x86/sdk/android-sdk_eng.tnorbye_mac-x86/

That exact path will not work for you, since it has various architecture and user names in it. So open the file chooser and drill down, first to out, then host, then your os and architecture, then sdk, and then some directory which contains "android-sdk_eng", your username and your platform.  Hit OK.

After a brief pause you should now see a platform listed in the listbox in the Android panel, and it will identify itself as version "AOSP". If you instead point to officially released versions of Android, you'll see versions like "5", "6" and "7". The reason we don't use a number for the trunk version is to prevent you from accidentally building an app compiled against this trunk version, and then releasing it on the market.

Now that you have your Android versions configured, you can create a new Android project. Do New > Android Project, and in the New project dialog, you need to go and choose the AOSP platform in the platform listbox.  Fill out the dialog. The only trick is that it asks you for a "Min SDK Version", and here you should again type "AOSP".  You can now go and try for example the Layout Editor, by opening res > layout > main.xml.

We're working actively on the layout editor, but as you can see we could use help in lots of areas, from feature development, to icon work, to bug fixing, to bug reporting! (Again, I'll talk more about contributions in a follow-up post.)

To stay up to date, just go to the command line, type "repo sync", wait for a while, and then redo the make steps (source build/envsetup.sh, lunch sdk-eng, make sdk, create_symlinks), and reopen Eclipse and rebuild all. If you get a build error, you can wipe out the entire out folder to build from scratch, but I rarely have to do that.

You can add additional Android platforms, such that you can choose which platform to use for WYSIWYG rendering (just like you can already switch theme, orientation, etc):

This is a new feature added just in the last couple of days. It lets you target older platforms but visually work with newer platforms, and quickly switch between them to see how your layout looks on different platforms.  Of course, to use this you need to point your root Android tree to an install containing multiple platforms. If you do add additional platforms, make sure you copy older platforms (android-6, android-7, etc) into the platforms/ folder of the newly built SDK, rather than the other way around. This is necessary because the directory structure changed recently.

Edit

Now that you can build and run, let's take a look at the source code. The main Eclipse project to look at is the "adt" project, which is the Eclipse plugin.

First, you can configure your editor to use our custom dictionary to help the spell checker recognize words that are in our comments but are not in the default Eclipse English dictionary.  Search for Spelling in the options dialog, and add sdk/eclipse/dictionary.txt as the "User Defined Dictionary" path.

Second, if you will be doing any editing of the source code, make sure you configure your editor to use our formatting preferences and our import order. Open the Java Code Style dialog, and for the formatter, import development/ide/eclipse/android-formatting.xml . For Organize Import, import development/ide/eclipse/android.importorder . The code style for the Android source code base is document in http://source.android.com/source/code-style.html.

***

That's it! You should now be able to get the source code, build, edit, run and debug the Android IDE tools!

Android

I'm now working on Android!

In particular, I'm working on tooling for Android. That means I get to work fully in open source again which I enjoyed in the NetBeans days. We use the public Android git repository and I submit my fixes into the Gerrit code review tool which is a really nice workflow tool for git; I highly recommend it.

As you may or may not know, the Android tooling is Eclipse based, so I'm learning both to use Eclipse as well as writing plugins for it. There are some things I'm missing from NetBeans, but there are also some new features I'm welcoming. I just need to rewire my motor reflexes for it all to feel natural!

In the past when I've posted NetBeans tips, many of you have helpfully listed the corresponding Eclipse (and IntelliJ) shortcuts. While I have found replacements for many of the features I'm used to, I'm still stumped with the following, so if any of you could enlighten me - either with existing keybindings or optional plugins, I would appreciate it:

• Most important: How can I, via the keyboard, navigate through my matches in the Find Usages search? This was my favorite keystroke in NetBeans (and I had meant to do a code tip about it) - if you press Cmd-period, you can navigate through find usages searches, build errors, diff view differences, etc. In Eclipse there is also Cmd-period, but it only works if the Search window has focus - and it doesn't work in the Diff view. (In fact, Cmd-period doesn't work in the Search window on the Mac; I don't know why, because it's listed as the shortcut, and Cmd-period does work in the editor view to navigate parsing errors. Luckily I've found the workaround of Cmd+Shift+period which navigates backwards, so I'm using that one, but I still have to give the Search window focus.)
• When I select a symbol in the editor, all the occurrences (of the same variable, class, method, etc) are highlighted. In NetBeans I could navigate through these via a shortcut. Is there a way to do that in Eclipse? (Somebody told me that I could select the text and then use Cmd-k to navigate, but that's text-search based and will jump to unrelated symbols that happen to be the same string, e.g. the same parameter name in a different function).
• How can I, via the keyboard, see the light bulb text? Sometimes I get a light bulb in the editor margin trying to tell me something. I have not found a way to see what it's saying other than hovering over it with the mouse. In NetBeans, Alt+Enter would show the light bulb text above the current line as well as the quickfix suggestions below the current line. In Eclipse I know I can press Cmd-1 to see the suggestions, but it doesn't show what the warning is and sometimes it isn't obvious from the suggestions what the problem is.  (For example, for an unused variable the suggestion is "Remove x").
• I'm really missing the NetBeans version control view (the one which lists the "status"; all your changed, added, and removed files and made it trivial to walk through and diff them). Maybe this is just because I'm working with git and there isn't a plugin which offers that view yet?
• This is minor, but when I initiate instant-renaming, it doesn't select the text, which NetBeans did, and I prefer that (so I can hit rename and type the new name). Is there a way to configure it to do that?

These are the answers I've asked some Eclipse users about and still haven't learned the answers to. One question I did ask, and eventually found the answer to is how to navigate files similar to NetBeans' Ctrl-Tab feature. The reason I'm bringing it up is that only one person finally showed me the right feature but most people didn't know about it - so if you're an Eclipse user, maybe you'll find this useful: "Next Editor". It's bound to something I can't remember in Eclipse, but I rebound the action called "Next Editor" to Ctrl-Tab (since Cmd-Tab is the equivalent application-switch shortcut on the Mac). You can press it to jump between two files - but the key thing is that when you're jumping between more than two files you can keep holding the Ctrl key after typing Tab to see a dialog of your recently edited files in most-recently-edited order.  I find this more intuitive than the jump-between-editing-locations since sometimes I jump to a file, look for something (without editing) and then I want to go back.

Anyway, I'm having a great time. I love working on tools, and perhaps even more, Android!

(By the way, if there are any SWT mavens out there, I would love an answer to this question which I posed on StackOverflow. If you know the answer but don't have or want to have a Stackoverflow account, feel free to post it here and I'll link back to it.)

Next Chapter

Today I'm starting in a new job at Google! I'm not sure what I'll work on yet, but I certainly have some strong preferences!

Google seems to be open to employee blogging, so hopefully I can continue writing about work related topics. And I realize I've forgotten to finish my IDE tips "series" - I have a few more tips that I planned to cover so I'll try to get around to that soon!

System.exit(0)

It's time for a new adventure!

 I joined Sun in 1996 and I've been here ever since -- including the Oracle acquisition six months ago. It's been a wonderful time. I've met many of the world's most talented engineers, I've had the chance to work on some interesting projects and I've learned a lot! I looked at my badge that I'm about to turn in (picture on left) and it has the original photo taken when I joined -- and I look like a kid!!

However, it's time to move on. I've chosen to take a new opportunity where I'll get to have even more fun. For now, I'd like to thank all my former Sun and Oracle coworkers for the great experiences you've given me and wish you luck with the future!

System.exit(0);

New blog home

I've migrated my blog from the old Sun Microsystems blogging site (http://blogs.sun.com/tor) to this new home on Blogger. It turned out to be a bit trickier than I had hoped, and as a consequence, there are probably some broken links and some bad formatting here and there. The old blogging platform was based on Apache Roller, and most other blogging platforms don't support import of Roller archives. I first tried to simply set up an account on jroller.com, but they don't allow new users. In case anyone else is trying to make a similar migration, here are some quick notes on what I did:

1. I exported from Roller into a MovableType format. 
2. I used the http://code.google.com/p/google-blog-converters-appengine/ project to convert the movable type file into a file suitable for Blogger import.
3. However, all the dates were broken; they all showed up with today's date. I tracked this down to the fact that the converter filter assumes the times in the archive are in 12 hour format (with am/pm at the end, and for me they were not.) So I fixed this by replacing the following line in google-blog-converters-r89/src/movabletype2blogger/mt2b.py :
   return time.strptime(mt_time, "%m/%d/%Y %I:%M:%S %p")
   with
   return time.strptime(mt_time, "%m/%d/%Y %H:%M:%S")
4. The biggest challenge was dealing with the images; the blog converted okay but still referenced all my image resources on blogs.sun.com. To fix this, I first uploaded all my images to picasaweb and made them world writable. Then I looked at the page in picasaweb which shows thumbnails for all the images in the album, and I saved it. This file contains image links. Unfortunately, the images all end up with many different url prefixes, so it's not as simple as just replacing the old image prefix with the new one. I wrote a short Java program to extract all the image links, and create a map from file base name to the full url. 
5. I then wrote a simple Java program to rewrite the Blogger import file such that it replaces all urls of the form http://blogs.sun.com/... with the corresponding new Picasaweb url using the above map. I also took the image urls from the thumbnails and removed the /s128/ portion of the url, which gets you the original image rather than the thumbnail. (Also, it turns out picasaweb insists on converting all .png images and .gif images to .jpg, so the URLs have to adjusted for those cases.)
6. I also did a little bit of post processing on the results; for example, I collapsed some repeated br tags that are no longer necessary, and I inserted a "This blog entry was imported and urls might be wrong"-warning at the top of each imported post.
7. One final tip: I discovered that a number of my images were missing. It turns out that Picasa by defaults hides small images (of which I had many) - so these were not uploaded! There are places to both go and enable these as well as adding .gif to the list of included file extensions, so handling this is easy once you're aware of the problem.

Hopefully this helps anyone else wanting to make a similar migration. I have the Java code for the image url manipulation if anyone wants it (but it's not generic so you'll need to do a little massaging for your own needs.)

Don't Use Implicit Return Types

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

JavaFX, like Scala, is a fully statically typed language. However, unlike Java, it allows you to omit type declarations in many places, since it can infer it. Coupled with the fact that you can use expressions as statements, and that the last expression in a function will be the return value, this lets you write really simple and clear code.

def s = "Hello";
def len = s.length();
function product(a: Number, b: Number) {
a * b
}

The function above shows an "Implicit Return Type": The compiler figures out the return type, which in this case will be a Number, and that is the signature computed for the function.

Unfortunately, this can sometimes lead to trouble! Here's a real-world example:

public function clearSelection() {
selected = null;
}

This function clears the selection in a node. The intention of the developer was probably for this function to have a "void" return type. But that is not the return type! The last statement is an expression, an assignment, which has the type of the left hand side, which is actually a node.
So the return type here is a Node! And it could easily have leaked an implementation class too.

This may not seem like a big deal here, but what if a subclass wanted to override this function? It would try this:

override function clearSelection() {
super.clearSelection();
moreStuff();
}

And this would fail compilation, with a surprising error message:

File.fx:1: incompatible types
found   : void
required: Node
override function clearSelection() {
1 error

That's right, your subclasses will now need to deal with this return type too! If it's an implementation class, they're in trouble!

There are other reasons implicit return types are bad too. Here's a function specifying a desired padding for a node:

public function padding() {
5
}

The user may have intended for this to be a Number, but since the default is "5" instead of "5.0", the implied type here is Integer, which means that subclasses will need to round their desired padding to an integer. And this is not always a good thing - see the pixel considerations article.

For all of the above reasons, our Coding Conventions Document states that you should not use implicit types, at least not for any non-private API. That includes package, protected and public functions. However, it's a simple mistake to make. We've all made it! For that reason, to help track these down, I've created a quickfix for NetBeans which identifies and fixes these problems. Here's what happens if you look at a function like the one described above:



The light bulb and underline warns you that there is a problem here. Clicking on the light bulb, or pressing Alt+Enter when the caret is on that line, shows this info:



It's telling you that you have an implicit return for a public function, and it's offering to insert the actual, current return type of the function. It's also offering to make the function Void.

Pressing return will fix the function:



There are certain return types you cannot express in JavaFX script - such as returns including generics signatures. The quickfix won't warn about those since there is nothing you can do.

You can find the module on the Plugin Center here. Hopefully it will be as useful to you as it has been to me!

P.S. We discussed this topic (implicit return types in JavaFX and Scala) for a while on the latest
Java Posse Podcast episode.

NetBeans 6.9 Released

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

NetBeans 6.9 has been released.

There are important improvements for everyone here - whether you're a Java developer, JavaFX developer, Ruby developer, PHP developer, C++ developer, ... or all of the above :-)

Since I spend most of my time writing JavaFX and Java code, by far the biggest improvement in NetBeans 6.9 for me is the JavaFX support. While there is still room for improvement, editing works pretty well now. In particular, the source code formatter works well on most source files and constructs, so I've used it to reformat some of our large and complicated source files, though to be on the safe side I've checked the diffs carefully before checking in (and turned on whitespace diffing in the version control). I occasionally have to make manual tweaks, but it's really helpful in cleaning up poorly formatted source files. And Go To Declaration now works reliably! The debugger still needs work. Apparently this needs some help from the compiler so hopefully the combination of JavaFX 1.3.1 and NetBeans 6.9.1 will address that.

One huge improvement in 6.9 is the reduction of scanning delays. In addition to performance fixes in that area, there are two life saver features:

1. Running is no longer blocked when a scan is in progress. Even if it's scanning, running starts immediately. This used to drive me crazy!
   
2. I can turn off some of the extra aggressive scanning! There is an option for this now, so I can turn off automatic timestamp checking and instead perform scanning manually if I should need it:
   
   
   
   (In case it's not obvious -- I'm talking about the checkbox at the bottom of the above dialog.)
   
   If you turn off automatic scanning, you can invoke it explicitly:
   
   
   

There are many other new features, and I'll get back to some of them in some other posts, but for now I'll leave you with a link:
Git Plugin compatible with NetBeans 6.9.

LCD Subpixel Rendering

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

While I'm on the topic of pixels -- here's another tip. This one is both Mac OSX and NetBeans (well, Swing) specific.

Subpixel rendering, an antialiasing technique, makes text look much sharper (wikipedia article).
Subpixel rendering was added to Java 6, so the NetBeans source editor looks great on Windows and Linux provided you are running Java 6. Antialiasing was never an issue on Macs, where the JDK used its own native graphics renderer which had subpixel rendering all along - until now!!

Here's NetBeans on Linux and Windows:



Let's zoom in. Notice the "strange" colors at the perimiter of the font strokes; they look strange here but oh-so-good at the proper resolution on an LCD:



Unfortunately, when Apple released Java 6 on the Mac, they switched the default graphics renderer over from their own "Quartz" renderer to the standard JDK one. That shouldn't be a problem, since Java already renders LCD text, right? Wrong! For reasons I don't know, subpixel rendering is NOT working on the Mac with the standard renderer. Therefore, by default, NetBeans text looks blurry (because it is only grayscale antialiased) on Macs.

This is easy to "fix". Just switch the rendering back to Quartz. You can do that with the following flag, added to the netbeans.conf config file:

-J-Dapple.awt.graphics.UseQuartz=true

The -J flag just tells the NetBeans launcher that the rest of the flag is a flag for the Java interpreter, and the -D flag says set the system property apple.awt.graphics.UseQuartz to true. Therefore, in your own Swing applications you can do the same thing, just drop the -J flag.

(The config file is in a place like /Applications/NetBeans/NetBeans 6.9 Beta.app/Contents/Resources/NetBeans/etc/netbeans.conf and you would add these flags to the netbeans_default_options line.)

Here is a before-versus-after screenshot; this will only make visual sense to you if you're viewing this on an LCD screen!



The subpixel rendering text is on the left.

Here's a zoomed in view, which shows the default (non-LCD renderered text):



And here's what you get when you turn on Quartz:



I presume Apple turned off Quartz by default in Java 6 for a reason. Does anyone know what it is? The JavaFX Release Notes mention some memory leaks. I've been using the flag to turn LCD text back on since I switched to Java 6 on the Mac a year ago, and it doesn't seem to have any adverse effects for normal NetBeans usage (mostly text oriented; I don't do UML editing etc).

More importantly, does anybody know why subpixel rendering from the JDK doesn't work on the Mac since it works everywhere else? Is there a way to trick it to work with rendering hints etc?

P.S. Speaking of launcher arguments - I have one other customization too. I add -J-d32. This tells the JDK that I want a 32-bit VM. I haven't measured personally what this buys me, but I saw that Charlie Nutter recommended it for NetBeans usage, and he certainly knows his way about VM tuning parameters!

Bay Area JUG Roundup 2010

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

If you live in the San Francisco bay area, I hope you can come to the Bay Area Java User Group Roundup 2010 tomorrow night! It is going to be held at the Oracle conference center, and both beer and dinner will be provided. There will be an update on the java.net migration, and the Java Posse are going to record a live podcast. Note - you need to sign up in advance so visit the event details page.

Hope to see you there!

Pixel Considerations

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

Antialiasing makes lines and shapes look smooth - though sometimes at the expense of sharpness. What if you're trying to draw a horizontal or vertical line where you don't need antialiasing? You might be under the impression that if you position your shapes at round integer positions, you will avoid antialiasing.
But it's not quite that simple - so avoid trying to be smart with layout code like this:

...
label.layoutX = (width - labelWidth) / 2 as Integer;
....

The idea here is that when you're placing something in the middle you might end up with a fractional value, say 42.5, and so you added the "as Integer" to round things to a whole number to avoid introducing antialiasing.

Well, this may have exactly the opposite effect! Take a look at the following picture, which shows two rectangles. Both rectangles have a stroke width of 1.0, and one of them is positioned at a round integer, and the other one is positioned at 0.5.



Here's a zoomed in view which makes things clearer:



Obviously, the rectangle on the left is blurry because antialiasing is attempting to show the line as being somewhere in the middle between them. The rectangle
on the right on the other hand is clear and crisp because the lines overlap EXACTLY with the pixel grid the line is rendered into.

Here's the thing though: The rectangle on the left is the one that was positioned at round integers, and the rectangle on the right is the one positioned
at round integer + 0.5 !

So should you run out and ensure that all your horizontal and vertical edges are positioned at 0.5? No. The key here is the stroke width. Take a look at the following figure, where I have position rectangles with different stroke widths (1, 2, 3) and different pixel positions (0, 0.25, 0.5).



Zoomed in:





As you can see, whether you match the pixel grid perfectly depends on the stroke width and the pixel positions. This actually makes sense. Think of your pixel grid as having x coordinates at the boundaries of each pixel. In other words, "0.0" is the left edge of the first pixel, and 1.0 is the right edge of the first pixel. The line position has to be the center of the stroke. So if you want to have a line of thickness 1, then that line will run exactly through the pixel, so we must position its center at x=0.5. When the stroke width increases to 2 however, the center will be in the middle (e.g. 1), and so we should position it at a round number. And so on.

When you're dealing with large shapes this isn't a big deal. But if you're trying to paint a grid (like the one below), a pattern, or small controls (like disclosure arrows - which is how I came to look into this), it can pay off.



By the way -- on OSX there's a nice screen zoom (hold the Option key and then do a two-fingered drag on the trackpad up or down) which makes it easy to zoom in and look at the pixels for anything on the screen. But unfortunately it doesn't show pixels as square, it does more blending, so it's much harder to tell what's going on at the individual pixel level. Get an image editor which lets you zoom in with clear pixel boundaries, or even a screen magnifying lens. Here's how the builtin screen zoom looks - as you can see it's not as clear as the pixel zooms above:



UPDATE: Marius taught me in the comments that you can turn off the OSX zoom smoothing in the Universal Access options. Sweet! I can now instantly check the pixels without going to an intermediate screenshot! Thanks!

Finally: Jonathan Giles from the JavaFX controls team has been doing a great job aggregating and summarizing interesting FX articles each week -- highly recommended if you're doing anything with JavaFX.

nbgit for NetBeans 6.9 beta

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

My brother was asking me for a version of nbgit (git support for NetBeans) that works with NetBeans 6.9 beta. I've uploaded one here. By the way, look at the Options wiki page, and especially the sections on Custom Actions, for useful configuration tips for nbgit.

IDE tip: Go To Implementation

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

One of the best parts of using an IDE instead of an editor is "Go To Declaration". When you hold the Control key (or the Command key on Macs), identifiers under the mouse turn into hyperlinks and when you click on the hyperlink you jump to the declaration point of say the method call.
You can also just hover over the hyperlink, and a tooltip will pop up and show you the signature of the identifier - the fully qualified type name, method arguments, and so on.

But what if the thing you are pointing at is an interface? Or an abstract class? Good Java programming style dictates that you should prefer interfaces over implementation types, so your code will typically only know about the interface or abstract type, and when you jump to the declaration, you go to the boring interface method definition, rather the interesting bits in the implementation!

This is what the Go To Implementation feature is for. And I have a confession to make: I didn't start using this until recently. At the recent IDE tips roundup session, somebody asked whether NetBeans supported Go To Implementation. By instinct I went to Google and looked for a NetBeans plugin to do it, and found one. Unfortunately, it was old and only worked with older versions of NetBeans. Uh oh. Then on a whim I decided to look in the Navigate context menu in NetBeans - and lo and behold, it's right there!



And you don't need to use a context menu to access this feature. The trick is to use a second modifier key to the normal ctrl-click on the identifier! In addition to the control key (or command on Macs), also hold the Alt key (Option on Macs).

Here's how it works. First, let's say you want to go to the actionPerformed implementation. You ctrl-click the identifier:



...and that takes you into the abstract method declaration. (In this specific case, the method is in a read-only file inside a .jar file, which is why NetBeans marks the whole file in gray)



That's not what we want. Go back to the previous editor location (Ctrl-left.) This time, hold both the Control and Alt keys (or Command and Option on Macs) when you click. When you do that you get this dialog:



This dialog has focus and you can use the arrow keys to navigate and press enter -- or just click with the mouse. As soon as you pick a specific override of this method, you jump to it:



That's all there is to it. It's doubly embarrassing that I didn't remember this feature, since I should know the implementation of it very well. You see, I shamelessly lifted most of the implementation of it for the language-infrastructure work I did to support Go To Declaration in Ruby, JavaScript etc:



P.S. I've updated the wiki page with a clarification that it's built in now.

P.S.2: Until recent builds, Go To Implementation in NetBeans only worked for interfaces, not methods extending abstract classes. So use 6.9.

IDE tip: Create Build and Execution Shortcuts

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

We're moving over to Oracle's mail and file storage, so I spent a couple of hours going through my old Sun home directory to see what I wanted to save. It was pretty fun coming across all kinds of old screenshots, source code and design documents for things I worked on 10-15 years ago.
I spent the first 5 years working on Sun's C/C++ development environment, and in particular the debugger. One feature I built was the ability to create custom buttons which would let you run frequent dbx commands (as well as reference builtin actions). Here's one screenshot I came across.


The CDE/Motif widgets look pretty dated

Anyway, this reminded me of another really useful feature in NetBeans: The ability to create custom shortcuts for ant targets. If you're using ant, you probably have a number of useful, extra build targets. One I find particularly useful for our setup (where we have lots of project dependencies) is to have a "just run" build target which skips all dependency checks and just runs whatever bits are there. This is particularly useful if I change something in an upstream project and I don't want any checking downstream.

Here's how you do it. First, let's say I've created an ant target to

Target ideas: Run findbugs. Run code coverage. Run an arbitrary script.

How to Write a Memory Leak Unit Test

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

Unit tests are great for ensuring that your functionality is correct. But how do you make sure you don't have leaks in your code, leaks that
eventually cause your application to crash when it runs out of memory? Unit tests don't typically don't run into memory problems directly because they tend to start up, run a little bit of code, and shut down -- getting a nice fresh memory environment on each test start.

One great way to help protect yourself from uptime problems is to write leak tests. These are unit tests where you first perform some operations, then do normal cleanup, and finally you assert that the objects you were using during the operation have been cleaned up completely. In other words, that there are no references left anywhere on the heap.

Testing for leaks isn't very hard; the basic trick is to create a weak reference to your to-be-cleaned object (by constructing a WeakReference wrapping your object), then you delete all your local references to the object, then you perform garbage collection, and finally you see whether the weak reference still contains your object. This works because the weak reference is handled specially by the garbage collection system.

It turns out things aren't quite as easy as that, because garbage collectors try to be smart, and simply calling System.gc() doesn't mean it's going to actually perform a complete and final garbage collection. This means that your unit test could incorrectly conclude you have a leak, since the reference is still held.

The NetBeans team has built a really good test infrastructure for this. When I wrote unit tests for NetBeans plugins, I could simply call their assertGC method to assert that the given weak reference should be garbage collected, and it would handle the rest. (Internally, they do things like actually allocating a bunch of memory chunks to really force the garbage collection to run.)

Here's an example:

// The following unit test fragment wants to ensure that a graphics object
// which shows the current selection in the authoring tool, is cleared up after
// the user clears the selection.
view.selectAll();
Node firstHandle = /* Node showing selection highlight, lookup code here */;
assertNotNull(firstHandle);

// Now clear view selection -- then assert that all the handles have disappeared
view.selectNone();

WeakReference<Node> ref = new WeakReference<Node>(firstHandle);
firstHandle = null;
assertGC("Selection handle leaked", ref);

When you write these tests you also have to make sure you null out any local variable references you are holding right there in the test.
The above test will pass if and only if the target object, firstHandle, is properly garbage collected.

But wait -- that's not the best part. Let's say you've written a leak test, and it fails. Now what? What do you do -- make the process pause and attach a profiler and try to hunt it down?

This is where their unit test support really shines. When the unit test discovers that the reference is still held somewhere, it uses its own heap walking library to hunt down the offending reference, and dumps it out as part of the test assertion failure message!!!

Here's the output of a real leak test failure:

Testcase: testMemoryLeak(SelectionHandlesTest):        FAILED
Selection handle leaked:
public static java.util.List javafx.scene.Scene@dc8a29-$dirtyCSSNodes->
java.util.ArrayList@4ed14172-elementData->
[Ljava.lang.Object;@3c22de9e-[41]->
javafx.scene.shape.Rectangle@612e4cd

In other words, we see that there is a static list in Scene named dirtyCSSNodes which is holding on to our target Rectangle. The syntax here is that you first see the class name for the object (where a prefix of [L means it's an array-of), then its system id, then the field name - and if it's an array, the array index.

When the unit test fails, it takes a while -- 20 seconds or so for the above test -- to actually produce the above trace. But the important part is that this is only slow when the test fails. It only has to work hard when you have a leak, and you don't want to have leaks!

I've been wanting to write leak tests for the authoring tool (which is written in JavaFX), since uptime matters a great deal in a tool which deals with potentially large objects (such as multimedia assets). And I realized that there is absolutely nothing NetBeans specific about the NetBeans leak unit test support. So I went and pulled out the relevant code into a separate library. The library basically contains two parts: a .jar file which contains the INSANE heap walking library, and a second jar which contains the assertGC() unit test assertion method and supporting infrastructure.

I have extracted this into standalone pieces (outside the NetBeans infrastructure) so you can get the bits easily - download leaktests.zip, then add the two jars in there on your test classpath and call assertGc() and assertSize() from your tests as described above. Here's the basic skeleton for all leak tests:

// (1) Create your objects
// (2) Look up the object you want to ensure gets cleaned up later:
Foo foo = // code to get foo
assertNotNull(foo);

// (3) Call your cleanup code which is supposed to free everything

// (4) Create a weak reference to your object reference, and null out
//   your reference
WeakReference<Foo> ref = new WeakReference<Foo>(foo);
foo = null;
assertGC("Foo leaked", ref);

All I've done is extract NetBeans code written by others so I've kept the licenses the same as for NetBeans. All credit goes to the author of INSANE, Petr Nejedly -- and to the authors of the memory assertion stuff in NbTestCase.java. All the source code for INSANE and NbTestCase are in the NetBeans mercurial repository.

In addition to assertGC, there is also assertSize(). This method can be used to ensure that the transitive size of an object graph is below a certain threshold! This can be good for writing tests to not only ensure that you don't have leaks, but that your data structures are of the rough expected size so you don't need excessive amounts of memory. There are more details on this on the INSANE home page.

One special note on JavaFX: The above leak isn't actually a leak; it is a deliberate optimization of the scenegraph, and the reference will be cleared up after the next scene pulse completes. Therefore, for unit leak tests, in addition to actually nulling out the weak references, you also want to run through a scene pulse as well. One really hacky, implementation-dependent and unsupported way to do that is to call scene.$scenePulseListener.pulse();. You probably want to isolate that in a utility method such that you can update it in one place when it needs to change...

Finally, note that I built this on JDK6. If there is interest perhaps we could create a wrapper project for this on Kenai or java.net, where people can also create say a Maven binary for this, a JDK 5 version (there is nothing JDK6 specific so it just needs a recompile, but I don't have JDK5 on this Snow Leopard Mac), etc. Hope you find this all as useful as I have!

IDE tip: AST-based Selection

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

AST-based selection is a feature which lets you select the surrounding "block", based not on syntactic clues (such as braces and parentheses, as is offered by practically all editors), but based on the parse tree created by the compiler (or the "AST", for abstract syntax tree).

In NetBeans, there are two complementary actions to deal with AST selection: Select Surrounding and its reverse. They have other names which I can never remember, but the keybindings are as follows:

Select surrounding block

• Mac: Ctrl-Shift-. (period)
  
• Windows/Linux/Solaris/Others: Alt-Shift-.
  

Reverse:

• Mac: Ctrl-Shift-, (comma)
  
• Windows/Linux/Solaris/Others: Alt-Shift-,
  

On my U.S. keyboard, the comma and period keys are marked with "<" and ">" for the shift key which is somewhat reminiscent of blocks.

These two actions cooperate and are stateful: they remember the original caret location such that you can "go back".
Here's how it works. You want to select something surrounding the caret, say the whole method. In that case you just
hit the surrounding action. With your fingers holding ctrl and shift you hit dot -- once, twice, bam bam bam. If you do it
quickly you might "overshoot" and go one too far and select say the whole class instead of just the method. If so just press
comma and voila, it goes back one step down the selection hierarchy you just came up.

This makes this a very quick way to select code blocks -- no more need to grab the mouse and try to hit exactly the selection
boundaries between characters.

Here's an example of where this comes in useful. I sometimes remove long expressions from method call arguments by pulling it
out as a separate variable. Here's my call, and I want to extract the whole Math.min expression.
First I move the caret anywhere within that expression:


Then I press Alt-Shift-. (or Ctrl-Shift-. on Mac):


One more time:


Let's pretend I was doing it quickly and hit it one extra time:


That's too far, so I press comma (with the modifier keys still pressed):


In practice (and with practice) this is done very quickly, faster than I can
accurately hit both ends of the expression with a mouse drag. And to finish
out the example, let me show a couple of more things related to the task.
Notice how there's a light bulb in the left hand side margin of the editor.
That tells me there are associated quick fixes. I press Alt Enter,
and the quick fix dialog is shown:


I choose "Introduce Variable" and press Return. This gives
a dialog where I can choose the variable name (and whether to replace all occurrences
when applicable). It suggests "min" since that's the function
name of the expression we're extracting.


Just press Return to apply the refactoring, and we've pulled out the argument
expression:


I just pressed return and got the default name. Then I realize that this may
not be a great name. What is that last parameter again? I can place the caret
somewhere within the last argument and press Cmd-P (or Ctrl+P
on other platforms) to get NetBeans to "Show Method Parameters".


Let's rename it to length instead. Press Ctrl-R (all platforms) to
apply rename refactoring, which (because this is a local or private variable)
will have "instant rename" applied where your edits are applied immediately
for every keystroke:


Just type the new name - notice that all the boxed expressions are edited synchronously.


Using AST-based selection to identify code blocks to extract as variables or methods
is useful, but it can be used in other ways as well. Let's say I want to add
a second validation step to this function. First I select the code block using
AST selection:



NetBeans has a keyboard command for duplicating the current line: Ctrl-Shift-Down
(and on Mac, it's Cmd-Shift-Down.) Note that when you have a selection, this
duplicates the entire selection. So all we have to do now is apply the duplicate command
and we've got a second if clause we can just tweak to whatever it is it needs to do.



AST based selection should be available in nearly all editor types in NetBeans.
In Ruby, JavaScript, Python, etc it also handles documentation
blocks well - so if you apply select surrounding in say a Ruby comment, it will first select
the current line, then the whole documentation block, then outwards in the AST hierarchy.

Hope you find this helpful. If there are Eclipse or IntelliJ readers who know corresponding
keybindings in Eclipse, feel free to leave a comment so others can find it easily.

IDE tips

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

One of the sessions we had at the Roundup was one entitled "IDE tips and tricks". We basically sat there with our laptops and did a show & tell of various "hidden" IDE features we knew about to help productivity. There were Eclipse, IntelliJ and NetBeans users there - and even some advocacy for Vim :) We discovered pretty quickly that this would not make for compelling listening, so we just turned off the recorder.

One thing I discovered (and which I saw during our languages coding dojo day as well) is that a lot of people don't use many of the productivity boosters. Therefore, I thought I'd bring some of these up here. Since I've been intending to start blogging more again, I'll make this a regular "column".

To kick things off, I'll just point to an old blog entry which is still relevant: Hippie Completion. Learn to use Ctrl-K while editing, it will be worth your while. (Eclipse has this too as of Eclipse 3.1; I believe it's bound to Alt-/).

Transparent windows on Linux

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

JavaFX makes it easy to create non-rectangular windows -- just set a StageStyle.TRANSPARENT on your stage.style.
We use rounded corners on our popup menus, and we also have a non-rectangular and alpha-blended splash screen.

The other day I was checking how the cursors were looking on different platforms, and imagine my horror when I discovered how our new splash screen and rounded menus looked on Linux! Here they are -- as you can see you end up with white rectangles around the supposed-to-be-blended areas:





That looks.... craptastic!

Historically, the JDK didn't support alpha blended windows on Linux. However, that was added a while ago (I'm not sure exactly which version, but I think it was JDK 6 update 14). It turns out that the FX code which initializes the native frame for the stage does not do conditional checking for this; it simply turns off transparency on Linux. Fortunately, there's a System property you can set to force it to respect the transparency flag. You would obviously only do this if you know you are running on a JDK which supports transparency. And that's easy!

In your startup code, do something like this:

if (Utils.IS_LINUX and Utils.jdkAtLeast(1, 6, 0, 14)) {
java.lang.System.setProperty("javafx.allowTransparentStage", "true");
}

There are a couple of utility methods here that are simple - just looking at some system properties to determine whether we're on Linux and whether we're on a particular version of the JDK or higher (if you run on non-Sun/Oracle JDKs you may want to check for that as well).

public def IS_LINUX = osName.contains("linux");
public def IS_MAC = osName.contains("mac");
public def IS_WINDOWS = osName.contains("windows");
public def IS_SOLARIS = osName.contains("solaris");

/**
* Is the version of the running JDK at least major.minor.micro_update?
* In 1.6.0_18 macro=1,minor=6,micro=0,update=18
*/
public function jdkAtLeast(macro: Integer, minor: Integer, micro: Integer, update: Integer): Boolean {
def runtimeVersion = java.lang.System.getProperty("java.runtime.version");
def pattern = java.util.regex.Pattern.compile("^(\\d)\\.(\\d)\\.(\\d)_(\\d+)-");
def matcher = pattern.matcher(runtimeVersion);
if (matcher.find()) {
def currentMacro = Integer.valueOf(matcher.group(1));
def currentMinor = Integer.valueOf(matcher.group(2));
def currentMicro = Integer.valueOf(matcher.group(3));
def currentUpdate = Integer.valueOf(matcher.group(4));
if (currentMacro < macro or currentMinor < minor or
currentMicro < micro or currentUpdate < update) {
return false;
}
}

true
}

With that modification at startup, we get much nicer results:





P.S. Yes, there's a JIRA issue tracking this to be automatically done by the platform, RT-4797.

P.S.2. This is for desktop; I'm not sure this code is mobile-safe.

300 Episodes

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

We had a milestone in the podcast last week: 300 episodes. That's more than one episode per week for the 4½ years we've been doing it!
We spent our 300th episode as a retrospective, looking back at highlights over the years.

We looked up some statistics, too. The most amazing one (to us) is that we are now up to 15,000 downloads per episode! Our feedback alias has over 10,000 e-mails (not counting spam). And our discussion forum has nearly 20,000 messages at this point! Wow! Thanks for listening and participating!

(Photo by Gunnar Hillert)

Java Posse Roundup

WARNING: This blog entry was imported from my old blog on blogs.sun.com (which used different blogging software), so formatting and links may not be correct.

I've spending the week in beautiful Crested Butte, Colorado at the Java Posse Roundup. It's the fourth year in a row, with record attendance. And as always, it's been fantastic. It's hard to explain an Open Space conference - it doesn't sound like it would work, and I was personally surprised to see how well it worked when I attended the first time. Now I simply expect it, and it always delivers. One key reason why this open space conference works so well compared to say "unconference" events attached to major face-forward conferences is that you need to be away from everything and really immerse yourself - and here up in the Rockies we certainly are isolated!

The day before the conference started we had a "Languages Dojo" day where people pick different languages they're interested and go off building something. We had planned in advance to build a JavaFX version of the "Ohm Writer", a Zen-like text editor that is full screen, has nice relaxing background music, background sounds and typing sounds. We had a great time, and made a lot of progress. Unfortunately, we spent a lot of the day fighting with git (the version control system). I'm a very happy Mercurial user, but I've had a little bit of git-envy since I know it can combine local changesets into a single changeset to be pushed to the repository. I could see myself using that a lot. And setting things up on github for collaboration was very easy. But that's where the fun ended - nothing worked, simple merges aborted, error messages were completely unhelpful, and in general we repeatedly ended up checking out new clean workspaces and hand applying changes. I liked Mercurial before but now I appreciate it even more. (I hear the guys who were doing functional programming also were ripping their hair out with git. P.S. Joel Spolsky just posted a Mercurial tutorial).

The app is functional, and more importantly we made the editor start up immediately, load images and audio in the background and gradually fade in the image as soon as it's available. We also had difficulty playing the keyboard "click" sound until we realized you don't want to just repeatedly call play() on a media player -- you have to reset it -- either setting mediaPlayer.currentTime = 0s or calling mediaPlayer.playFromStart().

We also ran into another bug -- and this is a gotcha I've seen before, so it seems useful to highlight it here: For JavaFX Strings, null and "" are the same! Therefore, you don't want to write code like this:

while ((line = reader.readLine()) != null) {
// use line
}

because this will terminate the loop as soon as you reach an empty line!! Be very careful about checking for nulls explicitly when dealing with Strings.

We've also had Lightning Talks in the evenings. If you're not familiar with Lightning Talks, these are very quick presentations, one after the other, on any subject, but limited to 5 minutes. Yes, with a HARD 5 minute limit. The advantage of limiting the lightning talks to 5 minutes is that it forces the presenter to really focus on the interesting parts of the subject, and if it's really not interesting, at least you're only bored for a couple of minutes! The topics this year were really great though - there was even a fire-eating demonstration!
One way to keep track of the time is to use a countdown timer. We wrote one in JavaFX last year, which displays a classic countdown timer (imported Photoshop graphics) and a sound file playing as the timer is expiring. It's available as a webstart app if you want to run it for your own lightning talks, or you can study the code.

In addition to the morning open space conference sessions, the evening lightning talks, the progressive dinners, we've had some afternoon technical presentations and coding sessions -- check out our improvised projector screen in our living room: