The IT Blog

Category: HowTos

Stuff that was hard to find & fix

  • Convert Mercurial repository to Git

    I have converted yet another googlecode Mercurial repository to Git – and as it took me (again) a bit too much time, here is my recipe:

    Prepare

    Log into a Linux shell (Windows will hardly work – at least it didn’t work for me). If you don’t have some remote shell, download an Ubuntu VM and fire it up (you might need to install VirtualBox if you haven’t installed it already).

    Check out your HG repo:

    hg clone https://code.google.com/p/MyHgProject/

    Check if you have to remap some author information:
    hg log MyHgProject

    If you want/have to remap, simple create an author map file
    oldemail@example.com=Firstname Lastname

    Convert

    Get fast-export and convert your repo:
    git clone git://repo.or.cz/fast-export.git
    mkdir new_git_repo
    cd new_git_repo
    git init
    /path/to/hg-fast-export.sh -A ../authormap -r /path/to/MyHgProject
    git checkout HEAD

    Push

    Now change the source setting in your google code repository to git and push the local codebase:
    git remote add origin https://code.google.com/p/MyHgRepo/
    git push --all

    At this point, your upload might fail with something like “error: RPC failed; result=35, HTTP code = 0”.
    This can happen if your upload takes too long and this is a documented bug.
    I simply solved this issue by pushig the git repo from a shell with a fast enough upload speed as my local connection obviously was too slow.

    Wiki

    Now you might realize that you have “lost” your complete wiki – don’t worry, it’s still there!
    Switch back your repo setting to Mercurial and repeat the process for your wiki which you can usually find at https://code.google.com/p/yourProjectName/source/checkout?repo=wiki

    That’s it. You should now be ready to use Git!

    Thanks to following sites hedonismbot, scrambled tofu

  • How to write useful error messages

    Each programmer has to deal with error mesages in some way. Either we have to check our data (and possibly raise an error/exception) or we are using libraries and have to deal with error messages or exceptions that are raised in these libraries. The purpose of this post is to point out how meaningfull error messages can save you time. Either your own time (by avoiding answering support questions) or your own debugging time.

    So how can an error message save or waste time? Imagine the situation where you try to open a file and all you get is an error. What do you do next? I guess you’ll probably check if you’re trying to open the correct path. Maybe you did, then you possibly check if the permissions are okay, if you’re trying to open a directory or a file, etc until you found the problem that is causing the error. Actually this can be quite some wasted time – why? Becasue you’re doing all the stuff that the system already checked. The system decided that one of the condidtions failed and exited. So – shouldn’t the system just TELL the exact reason? And even better: shouldn’t it also just tell you a possible solution? Well – actually I do not blame “the system” but the guy who write the line that threw the error without any further information. Everytime I get an absolutely uninformative error I think “was it really so damn hard to add one more line of code that just mentions WHAT went wrong?!”.

    Assume the situation where you try to open a file with the path being configured in an external config.properties file.
    I think the quality of an error message can be categorized in one of the three categories:

    1. Reporting: The system just reports that there was an error.
      Example: IO Exception occured.
    2. Informing: Same as 1, just with more information.
      Example:  IO Exception occured while opening file: <filename>, null=false, exists=true, is file=false, is directory=false, readable=false, writable=false
    3. Supporting: same as 2 with additional information of how to solve the problem.
      Example: IO Exception occured while opening file: <filename>, null=false, exists=true, is file=false, is directory=true, readable=false, writable=false; Was the pat set correctly in config.properties?

    In the past weeks I have seen very useful error messages in Google’s Android (“… Have you declared this activity in your AndroidManifest.xml?”) or in twitter4j (Displaying a shortURL to an FAQ page explaingin a very common error).
    Bad examples are plain NullPointerExceptions when 3 parameters were checked, one of them was null and the according is just followed by a throw new NullPointerException so that you even did not recognize from the message WHICH parameter was null.

    Conclusion: Please write robust code and please provide helpful and supportive error messages to the folks using your code!

  • An introduction to the Logging framework (a.k.a. System.out.println is evil)

    Motivation

    One of the first lines a programmer will write in a new language is surely “Hello World”. In Java you can write to the console or to the error stream quite easy with a simple System.out.println("Hello World") or System.err.println("Hello Error"). Great! When the code grows, bugs creep into the code and make live a bit harder. At this point programmers should definately start a deep and loving relationship with the debugger that is delivered with the IDE instead of using System.out/err.println() as debug method. Nevertheless – there are at cases, where a Debugger cannot (or hardly) be applied:

    1. The code runs in the IDE but not if startet directly. – What the hell’s going wrong?
    2. Handling of exceptions. An exception indicates a state that should not have happened and therefore it might be considered to be logged.
    3. The code is deployed to someone else and you cannot attach the debugger to his/her machine.

    At either point, beginners tend to use System.out/err.println() to trace the execution path. While this might be okay if the onlyone that is using the code is the programmer alone, this can be very annoying if you are working in a team: If you forget to remove the debug messages, you’re polluting someone elses console output. Even worse: if the code is deployed to a client which reports an error, you cannot raise/lower debug levels or just enable/disable debugging. Do you really want to send a “debug version”? (No you wouldn’t.)

    (more…)

  • Parse Error: There is a problem parsing the package / Beim Parsen des Pakets ist ein Problem aufgetreten

    Just finished my first (real) Android App and wanted to deploy it in my phone (not just in the emulator).
    So I sent an email to myself with the .apk attached as several tutorials said that you can just open it from your mail to install the app.
    On the phone, I opened my mail app K9 and opened the .apk. But instead of installing I unfortunately just got

    Beim Parsen des Pakets ist ein Problem aufgetreten.
(in english) Parse Error: There is a problem parsing the package.

    even though the minSdkVersion was set correctly.

    The solution was rather simple: Do NOT open the .apk in K9 directly but save it to the sdcard and open it via a filemanager like Astro.

  • Drupal shows warnings after migrating to new server

    Recently we migrated a Drupal managed website to  a new server. – Everything was fine, just on some pages, some php-warnings showed up that did not show up on the original site.
    After some investigation I checked php’s error_reporting setting and realized that the settings of the new new server were more sensitive than the old one.

    The solution was pretty simple. As I definately didn’t want to start hacking around in Drupal, I just turned off showing the errors to the user:

    Administer > Site configuration > Error reporting
    (or in german:) Verwalten > Einstellungen > Error reporting
    /admin/settings/error-reporting
    Error Reporting: Write errors to the log (instead: Write errors to the log and to the screen).

    Pretty easy.

  • TrafficMining Project goes open source

    Quite some time ago I wrote about a little demo that was published at SIGMOD 2010 and SSTD 2011 (see post1 and post2).

    The TrafficMining project could be described shortly as:

    An academic framework for routing algorithms based on OpenStreetMapdata. Actually this framework is not intended to replace current routing applications but to provide an easy to use GUI for testing and developing new routing algorithms on real OpenStreetMap data.

    Well, what makes this worth a post is the fact that we finally switched development over to GoogleCode with a discussion group at Google Groups.
    GoogleCode has the major advantage of a Mercurial repository, an issue tracker, easy code reviews and an miproved possibility to contribute code. If you just want to follow the development, just join the google group or keep a bookmark to the project’s update feed.

    By the way: the PAROS and MARiO downloads can be found there in the downloads section.

  • Sometimes AntiVirus Scanners just suck

    Due to some very strange and non reproducible bugs I deleted my NetBeans profile to get back to a fresh post-install state. This mainly hat two very different results: A positive one and the one why I am writing this post. The positive one is/was that the mystery-bug disappeared.

    After deleting the profile, NetBeans wants to update all its modules to the latest releases – which is okay.
    The problem however was that NB never managed to download all the modules and just got stuck randomly between 0% and 100%. NB just said “downloading …” but the task manager showed 0 bytes/sec traffic. First I thought that the download/update servers might be overloaded. After 3 days of repeatedly trying –  I was very douptful about that first guess. Finally I sent an email to the NetBeans mailinglist, if s.o else also experienced such a problem.

    The first answer came in just 70min after my question saying “I had this problem and found that if I turn off AVG I could get the plugins.”.
    Uhoh – I didn’t mention my AV scanner in my post – yet I am using AVG, too. And – surprise – after disabling resident shield, mail-scanner, link-scanner and identity protection – NB suddenly was able to update to the latest version. Unfortunately AVG never said that it was doing something of that kind!

    So the lesson learned: If some really weird stuff is happening, try to disable the AV scanner – but don’t forget to re-enable it!

    Update: same happens with Eclipse when downloading/installing plug-ins. – At least AVG is consistent with blocking Java-IDEs.

  • Linking API and Sources to your IDE’s JARs (Part 2)

    I tend to upload my complete project into version control. This includes the sources, tests, Jars and also the nbproject directory where NetBeans stores the project configuration. By doing so, I can check out the project on a different machine and start quickly without having to configure the project.

    Sources and API Docs of external libraries are not commited as they’re are not requried for compiling. I usually keep sources and docs in a separate place outside my project (let’s say <userdir>javaLibs...).

    When I check out the project on a different machine I can do coding but I do have neither the sources nor the API docs. Even worse: as I’ve commited the whole project including the configuration, I have also commited the nbproject/project.properties file which stores the pathes to the source and docs. Which is not a problem if the pathes on all the machines are the same. But when a new contributer wants to join in, (s)he either has to use the same directory structure (and possibly the same OS) or he has to overwrite the settings. Both not very desirable.

    (more…)

  • Linking API and Sources to your IDE’s JARs

    For productive programming, I think it is absolutely crucial to also have both the API documentation and the source code of the according libraries available and integrated in the IDE in order to gain maximum productivity. Integrating the API and sources is pretty easy in NetBeans (as well as in other IDEs):

    Right click the Project > Properties > Libraries > select the JAR for which you want to link source and API and hit the edit button on the right.
    Now you can select a folder, Zip file or Jar file for the API and sources, hit OK and you’re done.

    Whenever you’re using a class from this library, you now can step into this class (by Ctrl-Clicking for example) or quickly jump to the API by pressing ALT+F1 when the curser is at the corresponding class/method.

    If you are annoyed by swithing between IDE and Browser or if you just forget the Alt+F1 key combo that opens the browser with the correct API page, just enable the NetBeans inline Java-Doc viewer by selecting:
    Window > Other > JavaDoc
    This brings up a new panel which shows the JavaDoc comment of the class/method which is curently selected by the cursor. And you don’t even need to press any key for updating the view as it is updated automatically.

    If it doesn’t work, I usually experience the following two errors:

    1. JavaDoc doesn’t work: If I perform Alt+F1, the browser doesn’t open and the status bar on the bottom of the NetBeans window shoes a “Cannot perform Show Javadoc here”. Well – check the Path then. It should end in a directory that also contains the index.html, package-list, allclasses-frame.html etc.
    2. The source is not displayed – even though the path to the Jar/Zip is correct! In that case, The Zip/Jar often contains all the source code in src/mypackage/foo.java. NB expects only packagis in the Zip, so that the content list should look like: maypackage/foo.java. So simply build another src.zip with the contents of “src/” (in this case) and you’re done.

  • How to create Memory Leaks by using Inner Classes.

    The most recent Java Specialists Newsletter finally convinced me to start this post that I was having in mind for quite some time.

    One of the really huge advantages of Java is that you almost do not have to care about cleaning up your memory as the Garbage Collector usually does this for you as soon as Objects are no more referenced. Usually this works really really well so that you really don’t have to care about annything! But maybe once in a while you may be observing something like a memory leak. Some people then call the Garbage explicitly – which is usually just a bad idea and possibly also doesn’t help either so that the “leak” remains. The better solution in this case would be profiling so that you can see why some classes are not cleaned up.

    A nice source for memory leaks can be the use of anonymous inner classes. Assume the following class where you want to compute s.th and return a Result-Object which derives from an Interface:

    interface Result{}
class Outer {
    int[] data = null;
    public Outer(int s) { data = new int[s]; }
    Object getResult() { return new Result(){}; }
}

    So if you call new Outer(1).getResult(), you will still have an instance of Outer in memory even though you did not keep an explicit reference. As explained in the Java Specialists Newsletter, each instance of an anoymous inner class always keeps a reference to the outer class! This is not a big deal as long as

    • you don’t keep a lot of data in the Outer instance or
    • if the lifetime of the Result object is not long or
    • if you won’t create a lot of results anyways.

    Let’s have an example. If you execute

    ArrayList l = new ArrayList();
int i = 0;
while(true){
    l.add(new Outer(0).getResult());
    System.out.println(i++);
}

    with the above classes without memory constraints (-Xmx), this will run for quite some time because you are only holding 2 class references (Outer, Result) 1 field (the emtpty data array) and an implicit reference from Result to Outer. Which makes a total of 48 bytes on my Win7 64bit machine (according to this measurement).

    Now change the parameter in the constructor of Outer from 0 to 100000 and execute the code again. In my case I am getting an OutOfMemoryException after a bit more than 2000 created instances as now suddenly each iteration consumes 400.048 bytes (48 bytes as before + 100.000*4 bytes for the int-array) even though we only keep the explicit reference to the Result objects!

    So – if you are creating an inner class the next time – you might have a brief look at the outer class as well and think about memory consumption and lifetime.