Lenovo BIOS Update regarding Spectre

Update: since Ubuntu 18.04 this is not longer necessary as the software update triggers this and everything is done automatically.

In the following I describe the process to update the BIOS of a Lenovo Laptop that runs Linux for the recent problems related to Spectre.

1. Download the BIOS update “Bootable CD” from Lenovo, which is just an ISO image. For example for the Thinkpad T460 download it from here.

2. Extract the BIOS from the ISO as otherwise “dd” cannot copy the ISO properly:

geteltorito -o bios.img /home/user/Downloads/r06uj58d.iso

Check that the USB stick is really at /dev/sdb (confirm size)

sudo fdisk -l /dev/sdb

3. Copy the BIOS to some USB stick:

sudo dd if=bios.img of=/dev/sdb && sync

4. Reboot and while you see “To interrupt normal startup, press Enter” at boot press F1 to boot from USB. Follow the instructions after you connected your Laptop to the power cable, otherwise the update will not proceed.

Now when have updated the BIOS you should see a new BIOS version 1.32/1.11 (BIOS/EC ID: R06ET58W/R06HT31W) in the BIOS setup.

As I understand this fixes the Spectre vulnerability only. To improve the situation related to the so called Meltdown problem do a software update:

apt update && apt upgrade

Afterwards you should have at least kernel version 4.4.0-109:

$ uname -r
4.4.0-109-generic

Fun with Shapefiles, CRSs and GeoTools

Although I’m now in the “GIS business” for years I had never to deal with shapefiles directly. Now it was time also to investigate tools like QGIS and hack together a simple reader for shp files. At least I thought it was simple but calling me a GIS expert afterwards would be a ridiculous understatement.

GeoTools fun

A quick look and I decided to go with GeoTools as I knew it from name and I needed a tool in Java. Thanks to QGIS I understood quickly that in my case I had to deal with a list of a list of lines containing coordinates but how to read that via GeoTools? The internet provided several solutions, but I didn’t found complete examples for my case. As it turned out: I had to explicitly cast 2 times (!) first from “Feature” to “SimpleFeature” and then from “Geometry” to “MultiLineString”. Not sure if this is really necessary. At least this makes learning a new API very hard.

Now I had the initial code:

Map connect = new HashMap();
// a File is not sufficient as a shapefile consists of multiple files
connect.put("url", file.toURI().toURL());
DataStore dataStore = DataStoreFinder.getDataStore(connect);
String[] typeNames = dataStore.getTypeNames();
String typeName = typeNames[0];
FeatureSource featureSource = dataStore.getFeatureSource(typeName);
CoordinateReferenceSystem sourceCRS = featureSource.getSchema().getCoordinateReferenceSystem();
FeatureCollection collection = featureSource.getFeatures();
// allow for some error due to different datums ('bursa wolf parameters required')
boolean lenient = true;
MathTransform transform = CRS.findMathTransform(sourceCRS, targetCRS, lenient);

List<List<GPXEntry>> lineList = new ArrayList<>();
try (FeatureIterator iterator = collection.features()) {
    while (iterator.hasNext()) {
        SimpleFeature feature = (SimpleFeature) iterator.next();
        MultiLineString mlString = (MultiLineString) feature.getDefaultGeometry();
        ...
    }
}

How short and beautiful. But: It did not compile. And that although I was using the recommended “maven procedure”. It seems that GeoTools seems to follow a bit unusual path that it requires you to define the repositories in your pom.xml – I did only find a solution with the snapshot versions but this was sufficient for the time being.

CRS fun

At least it seemed to work then. But after further longish time I found out that the coordinates had just a tiny offset, so something was wrong with the source or target coordinate reference system (CRS) or with the transformation itself. Again QGIS helped me here and determined the source CRS correctly. But GeoTools was somehow wrong and initially I thought it was GeoTools fault.

But I quickly stumbled over another CRS issue and had to deal with exactly the same CRSs leading to different results. In my case it was CRS.decode(“EPSG:4326”) vs. DefaultGeographicCRS.WGS84 – so they are identical but the results were different!? It turns out that the coordinate axes are mixed! GeoTools fault? No! GeoTools even gave me the solution in its documentation:
“So if you see some data in “EPSG:4326” you have no idea if it is in x/y order or in y/x order”!

Deployment

Puh. Okay. I was ready for deployment and used my usual git and mvn assembly procedure to push stuff on my server but then I got exceptions while runtime about missing classes! Oh no – how can this be when I use maven?
As it turns out GeoTools requires the maven shade plugin in order to bundle the database for correct CRS transformation properly via a plugin architecture I think. And look: the whole jar is now nearly 12MB!

Conclusion

The GIS and Java world are called “enterprise” for a reason. I hope I can help others with my findings. Find the fully working code here.

Problems with nvidia driver 173 and kernel 3.2.0-52. Two monitors and card Quadro NVS 160M

After the recent kernel upgrade my old nvidia driver failed to load and X loads with a very low resolution and without detecting my second monitor. For my graphic card the nouveau driver is not an option. But I managed to fix the problems via removing current nvidia and nvidia-173. But probably you can just skip those commands and use the jockey-gtk as explained below.

sudo apt-get purge nvidia-current
sudo apt-get remove nvidia-173
sudo apt-get install –reinstall nvidia-173-updates
# now REBOOT!

Now it boots as normal with my second monitor but the nice configuration tool called ‘nvidia-settings’ did not work and says ‘The NVIDIA X driver on is not new enough to support the nvidia-settings Display Configuration page.’

Also my firefox 23 had performance problems for canvas rendering. I solved this via opening “about:config” and set layers.acceleration.force-enabled to true. I found this in the comments of this article.

Update: I was able to fix the all the issues just by switching to ‘(additional updates) (version 319-updates)’ when starting the tool called ‘jockey-gtk’! Afterwards also the tool xfce4-display-settings showed two monitors.

Java on iPhone or iPad

What options do I have to make my Java application working on the iPhone? It should not be necessary to jailbreak the phone. Also UI is not necessary for now although e.g. codenameone seems to support it. My application it not a complicated one, but uses Java 1.5 (generics etc) and e.g. memory mapped files. The good thing would be if the JUnit tests could be transformed too with the tool to check functionality.What I found is

• robovm
• code name one
• j2objc
• j2c
• xml vm
• jcgo
• outdated gcj?

Or somehow use the embedded JVM for ARM directly? Do you have experiences with this or more suggestions?

Rage Against the Android – Nearly solved! #eclipse #netbeans

After ranting against Android in my previous post I have mainly solved now the untestable situation Android is producing.

Thanks to Robolectric all my tests are finally fast and do not unpredictable hang or similar!

The tests execute nearly as fast as in a normal Maven project – plus additional 2 seconds to bootstrap (dex?). Robolectric was initially made for maven integration on IntelliJ. But it works without Maven and under Eclipse as described here. I didn’t get robolectric working via Maven under Eclipse – and it was overly complex e.g. you’ll need an android maven bridge but when you use a normal Android project and a separate Java projects for the tests then it works surprisingly well for Eclipse.

Things you should keep in mind for Eclipse:

• Eclipse and Robolectric description here
• In both projects put local.properties with sdk.dir=/path/to/android-sdk
• Android dependencies should come last for the test project as well as in the pom.xml for the maven integration (see below)

But now even better: when you are doing your setup via Maven you can use NetBeans! Without any plugin – so even in the latest NetBeans version. But how is the setup done for Maven? Read this! After this you only need to open the project with NetBeans (or IntelliJ).

Things you should keep in mind for Maven and NetBeans:

• Use Maven >= 3.0.3
• Again: put the android dependencies last!
• Specify the sdk path via bashrc, pom.xml or put it into your settings.xml ! Sometimes this was not sufficient in NetBeans – then I used ANDROID_HOME=xy in my actions
• Make sure “compile on save” is disabled for tests too, as I got sometimes strange exceptions
• deployment to device can be done to:
  mvn -DskipTests=true clean install      # produce apk in target or use -Dandroid.file=myApp.apk in the next step
  mvn -Dandroid.device=usb android:deploy # use it
  mvn android:run
• If you don’t have a the Android plugin for NetBeans just use ‘adb logcat | grep something’ for your UI to the logs

Thats it. Back to hacking and Android TDD!

3D Rotation in Gimp

1. Erstelle eine zusätzliche transparente Ebene
2. Selektiere nun die Ebene die 3D rotiert werden soll
3. Gehe zu Filter->Abbilden->Auf Objekt Abbilden
4. Wähle auf Quader abbilden
5. Klicke ‘transparenter Hintergrund’
6. Gehe zu Tab ‘Quader’. Eine Seite bekommt Ebene aus dem 1. Schritt. Alle anderen bekommen die Ebene aus dem 2. Schritt.
7. Gehe zu Ausrichtung->Rotation und verändere wie gewünscht

 

Heaven of Mergurial

The previous post about merging with named branches in mercurial was reposted at dzone. Daniel Neugebauer commented there that he wouldn’t use the named branch feature for every small issue instead he suggested to do the merging directly within the same repository and for bigger changes he would use a ‘clone per issue’. Although Fabrizio Giudici sayed that ‘branch per issue’ works really well in practise. Get his introduction!

And here I’ll quote python.org which compares clone vs. branch per issue:

Mercurial has two basic ways of using branches: cloned branches, where each branch is kept in a separate repository, and named branches, where each revision keeps metadata to note on which branch it belongs. The former makes it easier to distinguish branches, at the expense of requiring more disk space on the client. The latter makes it a little easier to switch between branches, but often has somewhat unintuitive results for people (though this has been getting better in recent versions of Mercurial).

The current proposal is to use named branches for release branches and adopt cloned branches for feature branches […]

Differences between named branches and cloned branches:

• Tags in a different (maintenance) clone aren’t available in the local clone
• Clones with named branches will be larger, since they contain more data

The Mercurial book discourages the use of named branches, but it is, in this respect, somewhat outdated. Named branches have gotten much easier to use since that comment was written, due to improvements in hg.

So, think what you want 😉

Here are the steps to do a merge from a cloned repository (clone per issue). Which is the recommended stragety for bigger changes. This merging-strategy is also covered in the official hg book.

1. The working directory is:

   app/

   Now commit the last changes.

2. cd ..;hg clone -r <oldrevisionnumber> app/ app-clone/

3. cd app-clone

   DO CHANGES (add, edit, rename, delete files)

4. hg commit -m "FIXED issueXY"

5. cd ../app;
   hg pull ../app-clone

   Now your working copy with the folder app/ contains 2 heads. This can be fixed easily via:

6. hg merge

7. hg commit -m "merged fix of issueXY into development sources"

Although this strategy needs one or two more commands it is also very clear and logically. Here is a tip from Daniel:

You can also search the history by “hg log -k <keyword>”, so if you use prefixed commit messages like “ISSUE123: …” you could find changesets belonging to that issue almost as easy as using a named branch.

And finally here are the steps to do a merge directly within the repository (direct merge), recommended for smaller changes only:

1. Working in you current repository at

   app/

   and commit the last changes

2. hg update -C <oldrevisionnumber>

3. DO CHANGES (add, edit, rename, delete files)

4. hg commit -m "FIXED issueXY"

   now hg says something about that a new head was created. Try

   hg heads

   to see that the previous commit created a new head in the repository. A

   hg update

   would fail now. But the following command resolves that ‘problem’ easily:

5. hg merge

   The working directory will now contain changes of both commits/heads!

6. Do not forget:

   hg commit -m "merged fix of issueXY into development sources"

   Now the command

   hg heads

   only contains one head

So only 5 easy steps for a quick merge!

Conclusion

All three strategies: “direct merge”, “clone per issue” and “branch per issue” are logically and easy to remember. Nice! Personally I prefer named branches. That page and this question could show its (dis)advantages.

BTW: for people looking for a git vs. hg site. Here are is one link.

Liquibase + Hibernate (annotations): Easy and solid Database Migration

As I pointed out earlier liquibase is a stable and nice migration tool for SQL databases in the Java land. The problem I had is that I couldn’t get it working with hibernate while using annotations.

Now just for my personal memory here are the steps to get it working. Download liquibase 1.9.5 (couldn’t get it working with 2.0.0 :-() and put the following libs in the liquibase/lib folder:

dom4j-1.6.1.jar
h2-1.2.137.jar #or your prefered jdbc database driver
hibernate-annotations-3.5.1-Final.jar
hibernate-commons-annotations-3.2.0.Final.jar
hibernate-core-3.5.1-Final.jar
hibernate-jpa-2.0-api-1.0.0.Final.jar
slf4j-api-1.6.0.jar

You will need to put the 4 hibernate jars into the classpath parameter, too. To do a diff and update the changelog file via the command line do the following:

liquibase --logLevel=FINE \
 --driver=org.h2.Driver \
 --classpath=$CP \
 --url=jdbc:h2:~/.myapp/h2db \
 --username=sa \
 --password=tmp \
 --changeLogFile=src/main/resources/dbchangelog.xml \
 diffChangeLog \
 --baseUrl=hibernate:src/main/resources/hibernate.cfg.xml

This means you compare the ‘old’ database with the new hibernate config. If you have problems while set-up you can look directly into the source file.

BTW: here is the pom snippet for the hibernate deps:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-core</artifactId>
   <version>3.5.1-Final</version>
 </dependency>
 <dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-annotations</artifactId>
   <version>3.5.1-Final</version>
 </dependency>

Mercurial Will Beam You in The Heaven of Merging

I didn’t understood why mercurial should be fantastic and why merging is soo much easier with hg compared to subversion. This week changed my point of view.

In subversion I couldn’t keep in mind the command to copy a revision difference e.g. into the current trunk. I nearly always was forced to do a manual and error-prone merging. I am probably too stupid for that. Now that I am using hg for some of my free time projects and even partially at work I get very comfortable with hg. Especially the fast and network independent commits are awesome when working with hg. The IDE integration – in my case NetBeans 6.X – is good although working with svn/hg on the command line is faster for me.

Then, this week, I read an article about merging and I started to understand how merging works (much easier) with hg.

Lets get started. Assume your users or your QA found an issue in your current release. In hg it is easy to fix such an issue.

1. go back to the state where you want to fix that issue or stay at the tip:

   hg update -C <oldRevisionNumber>

   Make sure you commited your changes before. Otherwise the -C won’t make backup files of changes!

2. now create a named branch for the issue:

   hg branch issueXY

   optionally do:

   hg commit -m "start working on issueXY"

3. NOW FIX THE ISSUE (add, edit, rename, delete files) in the code and commit:

   hg commit -m "fixed issueXY"

   optionally add

   --close-branch

4. Go back where you want to have the fixed code. Most of the time this will be ‘default’ (svn users known as ‘trunk’) but it also could be the releaseXY branch:

   hg update default

5. Now do the actually merging

   hg merge issueXY

6. … and commit the merge:

   hg commit -m "merged fix of issueXY into development sources"

This is very straightforward and worked very well in practice! In my case I had to merge manually 4 files, but for that kdiff3 popped up. You can change the diff application, of course and you could also use NetBeans for that. So merging was really easy and done within 2 minutes! In the fix I changed over 25 files and mercurial even recognized refactored classes (i.e. renamed files)!

I would have never thought that merging could be that easy … until I did it myself. So, try it out! Only 6 steps into the heaven of mergurial!

Hints:

• You can always go back to the issue via:

  hg update -C issueXY

• Before commiting things check that you are on the correct branch:

  hg branch

• Get all branches:

  hg branches

• View 5 commits of the logs if glog does not work for you:

  hg log -l 5

• The steps 4 until 6 could be applied to several branches, of course

UPDATE: check out also the comments of the reposting at dzone!

UPDATE: check out the new blog post with other merging strategies. (Although I prefer this one here)

UPDATE: If you want to try it yourself. Just follow these lines:

$ mkdir hg-test
$ cd hg-test/
/hg-test$ hg init
/hg-test$ echo -e 'Hello Work\nrelease 1.0' > hello.txt
/hg-test$ hg add hello.txt
/hg-test$ hg commit -m "initial release"

## released version. now further development

/hg-test$ echo -e 'Hello Work\ndev' > hello.txt
/hg-test$ hg commit -m "further dev"

## go back to released version to fix issue1

/hg-test$ hg update -C 0
1 Dateien aktualisiert, 0 Dateien zusammengeführt, 0 Dateien entfernt, 0 Dateien ungelöst

/hg-test$ hg branch issue1
Arbeitsverzeichnis wurde als Zweig issue1 markiert / working directory was marked as branch issue1

/hg-test$ echo -e 'Hello World\nrelease 1.0' > hello.txt
/hg-test$ hg commit -m "fixed to world"
neuer Kopf erzeugt / new head created

## go back to dev version and apply the issue

/hg-test$ hg update default
1 Dateien aktualisiert, 0 Dateien zusammengeführt, 0 Dateien entfernt, 0 Dateien ungelöst

/hg-test$ hg branches
issue1                         2:50560e696c8b
default                        1:10c88c9e32dd

/hg-test$ more hello.txt
Hello Work
dev

### now look how easy it is to apply the change! ###
### And only the atomic issue will change the code: 'Work' to 'World' ###

/hg-test$ hg merge issue1
merging hello.txt
0 Dateien aktualisiert, 1 Dateien zusammengeführt, 0 Dateien entfernt, 0 Dateien ungelöst
(Zweig-Zusammenführung, vergesse nicht 'hg commit' auszuführen)

/hg-test$ more hello.txt
Hello World
dev

/hg-test$ hg commit -m "merged issue1"