Releasing GraphHopper 0.3 – Plan Your Outdoor Trips Beyond Two Dimensions

Today we are happy to finally release version 0.3 of GraphHopper – the Open Source road routing engine. Here is a screenshot showing our slightly polished UI

gh-screenshot-0.3

It shows a route result where elevation was enabled. Try it yourself at GraphHopper Maps or see the variety of already implemented use cases!

Highlights:

  • You can easily enable elevation and Graphhopper will not only automatically import and display this, but you can actually use it for routing! E.g. for bike routing hills will be avoided. But also even for horse-routing.
  • GraphHopper runs on Raspberry Pi and even in the Browser!
  • GraphHopper is now available in 19 languages! See here for the details.
  • A more powerful and compact API, including GPX support with turn instructions and elevation information.
  • The address suggestions is done via photon and is available as part of our Web Routing API which is currently in closed alpha.

Thanks a lot to all contributors: NopMap, ratrun, konsoletyper, dardin88, lmar and b3nn0. And all the translators!

Details:

  • You can store two weights per edge (dependent on the direction) – this allows using GraphHopper for interesting new use cases:
    • Biking and hiking tours can now avoid hills as up- and down-hill speed can be different.
    • With traffic data you can make routing more precise as speed values has to be different for the two directions.
  • To make use of this new feature we implemented to read elevation data. For now only hgt files are supported but more will follow. This makes GraphHopper different to many routing engine, especially because we can scale with elevation to world wide coverage (although elevation data is only available for +-60°).
  • The elevation Leaflet widget from MrMufflon is very useful and has a nice dragging feature
  • Support to read OSM relations like for biking routes and overall improved bike routes as it is easier to tweak them
  • Support for more than 2 points – still we need an improved UI to use this feature
  • A new web API format which improves compactness and introduces the new but optional 3rd dimension. Additionally it makes it possible to retrieve the sub-path geometry for every turn instruction. See the documentation.
  • Upgraded the Android demo to the latest mapsforge 0.4.3, and a further fix of an ugly bug
  • Support to make it possible to click on instructions and see them on the map. Additionally navigation software can now easier find the next instruction
  • Lots of the documentation is now in the source, which is better for the user to have up-to-date information. Or where it is possible to look at old docs too.
  • and a lot more bug fixes

Last but not least some months ago there was an article about GraphHopper in the Java Magazine and a session at FOSDEM. Did I mention that you should check out our enterprise offers ;) ? Feel free to join our mailing list or follow us!

Have fun!

GraphHopper in the Browser: TeaVM makes Offline Route Planning possible in JavaScript!

Today I can make an exciting news.

Alexey Andreev is the author of a Java-bytecode to Javascript compiler called TeaVM. You may know GWT or Scala.js but TeaVM does not compile the sources, it compiles the bytecode to JavaScript so you cannot only use Java as the source language. Of course other advantages exists.

Now after a chat with him and lots of work (he did), he finally reached the goal to run GraphHopper in the Browser, either via a normal bidirectional Dijkstra or a fast algorithm called Contraction Hierarchies – see below for a demo!

But what would be a use case?

You might think: “Boooring! Yet another project running in JavaScript!” – Well, far away from boring! Think about FirefoxOS which currently has no offline navigation. About a year ago I had a discussion with (probably) some FirefoxOS guys on hackernews about this topic. I did not really believed that something like this could be possible at that time, but now I’m really proud that I can say that GraphHopper with TeaVM is one step in this direction!

Another use case could be a hybrid solution with GraphHopper in the back end and GraphHopperJS on the client. So this opens up new possibilities!

There were lots of problems.

One of the problems Alexey solved already months before the GraphHopper stuff was the missing ‘long’-type support in JavaScript. TeaVM emulates this via a ‘lo’ and ‘hi’ structure. But this is rather simple compared to other tricks he had to apply.

Another, not yet really solved, problem is the file-access: how would you load the graph only partially into memory? In Java I can just use memory mapping if there is only a few RAM available. But is there memory mapping in the browser from JS? (Unanswered question on stackoverflow for over a week now). Currently Alexey stores the data as JSON in the html file, but this is too inefficient and wastes too much memory. Hopefully we can find a better solution – maybe IndexedDB could be part of this solution.

Why is this only the first step towards an offline Navigation?

For offline navigation you will need geocoding which turns address strings into coordinates and also offline maps:

  • For geocoding we would need a search engine like Lucene (Java) or something like my prototype of a JavaScript search engine called JSii (now I finally know a use case for this ;)). Additionally we need data, maybe consume OSM data from Nominatim or if it is just a city feed the streets directly to an index – this should be very easy.
  • Offline maps is not that hard because you could use data from GraphHopper itself to print the map for the current nearby area, although this will be inefficient for larger areas it will be sufficient for most navigation applications. There are already some powerful libraries for in-browser rendering available – see here.

Try GraphHopper in the Browser!

Warning: this will load the routing data for Greater London ~14MB and can take a while!

gh-teavm

If you zoom a bit in and out to cache the tiles, you can disconnect from the Internet and calculate offline routes anywhere! This should also work for mobile phones, of course calculating will be ‘slightly’ slower as Alex reported from 10x slower and even worse: on my old phone every browser I tried even crashed. Another minor bug is when you click outside of the routing area the app will stop working.

Get Started

Build teavm

  1. git clone TeaVM
  2. mvn clean package

Build GraphHopperJS

  1. git clone modified GraphHopper
  2. mvn clean package -DskipTests=true
  3. unzip target/graphhopper-teavm-0.3-SNAPSHOT-teavm.zip
  4. mv graphhopper-teavm-0.3-SNAPSHOT graphhopper-teavm
  5. Create a gh-directory.js via the GraphhopperJsonGenerator class and specify “<your-osm-file> gh-directory.js” as command line arguments. TODO: Uncomment the query part as this requires to have moscow area.
  6. cp gh-directory.js graphhopper-teavm
  7. firefox index.html (don’t enable firebug as this slows down everything!)

Next steps

  • I’ll try to integrate the changes Alexey did into GraphHopper for the next release.
  • Currently this is just for standalone usage and not accessible from “outside JavaScript” (already done via TeaVM @PreserveOriginalName ?)
  • Finally a fully navigation experience with geocoding and offline maps based on this work would be interesting.

Similar Projects

The only offline route planner in JavaScript I found was a public transport library called localroute, but not sure if it also works in the Browser. Let me know if there are similar projects as it is really hard to search for Routing & JavaScript ;)

Thanks!

A huge thanks to Alexey for making this possible – don’t forget to look into TeaVM !

 

GraphHopper News: Article in Java Magazine and FOSDEM 2014

Today the new Java Magazine from Oracle was published. You’ll need to register to view it or download one of the apps (Android, iOS). In this latest magazine my article about GraphHopper was published. It is about how I solved some big (data) problems in Java as well as some OpenStreetMap things to know and more!

If you are interested to hear more about this you can visit FOSDEM in Brussels (2nd February) where I talk about GraphHopper.

Update: Get the slides here!

screenshot-javamag

GraphHopper an Open Source and Flexible Alternative to Google Maps API

Update: No need to install and maintain GraphHopper on your own expensive hardware. Sign up to the GraphHopper Directions API for business which makes it very easy, cost effective and reliable to consume GraphHopper routes with always up-to-date software and OpenStreetMap data.

On Monday we released version 0.2 of GraphHopper an Open Source road routing engine. GraphHopper is comparable fast to Google or Bing Maps, but if you use it in a LAN or directly from within your application it is a lot faster as it avoids the round trip over the internet, but furthermore if you need one-to-many or even many-to-many routes then GraphHopper is orders of magnitude faster. In this blog entry I’ll show you how to get everything you need to setup GraphHopper on your own servers. Hosting GraphHopper on your own servers also means: you can query it as much as YOU like, currently this means 300 to 600 queries per second which is dependent on your hardware, of course. This is measured from real life car routes larger than 100km. Now, as you’ll set up a fault tolerant system with at least two servers this means you can put a load of over 100 mio requests per day on your system. This is 1000 times more than the limit of the Google Maps Business API and again: for one/many-to-many the benefits are even larger.

1. Install GraphHopper

Do so, either via our quick start docs or directly from the source which is my preferred way as you can easily stay up-to-date and try out minor code changes a lot faster. Although it will require that you’ve installed git and the JDK:

$ git clone git://github.com/graphhopper/graphhopper.git
$ cd graphhopper
$ ./graphhopper.sh web europe_germany_berlin.pbf

You should see this console output: console-berlin-gh It will start a jetty server on port 8989 – so if no exception occured you can go to http://localhost:8989 and if you clicked on two positions you should see the route for a car (default – can be changed in config.properties): browser-berlin-gh

2. World Wide Routing Service

I assume you need car, pedestrian and bicycle routing for your personal world wide routing service. For this routing service you need two runtime servers (to make the system fault tolerant) and for each at least 52GB RAM. Additionally you need one ‘smaller’ preparation server with at least 32GB of RAM to prepare the graph for fast routing – you only need it for updating the OpenStreetMap data e.g. once a week. If you want to avoid this overhead then ask us as we already GraphHopper files regularly for world wide coverage. Site notes:

  • In Germany there is at least one provider where you get those three servers for under 250€ per months.
  • If you cannot effort 2*64GB servers you could have a look into the memory mapped settings but this is slower – maybe not that slow via SSD, not yet deeply benchmarked.
  • If you don’t need the speed up, e.g. if your routes are rather small or you want a personalizable request then you ‘only’ need 2*12GB of RAM. Or less for the memory mapped case, which could even make it suitable for world wide routing on your 64bit Desktop! (Okay, still you need the geocoding and tiles, but that is for another blog post ;))

World Wide Download

On the preparation server download the OpenStreetMap data for the whole world e.g. from here. Then do

 ln -s planet-latest.pbf planet-car.pbf
 ln -s planet-latest.pbf planet-bike.pbf
 ln -s planet-latest.pbf planet-foot.pbf

Import and Prepare

Still on the preparation server import and prepare the GraphHopper folders for car

 export GH_WEB_OPTS="-Dgraphhopper.osmreader.acceptWay=CAR"
 export JAVA_OPTS="-XX:PermSize=100m -XX:MaxPermSize=100m -Xmx28000m -Xms28000m"
 ./graphhopper.sh import planet-car.pbf

Do the same for bike and foot . This process takes 1h for the import and 2 to 5h for the preparation – depending on the vehicle.

3. Move to Leaflet and GraphHopper

Finally your client code needs to move away from Google Maps. Have a look into this description how to handle leaflet in general and into our own code how to handle the routing. Vote +1 for this issue to avoid this work :). Additionally to JavaScript we also provide a pure Java API useful for in-app communication.

Conclusion

Although GraphHopper is not yet feature complete it can already offer a production ready road routing service which is also easy to set up. Furthermore GraphHopper won’t take high administration costs as it is already very robust and used under high load in some companies for several months. Last but not least GraphHopper is easy to customize and can satisfy a wide range of different business needs.

Releasing GraphHopper 0.2 – Further & Faster Road Routing!

Today we’re releasing version 0.2 of our Open Source road routing engine GraphHopper written in 100% Java.

Faster!

  • All algorithms are faster due to bug fixes and fine tuning
  • A preparation is necessary for our optional speed-up technique called Contraction Hierarchy. This preparation is also faster.

Further!

  • We finally fixed GPS-exact routing so you don’t have to workaround a junction-to-junction results

More exciting news will follow …

Have fun and try GraphHopper Maps with world wide coverage for pedestrians, cars and bicycles! You need support? Have a look at our enterprise options!

Notes:

Free the Java Duke now and use it in Blender!

Duke, the Java mascot is a well known ambassador for Java. It is so important even Oracle still has a prominent site for it. The only problem is that the original source files can only be opened with a commercial application called lightwave3d.

For GraphHopper I took those files and created an OpenStreetMap variant with lightwave3d:

osm_duke

But I wondered why there is no possibility to use the files in Blender. Actually there is a plugin for Blender from Remigiusz to import the DXF files, but at the beginning it did not work for those none-standard files. So I contacted Remigiusz and the story begins. He not only improved his importer to make it working, but he invested hours to make a really nice Blender version for the Java Duke files!

Have a look:

duke

The files are at Github! Thanks a lot again to Remigiusz!

Setup Mapnik From Scratch

THIS document is a work in progress.

There are several options but mainly three web map servers (WMS): Mapnik, GeoServer and MapServer. Simple visualization:

 A) browser/client (leaflet, openlayers)
 |
 B) tile server (mod_tile, tile cache, tile stache, mapproxy, geowebcache)
 |
 C) map web service = WMS (MapServer, GeoServer, Mapnik)
 |
 D) Data storage (PostgreSQL, vector tiles)
 |
 E) OSM data (xml, pbf)

Comments

  • for C => Mapnik can use TileMill to style to map
  • leaflet can do tile layers (B) but also WMS (C)
  • Nearly always you need PostgreSQL, but in rare cases you can avoid it via vector tiles.
  • A common approach is to use apache2 with mod_tile. Serving the tiles from disc or creating the image via mapnik through renderd. But also nginx gets more popular. Tiledrawer has an old scripts also with mapnik and nginx.
  • You can also use GeoServer with a cache in front. Often it only serves some feature layers.
  • WFS = web feature service
  • MWS = map web service

Installation of PostgreSQL, Mapnik, mod_tile, renderd

PostgreSQL

First of all you need the OSM data in a PostgreSQL database via osm2psql import – an easy description is here. For a faster import you can try imposm instead of the more common osm2psql.

Install the Rest

Although nginx would be preferred I did not find a out of the box working solution for it. So you need to use apache2 and mod_tile: a simple installation. Also there is an older article which I did not try. There is also a good presentation with some useful information.

 sudo apt-get install python-software-properties
 sudo add-apt-repository ppa:kakrueger/openstreetmap
 sudo apt-get update
 # now install mod_tile and renderd with the default style
 sudo apt-get install libapache2-mod-tile
 sudo -u postgres osm2pgsql --slim -C 1500 --number-processes 4 /tmp/europe_germany_berlin.pbf
 sudo touch /var/lib/mod_tile/planet-import-complete
 sudo /etc/init.d/renderd restart
 # optionally pre-generate some tiles for faster rendering - see "How do I pre-render tiles ?"
 # difference to generate_tiles.py and generate_tiles_multiprocess.py?
 go to http://localhost/osm/slippymap.html
# to install mapnik-config do:
 sudo apt-get install libmapnik2-dev

Custom Styles

Here you can find newer style is constructed via CartoCSS
install carto either via just using TileMill or do

sudo apt-get install npm
sudo npm install -g carto

TODO Get mapbox osm-bright working. This new style requires a specific version XY of mapnik which wasn’t found in my ubuntu version!?

cp configure.py.sample configure.py
edit config name and path in configure.py
./make.py
# backup /etc/mapnik-osm-data/osm.xml before doing:
carto osm-bright/YourProject/project.mml > /etc/mapnik-osm-data/osm.xml
# the osm.xml is used from mod_tile which uses renderd /etc/renderd.conf
# mod_tile config is at: /etc/apache2/sites-available/tileserver_site
# The tiles are here: /var/lib/mod_tile/default/

TODO To update the style you can try

 sudo -u www-data touch /var/lib/mod_tile/planet-import-complete
 # important: do not delete 'default' itself
 sudo rm -R /var/lib/mod_tile/default/
 # how to restart renderd properly?
 sudo service renderd restart

OLD STUFF:

Mapnik

For mapnik installation you’ll also need a web server like nginx or apache2. For nginx you can have a look into the script available here (clone the git repo):

http://tiledrawer.com

Additionally you’ll need mod_tile and renderd or TileStache if you use nginx. To style the maps you can use TileMill.

GeoServer&GeoWebCache

Installation is a lot easier compared to Mapnik or MapServer. Only dependency is Java. Just grab the full zip of geo server and copy the war from web cache into webapps, then run ./bin/startup.sh

If you need mapbox tilemile styles for geoserver have a look into:

MapServer

http://dotrem.blogspot.ru/2013/06/mapservernginxfastcgi-on-ubuntu.html