GraphHopper Directions API Going Private Beta

Update: our Directions API is public beta now.

Today we are proud to announce that our Directions API goes into private beta. Contact us and take part to get an API key and try our latest features.

The GraphHopper Directions API includes

  • The Routing API, a fast web service to calculate world wide routes for walking, biking and car.
  • The Matrix API, based on the Routing API you can calculate so called distance matrices more efficient.
  • The Geocoding API, a world wide address search. Still under heavy development and not yet production grade although with good results in several European countries.
  • Daily OpenStreetMap updates
  • A mature service based on the open source routing engine GraphHopper. Read more about GraphHopper at opensource.com

See the Routing and Geocoding API in action at GraphHopper Maps!

GraphHopper now also Available for Offline Routing on iOS

Today we announce the first availability of GraphHopper for iOS. It is still in an experimental shape but we would like to engage people to play with it and report issues. Go directly to the git repository and continue reading.

With GraphHopper we are in the process of building a fast and open source alternative to existing routing solutions. We provide a world wide instance for car routing, biking and walking called GraphHopper Maps. Where you can see the routing engine GraphHopper in action, combined with map tiles and address search served from other software. As GraphHopper is written in Java we already made routing possible on several platforms like Linux, Windows, Mac OS X, Raspberry Pi, Android and even offline in the Browser. But still one major platform – iOS – was missing and we thought about ways to make this possible.

RoboVM?

The most simple solution to port a Java project these days is RoboVM. RoboVM makes a complete Android app running on iOS. Several users made their none-trivial games working with the help of RoboVM and so it should be also relative easy for an app which includes GraphHopper to be used on iOS too.

J2OBJC!

But we also wanted to have the possibility that a native iOS app can use GraphHopper as a library. And according to my investigations this is currently relative hard to achieve with RoboVM. So we decided to use the conversion tool j2objc which creates Objective-C code from Java code. The first running demo on an iPhone was made by Tobias which was a huge step in the right direction (pun intended) and really nice work! It showed the first time that it was possible at all and the dream of platform independence nearly came true. Still there were several glitches in the setup which I wanted to avoid in the first release. Tobias did not found enough time for this so I was seaking help in the GraphHopper community and I got in touch with Calin who recently had enough time and was polishing like crazy. Where ‘polishing’ means creating workarounds for j2objc bugs and using good old make to avoid Xcode limitations, telling me to fix things directly in GraphHopper, preparing a nice demo, finding bugs in other tools and the usual coder safari.

Success!

At the end he was able to produce the necessary scripts and adaptations to create a static library via j2objc and a simple-to-setup demo. Only very few so called compatibility classes had to be written in Java but Calin made it possible to completely avoid custom Objective-C code, even for the more advanced things like memory mapping. With that we can almost always automatically create the updated version out of ‘GraphHopper Java’. And GraphHopper on iOS is fast, for example a route through entire Germany takes only about 1sec on iPhone 6!

We again ask for you help and feedback!

And thanks again to Tobias & Calin!

The Flexibility of GraphHopper

I often hear some misconceptions about the flexibility of GraphHopper. In this post I speak about GraphHopper core.

Flexibility Mode

GraphHopper is designed to be fast and flexible. For example you can route through entire Germany in about 1 second on average without any speed-up method, I’ll call this ‘flexibility mode’. You have to keep in mind that this involves traversing millions of nodes in the road network. In this mode you have all possibilities and advantages like on-demand profiles, small base graph, support for multiple vehicles and so on. A naive implementation for OpenStreetMap data probably won’t reach this query speed, even if coded in C++. Additionally we fight with the memory waste and garbage collection in Java.

Speed up Mode

The main disadvantage for the flexibility mode is that long queries will need lots of RAM, which is also not very handy on mobile devices. So you’ll have to limit the length of the route, increase the heuristical nature of the algorithm OR which is our default mode: switch to the speed up mode which uses a special algorithm called contraction hierarchies. The speed up mode comes with disadvantes e.g. only one query profile is possible or a bigger base graph. But it dramatically reduces the necessary RAM per query and as a nice side effect makes the query 50-500 times faster, depending on the length of the route. These are the reasons that the default mode for GraphHopper is the speed up mode. (And our Directions API is provided only in this speed up mode.) With GraphHopper switching between modes is just a configuration change (chShortcuts=fastest or chShortcuts=no) and our architecture was build with this in mind and is open for new and completely different speed-up algorithms. The benefit of this simplicity is that you can play around and tune the routing of one or more vehicles with various options in the flexibility mode, and then if you need performance or want to use it on mobile devices you can switch to the speed up mode.

Digitalizing GPX Points or How to Track Vehicles With GraphHopper

Recently I was asked again if I know a method, or even better a tool, which makes GPX points more align to the existing streets in OpenStreetMap. E.g. currently in Mapilarry all recorded streets are not really smooth and digitalized if you look at them in detail. GraphHopper cannot do this out of the box but provides a toolchain which will make this digitalization easier.

Update: this is now open source.

Map Matching

The necessary process before an import would be Map Matching. Quoting (my own words ;)) from Wikipedia:

Map matching is the process of associating a sorted list of user or vehicle positions to the road network on a digital map. The main purposes are to track vehicles, analyze traffic flow and finding the start point of the driving directions.

Or as a picture:

map-matching

I’m explaining in this blog post the simple steps which can taken to implement something like this with a very simple method. This is not an invention by me, other papers suggesting a similar technique. Also we should use some open testing data additionally to the normal unit tests. Some wordings are necessary before we go deeper:

  • A GPS point is a tuple or object which contains latitude, longitude (, elevation) at which position it was recorded.
  • A GPX point is a tuple or object which contains latitude, longitude (, elevation) and the timestamp at which it was recorded. And a GPX list ist a list of GPX points.
  • In the digital world of maps we need a data structure to handle the road network. This data structure is a graph where the junctions from real world are nodes and the streets between them are the edges.

The question is also why would we need a routing engine at all?

Because:

  • firstly you don’t need to write the import process of OpenStreetMap and the closest edge lookup and probably some other tools necessary for map matching and
  • secondly the routing process will fill gaps between larger distances of two GPX points, and will be able to guess those gaps depending on the vehicle and
  • thirdly using a routing engine will make the outcome more realistic, e.g. avoiding going left and then returning the identical street to go further in the straight direction.
  • Lastly with GraphHopper you’ll be easily able to provide such a procedure even for the world wide case as it stores everything very compact and can be used in-memory or on-disc.

We start with an example: example1a

All maps are from Lyrk and data is from OpenStreetMap contributors

In the left map you see the street parts highlighted where only the edges closest to the GPX points are selected. Whereas in the right map the most probable way was taken via a routing engine like GraphHopper.

The simple algorithm

The results for the map on the left side are easy to get, see the gist for example 1. For the map on the right side can try the following procedure explained in pseudocode here or see example 2 as gist for a more complete one:

  1. Find 3 closest edges per GPX point
  2. Associate every edge with a weighting, which is the distance from the edge to the point. The larger the distance the smaller is the probability that the best path goes through it.
  3. Find the best path from the first GPX point to the last, but take into account the weighting and therefor the distance to the GPX points
  4. As a post-processing step you need to assign the GPX points to the resulting path, you can do so via the edge ids and to find the coordinates you can either use path.calcPoints or the lower level API ala edgeState.fetchWayGeometry.

This simple algorithm should work in the most cases and is very fast

The enhancements

There are two limitations of this simple algorithm:

  • If there are loops in the GPX list, then the Dijkstra from step 3 will throw data away
  • and it will happen that another route than the route marked with GPX points is taken. E.g. in the case if the GPX signal is not that frequent and the edge of one point is rather short, which means it will have a small influence on the overall weighting of the taken route.

The following improvements are possible

  • You could do a simple workaround and cut the incoming GPX list when you detect a loop and feed those GPX-list-parts to the algorithm.
  • Or find the best route not from start to end but ‘step by step’. For example from pointX to pointX+4, then pointX+4 to pointX+8 and so on, where you need to merge the results. To avoid suboptimal paths at the intersection (here pointX+4) you’ll need to ‘overlap’ the routes. So you will need to calculate the routes for pointX to pointX+4, then pointX+3 to pointX+7, …. instead. Merging will be more complex probably.
  • A more complex solution could be to maintain a list of e.g. 3 alternative end-results. As you already get 3 edge candidates for every GPX point you can then try to find all routes between them. You could think that this will slow down the search a lot but with GraphHopper you can tweak e.g. the Dijkstra or DijkstraOneToMany algorithm to do this in an efficient manner. E.g. if you want to get the best result but you want to search from multiple destinations you can do: DijkstraBidirectionRef dijkstra = new DijkstraBidirectionRef(hopper.getGraph(), flagEncoder, weighting); dijkstra.initFrom(nodeFrom, distanceToPossibility1); dijkstra.initFrom(node, distanceToPossibility2); … dijkstra.initTo(nodeTo, distanceToEndPossibility1); dijkstra.initTo(nodeTo, distanceToEndPossibility2); …

There are lots of other problems which you’ll encounter, that is the ugly real life with GPS signal errors, tunnels and so on. One example problem is if you are tracking e.g. a truck waiting in front of traffic lights in a one way street. You’ll see several nearly identical GPX points but as they have an error associated they won’t be identical and it could happen that they go ‘backwards’ in the one-way which will result in an extra loop if use the ‘step-by-step’ variant of the shortest path calculation:

example2

Languator: Language Detection for Local Queries

When you have an address lookup service like photon you need language detection for the short local queries like “Berlin Alexanderplatz” to e.g. apply a specific analysis chain. Of course German language is rather simple, but detecting if it is French or Italian etc is much more complex.

I tried the language-detection from google code. And although it has high precision for normal text, it is relative slow (creating the profile and querying) and not specific to local queries although short generic queries should work (e.g. Twitter data). Another problem is the complex and inflexible Java API as it uses lots of statics etc. E.g. the size of the n-grams is not configurable.

So I hacked together a new language detection called languator in Java specific for short local queries and fed it with OpenStreetMap data. E.g. I used the germany.pbf to create the German language profile and france.pbf for the French profile and so on. If you keep in mind that you have lots of foreign names for POI and street in every country the following error values are okayish and better than what I found from other tools:

English: 22.9%, German: 7%, French: 20%, Italian: 18.8%

Another good thing is that you can feed languator with these 4 countries in 20 minutes and create millions of queries from those countries which takes ~10 minutes, which is at least 2 times faster than the google code tool.

I invite you to beat my numbers: fork and have fun!

Releasing GraphHopper 0.3 – Plan Your Outdoor Trips Beyond Two Dimensions

Today we are happy to finally release version 0.3 of GraphHopper – the Open Source road routing engine. Here is a screenshot showing our slightly polished UI

gh-screenshot-0.3

It shows a route result where elevation was enabled. Try it yourself at GraphHopper Maps or see the variety of already implemented use cases!

Highlights:

  • You can easily enable elevation and Graphhopper will not only automatically import and display this, but you can actually use it for routing! E.g. for bike routing hills will be avoided. But also even for horse-routing.
  • GraphHopper runs on Raspberry Pi and even in the Browser!
  • GraphHopper is now available in 19 languages! See here for the details.
  • A more powerful and compact API, including GPX support with turn instructions and elevation information.
  • The address suggestions is done via photon and is available as part of our Web Routing API which is currently in closed alpha.

Thanks a lot to all contributors: NopMap, ratrun, konsoletyper, dardin88, lmar and b3nn0. And all the translators!

Details:

  • You can store two weights per edge (dependent on the direction) – this allows using GraphHopper for interesting new use cases:
    • Biking and hiking tours can now avoid hills as up- and down-hill speed can be different.
    • With traffic data you can make routing more precise as speed values has to be different for the two directions.
  • To make use of this new feature we implemented to read elevation data. For now only hgt files are supported but more will follow. This makes GraphHopper different to many routing engine, especially because we can scale with elevation to world wide coverage (although elevation data is only available for +-60°).
  • The elevation Leaflet widget from MrMufflon is very useful and has a nice dragging feature
  • Support to read OSM relations like for biking routes and overall improved bike routes as it is easier to tweak them
  • Support for more than 2 points – still we need an improved UI to use this feature
  • A new web API format which improves compactness and introduces the new but optional 3rd dimension. Additionally it makes it possible to retrieve the sub-path geometry for every turn instruction. See the documentation.
  • Upgraded the Android demo to the latest mapsforge 0.4.3, and a further fix of an ugly bug
  • Support to make it possible to click on instructions and see them on the map. Additionally navigation software can now easier find the next instruction
  • Lots of the documentation is now in the source, which is better for the user to have up-to-date information. Or where it is possible to look at old docs too.
  • and a lot more bug fixes

Last but not least some months ago there was an article about GraphHopper in the Java Magazine and a session at FOSDEM. Did I mention that you should check out our enterprise offers ;) ? Feel free to join our mailing list or follow us!

Have fun!

GraphHopper in the Browser: TeaVM makes Offline Route Planning possible in JavaScript!

Today I can make an exciting news.

Alexey Andreev is the author of a Java-bytecode to Javascript compiler called TeaVM. You may know GWT or Scala.js but TeaVM does not compile the sources, it compiles the bytecode to JavaScript so you cannot only use Java as the source language. Of course other advantages exists.

Now after a chat with him and lots of work (he did), he finally reached the goal to run GraphHopper in the Browser, either via a normal bidirectional Dijkstra or a fast algorithm called Contraction Hierarchies – see below for a demo!

But what would be a use case?

You might think: “Boooring! Yet another project running in JavaScript!” – Well, far away from boring! Think about FirefoxOS which currently has no offline navigation. About a year ago I had a discussion with (probably) some FirefoxOS guys on hackernews about this topic. I did not really believed that something like this could be possible at that time, but now I’m really proud that I can say that GraphHopper with TeaVM is one step in this direction!

Another use case could be a hybrid solution with GraphHopper in the back end and GraphHopperJS on the client. So this opens up new possibilities!

There were lots of problems.

One of the problems Alexey solved already months before the GraphHopper stuff was the missing ‘long’-type support in JavaScript. TeaVM emulates this via a ‘lo’ and ‘hi’ structure. But this is rather simple compared to other tricks he had to apply.

Another, not yet really solved, problem is the file-access: how would you load the graph only partially into memory? In Java I can just use memory mapping if there is only a few RAM available. But is there memory mapping in the browser from JS? (Unanswered question on stackoverflow for over a week now). Currently Alexey stores the data as JSON in the html file, but this is too inefficient and wastes too much memory. Hopefully we can find a better solution – maybe IndexedDB could be part of this solution.

Why is this only the first step towards an offline Navigation?

For offline navigation you will need geocoding which turns address strings into coordinates and also offline maps:

  • For geocoding we would need a search engine like Lucene (Java) or something like my prototype of a JavaScript search engine called JSii (now I finally know a use case for this ;)). Additionally we need data, maybe consume OSM data from Nominatim or if it is just a city feed the streets directly to an index – this should be very easy.
  • Offline maps is not that hard because you could use data from GraphHopper itself to print the map for the current nearby area, although this will be inefficient for larger areas it will be sufficient for most navigation applications. There are already some powerful libraries for in-browser rendering available – see here.

Try GraphHopper in the Browser!

Warning: this will load the routing data for Greater London ~14MB and can take a while!

gh-teavm

If you zoom a bit in and out to cache the tiles, you can disconnect from the Internet and calculate offline routes anywhere! This should also work for mobile phones, of course calculating will be ‘slightly’ slower as Alex reported from 10x slower and even worse: on my old phone every browser I tried even crashed. Another minor bug is when you click outside of the routing area the app will stop working.

Get Started

Build teavm

  1. git clone TeaVM
  2. mvn clean package

Build GraphHopperJS

  1. git clone modified GraphHopper
  2. mvn clean package -DskipTests=true
  3. unzip target/graphhopper-teavm-0.3-SNAPSHOT-teavm.zip
  4. mv graphhopper-teavm-0.3-SNAPSHOT graphhopper-teavm
  5. Create a gh-directory.js via the GraphhopperJsonGenerator class and specify “<your-osm-file> gh-directory.js” as command line arguments. TODO: Uncomment the query part as this requires to have moscow area.
  6. cp gh-directory.js graphhopper-teavm
  7. firefox index.html (don’t enable firebug as this slows down everything!)

Next steps

  • I’ll try to integrate the changes Alexey did into GraphHopper for the next release.
  • Currently this is just for standalone usage and not accessible from “outside JavaScript” (already done via TeaVM @PreserveOriginalName ?)
  • Finally a fully navigation experience with geocoding and offline maps based on this work would be interesting.

Similar Projects

The only offline route planner in JavaScript I found was a public transport library called localroute, but not sure if it also works in the Browser. Let me know if there are similar projects as it is really hard to search for Routing & JavaScript ;)

Thanks!

A huge thanks to Alexey for making this possible – don’t forget to look into TeaVM !