OSM Baustelle

I will have to take the service offline for a day or two again, starting ca. Friday, Nov. 20th 18:00 UTC, for two reasons:

* Database size creep – I’m starting to get close to disk size alert limits again as OSM diff imports let the database size grow ever so slightly not only due to additional data, but also due to fragmentation issues. A re-import of the full planet data from scratch will reduce the database size, but will also take at least a full day

* Upgrading the server from Debian 12 “Bookworm” to 13 “Trixie”, for several reasons. One of them being that the most recent Mapnik renderer version with ArcSymbolizer support for the SeaMap overlay can only be built on “Trixie”, some build tools on “Bookworm” being too old for that.

After way too many years in the making the Mapnik renderer is now finally able to draw circle arcs, with and without radius lines and textual labels. I had been trying to implement this several times over the years, but always got stuck on details like how to use the agg render backend — who’s author unfortunately as deceased and can’t be asked any more 🙁

Now with a little help from a companion named Claude I was able to complete that work (minus some final touches), and the code got accepted into Mapnik now, so it will be in future Mapnik releases.

See also e.g. https://github.com/mapnik/mapnik/pull/4589

And with that the one missing piece that prevented me from offering an OpenSeaMap-like style for printing is now in place (OpenSeaMap always used their own home grown rendering solution instead of Mapnik / CartoCSS).

So I have no also started to build a full OpenSeaMap-lookalike overlay style using the new Mapnik ArcSymbolizer feature.

See also https://github.com/hholzgra/mapnik-seamap-overlay

I’m not fully done yet, there are still some aspects of the OpenSeaMap style missing, most importantly all buoys and beacons still miss their coloring and their top markers; only the base shapes are rendered for now.

But I’m making good progress, and hope to finish this by Christmas latest, so that we can offer to print finished OpenSeaMap-like maps for the first time on the 40C3 Chaos Communication Congress in Hamburg at the end of this year.

With the event happening near the largest German harbor this seems like the perfect occasion …. 🙂

For now the SeaMap overlay style is already available on print.get-map.org, under “Special Interest” overlays, but still marked as experimental.

This service has now been running for almost ten years, and last week it crossed the 500 000 rendering job threshold.

The 500 millions job was this multi page map of Philadelphia:

https://print.get-map.org/maps/500000

Hetzner has replaced the previous machine, keeping only the original SSDs, so far there have not been any further hangups for the last ca. 36 hours. Will continue to monitor, but things look good so far.

The server hung up again last night, with no invalid SSH login attempts logged since yesterdays changes; so these do not seem to have been the culprit after all 🙁

Also the frequency of events has increased, the machine got stuck several times today.

Now checking with the hosting company whether they can identify a hardware fault …

The server actually ran fine again for almost a year, to now have just stopped three times within the last 30 days.

What the three recent failures had in common was that each time the last logged message was the first line of a two-line failed SSH login attempt message.

I have now taken a few steps to harden the SSH setup, and will lock it down even further as soon I’m back home next week to only accept requests from my fixed home IP address.

Now anxiously waiting for the next failure to happen — or better not happen — to see whether this really fixed things, or whether there’s actually a different problem and the SSH log messages only having been a coincident …

The server got stuck twice this month without logging any reason, and had given a hardware related warning earlier during the previous maintenance at the start of this month.

I opened a service request with the server hosting company, and was offered to either replace the server, but keep the disks; or to replace both which would require me to take care of backing up everything, and then setting up the new machine from scratch.

I tend towards the later, but am not sure yet when I’ll have sufficient free time to take care of that, maybe not before the last weekend in May …

Update: the below should hopefully also fix the problem with PDFs not being processed correctly while other file formats are being created just fine (affecting about 1% of the rendering jobs right now)

It is time for some maintenance again, I’m planing to do a complete re-import of the OSM planet data on the first weekend in April, starting in the late European afternoon on Friday, April 4th.

It is not going to be a complete re-install this time, I should be able to keep most things up for most of the time, but expect some map styles to be unavailable at different times throughout the weekend.

It should still be able to file render requests for all styles during the whole weekend, minus a few minutes here and there while restarting updated services, but actual rendering of submitted jobs can be paused until re-import is complete.

would you please get in touch with me via hartmut@get-map.org ?

I see some 40+ failed attempts now, so I’d like to discuss with you what you are trying to achieve and to find out why your geoJSON files are not being accepted by the renderer.

I’m going over the different map styles installed on this service right now, to get those that are still actively maintained up to the most recent version.

So far I have updated:

• the “Baumkarte” style – an OSM carto variant focused on detail information on trees and allotment sites at high zoom levels
• the Belgian variant of the OSM carto style which had been re-done completely

Next to follow are:

• the CyclOSM style
• the OpenOrienteering styles
• the OpenStreetMap Carto standard styles
• …