Last weekends Ubuntu upgrade to 18.04 LTS went mostly well, the Mapnik Python Binding problem from 17.10 has been fully fixed etc.
What didn’t go so well though was the database upgrade from PostgreSQL 9.6 / PostGIS 2.3 to 10 / 2.4.
irst of First of all the process for upgrading PostgreSQL and PostGIS to a new version at the same time is not as trivial as it could be, and requires some file name faking hacks before running the actual pg_upgrade:
Once I found that information the actual pg_upgrade ran fine, and so did the extension upgrade for PostGIS, too.
Map rendering also looked fine, but then when I tried to reactivate import of minutely diff jobs that should only take secondes, or minutes at best, just stalled, and eventually made one postgresql process run out of memory after allocating over 60GB of RAM over some 20 minutes.
https://github.com/openstreetmap/osm2pgsql/issues/846
In the end we came to the conclusion that something must already have gone wrong on the previous full planet import, somehow not causing problems on the older versions but triggering somethng strange on the more recent ones.
I dind not put much more effort into debugging this at that point, and to prevent possible spread of bad data decided to cut my losses and start a new full import right away without pre-announcing a downtime notice a week ahead of time.
Fortunately that import went well, and so does import of minutely diffs now. The system has since caught up and is less than 15 minutes behind on averaege.
I also have a gut feeling that rendering jobs are processed a little faster now, but unfortunately didn’t properly preserve timing results from test runs yet.