Database bloat has again taken its toll again, and I’m running out of SSD disk space for the main databases soon.
So I’m going to take the server down over the next weekend, staring late Friday Feb 20th, around 20:00UTC. Assuming that everything goes well everything should be back up and running on Sunday afternoon.
Background
Database bloat
After initial import the OSM database tables start to grow in disk size over time as regular diff imports are happening. This is not so much due to actual growth in OSM data size, but mostly due to table fragmentation.
After the last import I made the mistake to leave autovacuum disabled, which lead to a database growth of about 20GB per week. With autovacuum enabled this goes down to about half of that at 10GB per week. Unfortunately turning on autovacuum in November freed a bit of extra disk space, but far from what would still have been available if I had it enabled right after the import at the beginning of September (see the light green graph in this Munin report).
OSM Carto v5.3 schema changes
Starting with OSM Carto style version 5.3 a few additional database indexes need to be added, and data from shapefiles is now being imported into the database, too.
This does not really add that much of extra data to store, but I was already pretty close to running out of space due to the bloat mentioned above, and this extra data would have pushed things over the top.
Summary
Doing the re-import, and enabling autovacuum after that right away, should get the main database tablespace size on SSD back to about 75-80%, even with the extra data required for the additional indexes and the shapefile import.
Database bloat due to regular OSM diff imports will then eat about half a percent of the total SSD disk space per week, so that the next re-import would only be needed in something between nine month and a year from now.