Feature #2343
openUpdate OpenAtlas software to Debian 13 (Trixie)
Description
This issue provides information about the update and the new software versions being used with OpenAtlas, see comments for more details.
Branch: feature/trixie
- Python from 3.11.2 to 3.13.5 (changelog 3.12, 3.13)
- Flask from 2.2.2 to 3.1.1 (changelogs)
- PostgreSQL from 15 to 17 (changelogs)
- Jinja2 from 3.1.2 to 3.1.6 (changelogs)
- Backup everything, and specific for OpenAtlas:
- Your databases, e.g. with pg_dumpall for all PostgreSQL databases
- The folders at files/uploads for every instance
- Update current system
sudo apt update && sudo apt full-upgrade -y - Change all occurrences of
bookwormtotrixiein /etc/apt/sourcessudo apt edit-sources - Upgrade - when prompted to upgrade PostgreSQL, choose No – the database will be upgraded manually later.
sudo apt update && sudo apt upgrade --without-new-pkgs sudo apt update && sudo apt upgrade --with-new-pkgs sudo apt update && sudo apt dist-upgrade - Clean up
sudo apt --purge autoremove -y sudo apt autoclean - Reboot the system
- PostgreSQL cluster upgrade
As postgres user:sudo apt install postgresql-postgis sudo systemctl stop postgresql sudo pg_dropcluster 17 main sudo pg_upgradecluster 15 main sudo pg_dropcluster 15 main sudo systemctl start postgresqlreindexdb --all
Updated by Alexander Watzinger 6 months ago
- Precedes Feature #2093: API: New and faster object serialization added
Updated by Bernhard Koschiček-Krombholz 2 months ago
Work in Progress Upgrade List¶
Major package upgrades:- flask (*)
- flask-babel (*)
- pandas
- flask-cors
- rdflib (*)
- email-validator
- shapley (*)
- pylint
(*) will definitively break OpenAtlas 8.x.x instances
Debian 12 → 13 (Bookworm → Trixie) Upgrade Notes¶
Previous upgrade issue: #2038
Steps Performed¶
Update current system¶
sudo apt update && sudo apt full-upgrade -y
Adjust APT sources¶
sudo apt edit-sources
Change all occurrences of bookworm to trixie.
Refresh and perform safe upgrade¶
sudo apt update && sudo apt upgrade --without-new-pkgs
When prompted by rkhunter, choose N (keep installed settings).
Continue upgrade¶
sudo apt update && sudo apt upgrade --with-new-pkgs
When prompted to upgrade PostgreSQL, choose No – the database will be upgraded manually later.
Clean up unused packages¶
sudo apt --purge autoremove -y
sudo apt autoclean
Reboot system after completion¶
sudo reboot
PostgreSQL cluster upgrade¶
sudo apt install postgresql-postgis
sudo systemctl stop postgresql
sudo pg_dropcluster 17 main
sudo pg_upgradecluster 15 main
sudo pg_dropcluster 15 main
sudo systemctl start postgresql
Get postgres user and reindex each database:
psql <database>
REINDEX DATABASE;
Or re-index all databases at once:
reindexdb --all
In case of collation warnings you could also call this statement:
ALTER DATABASE <database> REFRESH COLLATION VERSION;
Package Version Map (Debian 12 → Debian 13)¶
| Package | Version in Debian 12 (bookworm) | Version in Debian 13 (trixie) |
|---|---|---|
| python3 | 3.11.2 | 3.13.5 |
| python3-bcrypt | 3.2.2 | 4.2.0 |
| python3-dateutil | 2.8.2 | 2.9.0 |
| python3-psycopg2 | 2.9.5 | 2.9.10 |
| python3-fuzzywuzzy | 0.18.0 | 0.18.0 |
| python3-flask | 2.2.2 | 3.1.1 |
| python3-flask-babel | 2.0.0 | 4.1.0 |
| python3-flask-login | 0.6.2 | 0.6.3 |
| python3-flaskext.wtf | 1.1.1 | 1.2.2 |
| python3-markdown | 3.4.1 | 3.7 |
| python3-numpy | 1:1.24.2 | 1:2.2.4 |
| python3-pandas | 1.5.3 | 2.2.3 |
| python3-jinja2 | 3.1.2 | 3.1.6 |
| python3-flask-cors | 3.0.10 | 6.0.1 |
| python3-flask-restful | 0.3.9 | 0.3.10 |
| p7zip-full | 16.02 | 16.02 |
| python3-wand | 0.6.11 | 0.6.13 |
| python3-rdflib | 6.1.1 | 7.1.1 |
| python3-dicttoxml | 1.7.15 | 1.7.16 |
| python3-rdflib-jsonld | 0.6.1 | now included in python3-rdflib |
| python3-flasgger | 0.9.5 | 0.9.7.2 |
| python3-requests | 2.28.1 | 2.32.3 |
| exiftran | 2.10 | 2.14 |
| python3-email-validator | 1.3.0 | 2.2.0 |
| python3-svgwrite | 1.4.3 | 1.4.3 |
| python3-shapely | 1.8.5 | 2.1.0 |
| python3-xmltodict | 0.13.0 | 0.13.0 |
| python3-validators | 0.20.0 | 0.20.0 |
| python3-jwt | 2.6.0 | 2.10.1 |
| python3-python-flask-jwt-extended | 4.4.4 | 4.7.1 |
| python3-bs4 | 4.11.2 | 4.13.4 |
| apache2 | 2.4.65 | 2.4.65 |
| libapache2-mod-wsgi-py3 | 4.9.4 | 5.0.2 |
| gettext | 0.21 | 0.23.1 |
| npm | 9.2.0 | 9.2.0 |
| postgresql | 15.14 | 17.6 |
| postgresql-15/17-postgis-3 | 3.3.2 | 3.5.2 |
| postgresql-15/17-postgis-3-scripts | 3.3.2 | 3.5.2 |
| iipimage-server | 1.1 | 1.1 |
| libvips-tools | 8.14.1 | 8.16.1 |
| libapache2-mod-fcgid | 1:2.3.9-4 | 1:2.3.9-4 |
| python3-coverage | 6.5.0 | 7.6.0 |
| python3-pytest | 6.5.0 | 8.3.5 |
| mypy | 1.0.1 | 1.15.0 |
| pylint | 2.16.2 | 3.3.4 |
Reference¶
Upgrade guide: https://fullmetalbrackets.com/blog/upgrade-debian-12-bookworm-debian-13-trixie/
Updated by Alexander Watzinger 2 months ago
- Status changed from Acknowledged to In Progress
- Assignee set to Alexander Watzinger
Thanks a lot @Bernhard Koschiček-Krombholz for testing and providing detailed information.
Like discussed this will be also part of the major 9.0.0 release.
Updated by Alexander Watzinger 2 months ago
- Related to Feature #2464: OpenAtlas major version 9.0.0 added
Updated by Bernhard Koschiček-Krombholz about 1 month ago
You need to REINDEX each database with:
REINDEX DATABASE <database>;
And I just removed the @babel.localeselector from init.py line 39, and added locale_selector to Babel (Babel(app, locale_selector=get_locale))
There are still issues, but to get it running and basically working, just this was needed.
A added a branch: feature/trixie
Updated by Bernhard Koschiček-Krombholz about 1 month ago
- Description updated (diff)
I added LazyString (babel) for type checking to some functions, and tests are now at 100% coverage. (one test is not 100% correct, I left a Todo there)
There are a lot of mypy issues with the forms, because they expect a str and not LazyString. It can be handled with str(), but maybe there is a sophisticated method to do this.
Updated by Alexander Watzinger about 1 month ago
Thanks a lot for this. You can leave the Mypy stuff to me. Except for very API specific stuff which I may not dare to touch.
Anyway, it would be ok to leave some Mypy issues for the 9.1.0 release, although a nice to have, there are more important things.
Updated by Alexander Watzinger 4 days ago
cp -p /usr/lib/iipimage-server/iipsrv.fcgi /var/www/iipsrv/
Not sure if this was only at this machine or is needed for the upgrade in general (and should be added to your install instruction note). Also:
- I merged feature/new_classes to feature/trixie which will now be the main develop branch for the next version
- There were multiple merge conflicts but I hope I solved all of them correctly
- I added an SQL command to re-index all databases and a collation SQL fix to your install note
- I managed to solve all Mypy issues and will look into the Pylint issues next
Updated by Alexander Watzinger 3 days ago
- Description updated (diff)
- Assignee changed from Alexander Watzinger to Bernhard Koschiček-Krombholz
@Bernhard Koschiček-Krombholz:
I think I'm finished with this issue and assigned it to you. You did the most work anyway and I also spotted TODOs in your notes so feel free to close it anytime.
I put a compact server upgrade instruction in the description to make it more accessible. In case we find more we should update it there.
About Pylint:
I solved all errors and warnings and was surprised that afterwards only 28 refactoring issues remained (they were about 80 before).
So I dug into the changelogs and it seems they solved a lot of false positives in the last 2 years.
Although it doesn't change anything in functionality, it definitely feels a lot better now :)
Updated by Alexander Watzinger 2 days ago
- Subject changed from Update Debian to 13 (Trixie) to Update Debian to 13 (Trixie)
- Description updated (diff)
Updated by Alexander Watzinger 1 day ago
- Subject changed from Update Debian to 13 (Trixie) to Update OpenAtlas software to Debian 13 (Trixie)