• Table of contents
• To do
  • Automate upgrades
• Documentation
  • Packages
  • File folders
  • Update with new OpenAtlas releases
  • Backup

Issue #1542

To do¶

• Fix upload: uploading a 14 MB image fails with 413 Request Entity Too Large although limit is set to 20 MB (in OpenAtlas admin)
• Fix disappearing database: interface broke because JavaScript and CSS couldn't be loaded anymore (500). Redeploy on Rancher didn't help. A new build with gitlab pipeline wiped the database clean, although the code in start.sh should have prevented that.
• Remove need for extra branches for each instance
• Implement backup for daily, weekly, quarterly and test it (only database dumps or with files?)
• Implement and test a presentation site with OpenAtlas discovery. For the ACDH-CH demo we will use https://discovery-demo-acdh-ch.openatlas.eu
• Implement using Debian packages (instead pip) with a script for daily updates for security, reliability and stability reasons

Automate upgrades¶

Debian package updates

• Implement a mechanism for software updates with new/different packages

Database updates

• Update SQL scripts are already provided in install/upgrade named after the specific version e.g. 6.0.0.sql. Upgrades of former major versions are stored in install/upgrade/archive.
• We would need a mechanism that has to work outside/before the main application because changes may interfere with the initialization process in openatlas/__init__.py e.g. all types are loaded in the before_request() function -> Since every update will make new setup, I don't think, this will be a problem.
• In case that the database update, which runs in a transaction, fails the software shouldn't be updated but we need the software update to get the upgrade file. Maybe Kubernetes can help there, e.g. abort the update all together. -> If the transaction fails, the pipeline should fail and therefore not be deployed.
• The version update process should be "aware" if an update SQL is needed. The application code "knows" it's version, it's tracked as VERSION in config/default.py. To make the database aware of it's version we could add a value in the web/settings table but we still need functionality that checks if, which and in what order upgrade scripts are needed and can deal with failed update SQLs. -> We need to make the database aware of it's version in order properly check if a database exists and needs updates.

Documentation¶

Packages¶

We use Pipfile for now with Python and Heroku. Heroku will take care of security updates.
To add a new package:

pipenv install <package==version>

To update pipfile.lock:

pipenv update

• More on Pipfile: https://docs.pipenv.org/ https://github.com/pypa/pipfile
• How Heroku works: https://devcenter.heroku.com/articles/how-heroku-works
• What is Heroku: https://stackoverflow.com/tags/heroku/info
• Python Dependencies via Pip: https://devcenter.heroku.com/articles/python-pip

File folders¶

The folder openatlas/uploads, openatlas/export and openatlas/image_processing will contain files which users uploaded or generated. Therefore, additional volumes has to be created with the mount point /app/openatlas/<folder> (/app/openatlas/uploads) and read-only false.

Update with new OpenAtlas releases¶

• For Python packages Heroku/Kubernetes will use the Pipfile
• Update of npm packages is done via start.sh
• Database updates are done manually

Backup¶

Backups are made every day at 03:20 to a separate volume, which can be mounted to the container. At the moment only the last 10 Backups are kept.