Project

General

Profile

Actions

Feature #1542

closed

OpenAtlas on Kubernetes

Added by Alexander Watzinger almost 3 years ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Category:
Backend
Target version:
Start date:
2021-06-30
Estimated time:

Description

There is a workflow to create and update Kubernetes OpenAtlas instances.

Some issues have to be solved to move production sites to ACDH-CH Kubernetes, but these are internal issues.

For external users it is workable, but we still advise using a standard Debian (12) server for production and cannot guarantee a problem-free integration into Kubernetes.

Demo Version: https://openatlas-demo.acdh-ch-dev.oeaw.ac.at


Related issues 2 (0 open2 closed)

Related to OpenAtlas - Feature #1582: GitHub test actionsClosedBernhard Koschiček-Krombholz2021-09-21Actions
Related to OpenAtlas - Feature #2025: Daily Debian security updates for GitHub containersClosedBernhard Koschiček-Krombholz2023-06-13Actions
Actions #1

Updated by Alexander Watzinger almost 3 years ago

Today we have a Kubernetes meeting

Actions #2

Updated by Alexander Watzinger almost 3 years ago

Today I did some cleanup at the ACDH-CH GitLab repository. I deleted the main and develop branch and uploaded them again because, like we discussed, Kubernetes specific changes should only be made in the feature_kubernetes branch. Once everything is working with Kubernetes we may look into merging it to main but have to do this carefully to be sure to not put our productive systems at risk, e.g. now it seems like passwords may be leaked, more about that further down.

Next I wanted to merge our current develop branch to feature_kubernetes. Originally it was a copy of main but I think it would be better to be a branch of develop to have the latest changes already merged. However, when pulling the latest feature_kubernetes form GitLab I noticed a few problematic changes made there:
  • A SQL dump was added to the Git repository by Berni (install/demo-dev-dump.sql) which shouldn't be part of the OpenAtlas repository so please clean this up (e.g. delete and put in .gitignore).
  • Dalibor removed the instance/prodution.py from .gitignore, presumably to change configuration. This is not how it is supposed to work, e.g. passwords are saved in instance/production.py. If persistent changes are needed in the repository they should be made in config/default.py or, if they are Kubernetes specific we should add a config/kubernetes.py.
Please keep in mind Berni and Dalibor that we don't want a super special Kubernetes version that is incompatible with our already proofed, working and productive OpenAtlas main version. So next step would be:
  • fixing .gitignore to ignore instance/ again except specified files (see .gitignore in main branch for how it should be)
  • removing database dump from repository
  • merge develop to feature_kubernetes to be in sync with current development

However, it's great to see that you managed to get already something running at https://demo-acdh-ch.openatlas.eu/.

Actions #3

Updated by Dalibor Pancic almost 3 years ago

Dalibor removed the instance/prodution.py from .gitignore, presumably to change configuration. This is not how it is supposed to work, e.g. passwords are saved in instance/production.py. If persistent changes are needed in the repository they should be made in config/default.py or, if they are Kubernetes specific we should add a config/kubernetes.py.

The openatlas app must be reconfigured that it can check if variables introduced over instance/prodution.py in feature_kubernetes are present as env variables.
They have to be introduced as Gitlab CI/CD variables for feature_kubernetes branch. By default, Openatlas expects that variables are hard-coded in the instance/prodution.py.
We can return instance/prodution.py to .gitignore after we adjust Openatlas that it can take variables from the "os.environ"

DATABASE_NAME = os.environ.get('POSTGRES_DB')              
DATABASE_USER = os.environ.get('POSTGRES_USER') 
DATABASE_PASS = os.environ.get('POSTGRES_PASSWORD') 
DATABASE_HOST = os.environ.get('POSTGRES_HOST') 
DATABASE_PORT = os.environ.get('POSTGRES_PORT')
DATABASE_URL = os.environ.get('DATABASE_URL')
SECRET_KEY = os.environ.get('SECRET_KEY')
Actions #4

Updated by Alexander Watzinger almost 3 years ago

  • Description updated (diff)
Actions #5

Updated by Dalibor Pancic almost 3 years ago

New domain https://discovery-demo-acdh-ch.openatlas.eu is created, pointed to the cluster and can be used for deployment.

Actions #6

Updated by Dalibor Pancic over 2 years ago

The problem with the production.py is solved and production.py is returned to the .gitignore.
All Kubernetes related configuration is introduced over requirements.txt, Procfile, start.sh and Gitlab CI/CD evironment variables.
Only these three new files are added to the default Openatlas repo, and they are located in the root of the feature_kubernetes branch.

In order to merge the develop with the feature_kubernetes branch, we just need to remove install/demo-dev*.sql dumps.

Actions #7

Updated by Alexander Watzinger over 2 years ago

Thanks a lot Dalibor for solving this. I really hope we see you at our OpenAtlas Summer Meeting next week. Would be nice to see in person again and your first drinks will be on us for all your help :)

Actions #8

Updated by Alexander Watzinger over 2 years ago

  • Target version changed from Wishlist to 252

Moved this issue to the INDIGO roadmap version because we will need it for that cooperation first.

Actions #9

Updated by Alexander Watzinger over 2 years ago

  • Target version changed from 252 to Wishlist

Moving to wishlist again while resolving the INDIGO version.

Actions #10

Updated by Bernhard Koschiček-Krombholz over 2 years ago

  • Status changed from In Progress to Acknowledged
  • Assignee deleted (Bernhard Koschiček-Krombholz)
Actions #11

Updated by Bernhard Koschiček-Krombholz over 1 year ago

  • Description updated (diff)
  • Status changed from Acknowledged to Assigned
  • Assignee set to Bernhard Koschiček-Krombholz
  • Target version changed from Wishlist to 7.15.0

GitHub Action are working, but some tests fail.
We deployed a first version on ranger and it looks good. Now we can work on the harder issues like persistence, migration and update.

Actions #12

Updated by Bernhard Koschiček-Krombholz over 1 year ago

Actions #13

Updated by Bernhard Koschiček-Krombholz over 1 year ago

  • Description updated (diff)
Actions #14

Updated by Bernhard Koschiček-Krombholz over 1 year ago

  • Description updated (diff)
Actions #15

Updated by Bernhard Koschiček-Krombholz over 1 year ago

  • Description updated (diff)
Actions #16

Updated by Bernhard Koschiček-Krombholz over 1 year ago

  • Description updated (diff)
Actions #17

Updated by Bernhard Koschiček-Krombholz over 1 year ago

  • Description updated (diff)
Actions #18

Updated by Bernhard Koschiček-Krombholz over 1 year ago

  • Description updated (diff)
Actions #19

Updated by Bernhard Koschiček-Krombholz over 1 year ago

  • Description updated (diff)
Actions #20

Updated by Bernhard Koschiček-Krombholz about 1 year ago

  • Description updated (diff)
Actions #21

Updated by Bernhard Koschiček-Krombholz about 1 year ago

  • Description updated (diff)
Actions #22

Updated by Omar Siam about 1 year ago

Helm chart now should support external database as well as a database installed next to OpenAtlas. The docker-k8s feature branch at the moment deploys to a separate postgis database deployed on Rancher.
There is a pgadmin4 instance (credentials can be found in Rancher) to maintain database on this "external" PostgreSQL server.
We now need to test if it deploys correctly with the "embedded" PostgreSQL server. I am quite sure it should.
That should finish the Helm Chart tasks
I think we have a Workable demo version as well as a version that can be combined with OpenAtlas-Discovery.

Daily security updates:
This can be achieved using github actions cron jobs. We probably should make sure a new version of the container image is not published if there is no updates.

Backup:
We need to create K8s cron jobs for this an add them to the helm chart. That should be a short task.

Actions #23

Updated by Omar Siam about 1 year ago

There is a (new) permission setting in the actions yaml which has to state that a job wants to write to the container registry.

When forking actions are usually disabled. So no testing is done at all in a forked repository at first.

Actions #24

Updated by Omar Siam about 1 year ago

Deployment is now guarded with an if in acdh-oeaw clause.
The rest of the actions can run in other orgs or personal gh namespaces.

Actions #25

Updated by Bernhard Koschiček-Krombholz about 1 year ago

Thank you, Omar!
Working on first demo version, have some problems with URL. (Going for https://openatlas-demo.acdh-dev.arz.oeaw.ac.at/)

My next tasks are
  • GitHub schedule: Make a new GitHub action file to pull build, check if there are updates available. If updates are available, rebuild everything using workflow_dispatch: {}
  • Create cron job to save backups (files and database dumps) to the backup samba share
Actions #26

Updated by Omar Siam about 1 year ago

openatlas-demo.acdh-dev.arz.oeaw.ac.at
Your options are
Actions #27

Updated by Bernhard Koschiček-Krombholz about 1 year ago

  • Description updated (diff)
Actions #28

Updated by Omar Siam about 1 year ago

file management on k8s probably with WebDAV

Actions #29

Updated by Alexander Watzinger about 1 year ago

  • Description updated (diff)
  • Status changed from Assigned to In Progress
Actions #30

Updated by Bernhard Koschiček-Krombholz 12 months ago

  • Description updated (diff)
Actions #31

Updated by Bernhard Koschiček-Krombholz 12 months ago

  • Related to Feature #2025: Daily Debian security updates for GitHub containers added
Actions #32

Updated by Alexander Watzinger 10 months ago

  • Target version changed from 7.15.0 to 8.0.0

Assigning to a later version because of vacations and to not to interfere with our releases. It can of course be moved again once finished.

Actions #33

Updated by Alexander Watzinger 10 months ago

Just because Omar asked about the status the other day: we still need to test it in combination with a presentation site.
Once implemented please also add an entry with a link to the ACDH-CH demo presentation site to the "online sites list".

Actions #34

Updated by Alexander Watzinger 10 months ago

  • Description updated (diff)

I'm not sure if already covered so I added NPM package update to the description at "to do".
This would be the case for e.g. 7.15.0 release with chart.js for type stats display.

Actions #35

Updated by Alexander Watzinger 7 months ago

  • Subject changed from OpenAtlas on Kubernetes to OpenAtlas on Kubernetes (ongoing)
  • Target version deleted (8.0.0)
Actions #36

Updated by Bernhard Koschiček-Krombholz 6 months ago

  • Description updated (diff)
Actions #37

Updated by Bernhard Koschiček-Krombholz 6 months ago

  • Description updated (diff)
Actions #38

Updated by Bernhard Koschiček-Krombholz 6 months ago

  • Description updated (diff)
Actions #39

Updated by Bernhard Koschiček-Krombholz 6 months ago

  • Description updated (diff)
Actions #40

Updated by Bernhard Koschiček-Krombholz 6 months ago

Apache access log example JSON:

    "remote-address": tokens["remote-addr"](req, res),
    time: tokens["date"](req, res, "iso"),
    method: tokens["method"](req, res),
    url: tokens["url"](req, res),
    "http-version": tokens["http-version"](req, res),
    "status-code": tokens["status"](req, res),
    "content-length": tokens["res"](req, res, "content-length"),
    referrer: tokens["referrer"](req, res),
    "user-agent": tokens["user-agent"](req, res)
Actions #41

Updated by Bernhard Koschiček-Krombholz 6 months ago

  • Description updated (diff)
Actions #42

Updated by Bernhard Koschiček-Krombholz 4 months ago

  • Description updated (diff)
Actions #43

Updated by Bernhard Koschiček-Krombholz 4 months ago

  • Description updated (diff)
Actions #44

Updated by Bernhard Koschiček-Krombholz 4 months ago

  • Subject changed from OpenAtlas on Kubernetes (ongoing) to OpenAtlas on Kubernetes
  • Category changed from Administration to Backend
  • Status changed from In Progress to Closed
  • Target version set to 8.2.0
Actions #45

Updated by Bernhard Koschiček-Krombholz 4 months ago

  • Description updated (diff)
Actions

Also available in: Atom PDF