Actions
Bug #2501
closed
Error when using ampersand (sometimes)
Status:
Closed
Priority:
Normal
Assignee:
Category:
Backend
Target version:
Start date:
2025-04-09
Estimated time:
Found in version:
Description
View:
https://demo.openatlas.eu/insert/external_reference
Description:
When entering the URL above in the Name field, I get a Transaction error.
Stacktrace:
psycopg2.errors.NotNullViolation: null value in column "name" of relation "entity" violates not-null constraint DETAIL: Failing row contains (256517, E31, null, null, 2025-04-09 13:02:04.336508, null, null, null, null, null, null, null, null, external_reference).
I guess, this is because of the sanitization, which is happening in the field. Without the & sign, it works.
Updated by Bernhard Koschiček-Krombholz 9 months ago
Full stack trace:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/flask/app.py", line 2548, in __call__
return self.wsgi_app(environ, start_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask/app.py", line 2528, in wsgi_app
response = self.handle_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 271, in error_router
return original_handler(e)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 271, in error_router
return original_handler(e)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 271, in error_router
return original_handler(e)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 271, in error_router
return original_handler(e)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bkoschicek/www/openatlas/openatlas/display/util.py", line 349, in wrapped
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/bkoschicek/www/openatlas/openatlas/views/entity.py", line 84, in insert
return redirect(save(manager))
^^^^^^^^^^^^^^
File "/home/bkoschicek/www/openatlas/openatlas/views/entity.py", line 246, in save
manager.insert_entity()
File "/home/bkoschicek/www/openatlas/openatlas/forms/manager_base.py", line 230, in insert_entity
self.entity = Entity.insert(self.class_.name, self.form.name.data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bkoschicek/www/openatlas/openatlas/models/entity.py", line 469, in insert
db.insert({
File "/home/bkoschicek/www/openatlas/openatlas/database/entity.py", line 146, in insert
g.cursor.execute(
File "/usr/lib/python3/dist-packages/psycopg2/extras.py", line 146, in execute
return super().execute(query, vars)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.NotNullViolation: null value in column "name" of relation "entity" violates not-null constraint
DETAIL: Failing row contains (256517, E31, null, null, 2025-04-09 13:02:04.336508, null, null, null, null, null, null, null, null, external_reference).
Updated by Alexander Watzinger 9 months ago
- Subject changed from Transaction error when entering URL to Error when using ampersand (sometimes)
- Status changed from New to Acknowledged
- Target version set to 8.11.0
This is interesting one. So far I found out that:
- It's not limited to the URL field, e.g. same effect in a description field
- It's not related to sanitize functions and seems to be a database issue
- It has something to do with the & character but it seems dependent on the surrounding characters e.g.
- Hänsel & Gretel works
- Hänsel &Gretel doesn't work
- Hänsel &Gretel and Alex works
It doesn't work means an empty value is saved but it is easier to test in the description field because it doesn't throws an error (because the description isn't required).
Sadly I haven't much time at the moment but will take a look at it at some later point in case nobody else can figure it out it in the meantime.
Updated by Bernhard Koschiček-Krombholz 9 months ago
The problem is the MLStripper() class in the sanitize() function. It sees the & with following characters without spaces as HTML and returns None.
The simplest solution would be to add a replace function before and after the sanitizer:
string = string.replace('&', '&')
stripper = MLStripper()
stripper.feed(string)
sanitized = stripper.get_data().strip()
sanitized.replace('&', '&')
But I don't know where this could break something.
And we should not forget to adapt one test case with something like 'Hänsel &Gretel'.
Updated by Alexander Watzinger 9 months ago
- Status changed from Acknowledged to Closed
- Assignee set to Bernhard Koschiček-Krombholz
Thanks a lot Bernhard.
- You were right, the problem was actually related to the sanitize function, my bad.
- Thanks for the suggested fix, it works great and I like the simplicity.
- It is already in develop and online on demo-dev and the LMU workshop instance.
Actions