Issue
I have a Django project that I’ve recently upgraded from Django 2.2 to 3.2. In this project, I use a custom UUIDField that saves UUIDs to MySQL as char(36) in the following format: 12345678-1234-5678-1234-567812345678.
import uuid
from django.db import models
class UUIDField(models.UUIDField):
"""
Overrides Django UUIDField to store full UUID's including dashes.
"""
def __init__(self, verbose_name=None, **kwargs):
super().__init__(verbose_name, **kwargs)
self.max_length = 36
def get_internal_type(self):
return "CharField"
def get_db_prep_value(self, value, connection, prepared=False):
if value is None:
return None
if not isinstance(value, uuid.UUID):
try:
value = uuid.UUID(value)
except AttributeError:
raise TypeError(self.error_messages['invalid'] % {'value': value})
if connection.features.has_native_uuid_field:
return value
return str(value)
After the upgrade, I noticed that searching for full UUIDs didn’t work anymore. If I only provide the first part of the UUID (up to the first character after the first hyphen) it works as expected.
Python 3.6.9 (default, Mar 15 2022, 13:55:28)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from foobar.foo.models import Foo
>>>
>>> Foo.objects.all()
<QuerySet [<Foo: Foo object (34c46fe8-caf0-11ec-bdb9-482ae362a4c0)>]>
>>>
>>> Foo.objects.filter(id__icontains='34c46fe8-caf0-11ec-bdb9-482ae362a4c0')
<QuerySet []>
>>>
>>> Foo.objects.filter(id__icontains='34c46fe8-')
<QuerySet [<Foo: Foo object (34c46fe8-caf0-11ec-bdb9-482ae362a4c0)>]>
>>>
>>> Foo.objects.filter(id__icontains='34c46fe8-c')
<QuerySet []>
>>>
I’ve played with the UUIDField methods, but I can’t seem to figure out what went wrong. Here’s a link to a Gist using a simplified model where I got the above shell example from.
Solution
After a quite long pdb debugging session, I managed to find what the problem is. I was hoping to find raw SQL snippets along the way, but instead an object called WhereNode caught my attention.
> /home/milanb/temp/django_custom_uuid_field/django_playground/lib/python3.6/site-packages/django/db/models/sql/query.py(1399)build_filter()
-> return clause, used_joins if not require_outer else ()
(Pdb) n
--Return--
> /home/milanb/temp/django_custom_uuid_field/django_playground/lib/python3.6/site-packages/django/db/models/sql/query.py(1399)build_filter()->(<WhereNode: (...f9c4f880be0>)>, {'foo_foo'})
-> return clause, used_joins if not require_outer else ()
(Pdb) retval
(<WhereNode: (AND: <django.db.models.lookups.UUIDIContains object at 0x7f9c4f880be0>)>, {'foo_foo'})
(Pdb) pp locals()
{'__return__': (<WhereNode: (AND: <django.db.models.lookups.UUIDIContains object at 0x7f9c4f880be0>)>,
{'foo_foo'}),
'alias': 'foo_foo',
'allow_joins': True,
'allow_many': True,
'arg': 'id__icontains',
'branch_negated': False,
'can_reuse': {'foo_foo'},
'check_filterable': True,
'clause': <WhereNode: (AND: <django.db.models.lookups.UUIDIContains object at 0x7f9c4f880be0>)>,
'col': Col(foo_foo, foo.Foo.id),
'condition': <django.db.models.lookups.UUIDIContains object at 0x7f9c4f880be0>,
'current_negated': False,
'filter_expr': ('id__icontains', '34c46fe8-caf0-11ec-bdb9-482ae362a4c0'),
'join_info': JoinInfo(final_field=<foobar.foo.fields.UUIDField: id>, targets=(<foobar.foo.fields.UUIDField: id>,), opts=<Options for Foo>, joins=['foo_foo'], path=[], transform_function=<function Query.setup_joins.<locals>.final_transformer at 0x7f9c4f89ca60>),
'join_list': ['foo_foo'],
'lookup_type': 'icontains',
'lookups': ['icontains'],
'opts': <Options for Foo>,
'parts': ['id'],
'pre_joins': {},
'reffed_expression': False,
'require_outer': False,
'reuse_with_filtered_relation': False,
'self': <django.db.models.sql.query.Query object at 0x7f9c4f902c18>,
'split_subq': True,
'targets': (<foobar.foo.fields.UUIDField: id>,),
'used_joins': {'foo_foo'},
'value': '34c46fe8-caf0-11ec-bdb9-482ae362a4c0'}
(Pdb) pp clause
<WhereNode: (AND: <django.db.models.lookups.UUIDIContains object at 0x7f9c4f880be0>)>
Here I noticed this particular line:
'clause': <WhereNode: (AND: <django.db.models.lookups.UUIDIContains object at 0x7f9c4f880be0>)>,
I didn’t know what a django.db.models.lookups.UUIDIContains object was, so went straight to Django’s source code to find out. This is en empty class made of two mixins IContains and UUIDTextMixin. The latter was the culprit in my case as it removed all hyphens from the lookup value. This was introduced as a fix after version 2.2, which I used before the upgrade, in order to support database backends without a native UUID type (e.g. MySQL 5.7 in my case).
The fix was very easy, I just had to register a custom UUIDIContains class without the UUIDTextMixin for my custom UUIDField.
My new fields.py:
import uuid
from django.db import models
from django.db.models.lookups import IContains
class UUIDField(models.UUIDField):
"""
Overrides Django UUIDField to store full UUID's including dashes.
"""
def __init__(self, verbose_name=None, **kwargs):
super().__init__(verbose_name, **kwargs)
self.max_length = 36
def get_internal_type(self):
return "CharField"
def get_db_prep_value(self, value, connection, prepared=False):
if value is None:
return None
if not isinstance(value, uuid.UUID):
try:
value = uuid.UUID(value)
except AttributeError:
raise TypeError(self.error_messages['invalid'] % {'value': value})
if connection.features.has_native_uuid_field:
return value
return str(value)
@UUIDField.register_lookup
class UUIDIContains(IContains):
pass
Answered By – Milo
This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0