error while using "db_field" parameter in multi-level embedded documents

[ms5]

The following code is an example to reconstruct what's going wrong. The main problem seems to be the use of db_field in the second level of the embedded document (In the example this is class B)

data = {'b': {'c': [{'x': 1.0, 'y': 2.0}]}}

class C(EmbeddedDocument):
    x = FloatField(db_field='fx')
    y = FloatField(db_field='fy')

class B(EmbeddedDocument):
    c = ListField(EmbeddedDocumentField(C), db_field='fc')

class A(EmbeddedDocument):
    b = EmbeddedDocumentField(B, db_field='fb')

a = A(**data)
a.validate()

If I run this, it throws an exception:

  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/mongoengine/base/document.py", line 424, in validate
    raise ValidationError(message, errors=errors)
mongoengine.errors.ValidationError: ValidationError (A:None) (c.Invalid embedded document instance provided to an EmbeddedDocumentField: ['b'])

But it runs through if I remove the db_field attribute in class B:

class B(EmbeddedDocument):
    c = ListField(EmbeddedDocumentField(C))

While digging in to this problem I found out that the exception is actually correct when saying the Document has the wrong format.

while running a simple print on the working an non working version, the type differs.

print(a.b.c[0])
print(type(a.b.c[0]))

# wrong
{'x': 1.0, 'y': 2.0}
<class 'mongoengine.base.datastructures.BaseDict'>

# good
C object
<class '__main__.C'>

[ms5]

I narrowed it down a bit and found in base/document.py the following block around line 680

for field_name, field in fields.iteritems():
    field._auto_dereference = _auto_dereference
    if field.db_field in data:
        value = data[field.db_field]

As far as I understand whats going on the problem resides in the line:

if field.db_field in data:

where db_field contains the supposed db_field name (ex: fc) but in data we have only c.

So i was looking up, where data is coming from and found:

data = dict(("%s" % key, value) for key, value in son.iteritems())

there is obviously no conversation from python field name to db field name. So I came up with a little patch:

         # class if unavailable
         class_name = son.get('_cls', cls._class_name)
-        data = dict(("%s" % key, value) for key, value in son.iteritems())
+        data = dict(("%s" % cls._db_field_map.get(key,key), value) for key, value in son.items())

         # Return correct subclass for document type
         if class_name != cls._class_name:

This seems to work for my short testing, but I'm actually not sure if I will break something later or if this is good practice. I'm also not sure, if I'm actually supposed to use _db_field_map here and if this data is reliable (= always correctly populated)

Before I create a pull request, it would be nice if a mongoengine developer could give me some input. Thanks a lot!!

[MRigal]

nice job @ms5 ! Would you eventually make a PR out of this? Formalising also the description above inside a test? It would be great !

[ms5]

ok, no problem. give me some time to write the tests, I'll then create a pull request.