Some problems with MapperExtension of sqlalchemy - insert

Question

There are two classes: User and Question
A user may have many questions, and it also contains a question_count
to record the the count of questions belong to him.
So, when I add a new question, I want update the question_count of the
user. At first, I do as:
question = Question(title='aaa', content='bbb')
Session.add(question)
Session.flush()
user = question.user
### user is not None
user.question_count += 1
Session.commit()
Everything goes well.
But I wan't to use event callback to do the same thing. As following:
from sqlalchemy.orm.interfaces import MapperExtension
class Callback(MapperExtension):
def after_insert(self, mapper, connection, instance):
user = instance.user
### user is None !!!
user.question_count += 1
class Question(Base):
__tablename__ = "questions"
__mapper_args__ = {'extension':Callback()}
....
Note in the "after_insert" method:
instance.user # -> Get None!!!
Why?
If I change that line to:
Session.query(User).filter_by(id=instance.user_id).one()
I can get the user successfully, But: the user can't be updated!
Look I have modified the user:
user.question_count += 1
But there is no 'update' sql printed in the console, and the
question_count are not updated.
I try to add Session.flush() or Session.commit() in the
after_insert() method, but both cause errors.
Is there any important thing I'm missing? Please help me, thank you

Answer 1

The author of sqlalchemy gave me an useful answer in a forum, I copy it here:
Additionally, a key concept of the
unit of work pattern is that it
organizes a full list of all
INSERT,UPDATE, and DELETE statements
which will be emitted, as well as the
order in which they are emitted,
before anything happens. When the
before_insert() and after_insert()
event hooks are called, this structure
has been determined, and cannot be
changed in any way. The
documentation for before_insert() and
before_update() mentions that the
flush plan cannot be affected at this
point - only individual attributes on
the object at hand, and those which
have not been inserted or updated yet,
can be affected here. Any scheme
which would like to change the flush
plan must use
SessionExtension.before_flush.
However, there are several ways of
accomplishing what you want here
without modifiying the flush plan.
The simplest is what I already
suggested. Use
MapperExtension.before_insert() on the
"User" class, and set
user.question_count =
len(user.questions). This assumes
that you are mutating the
user.questions collection, rather than
working with Question.user to
establish the relationship. If you
happened to be using a "dynamic"
relationship (which is not the case
here), you'd pull the history for
user.questions and count up what's
been appended and removed.
The next way, is to do pretty much
what you think you want here, that is
implement after_insert on Question,
but emit the UPDATE statement
yourself. That's why "connection" is
one of the arguments to the mapper
extension methods:
def after_insert(self, mapper, connection, instance):
connection.execute(users_table.update().\
values(question_count=users_table.c.question_count +1).\
where(users_table.c.id==instance.user_id))
I wouldn't prefer that approach since
it's quite wasteful for many new
Questions being added to a single
User. So yet another option, if
User.questions cannot be relied upon
and you'd like to avoid many ad-hoc
UPDATE statements, is to actually
affect the flush plan by using
SessionExtension.before_flush:
class
MySessionExtension(SessionExtension):
def before_flush(self, session, flush_context):
for obj in session.new:
if isinstance(obj, Question):
obj.user.question_count +=1
for obj in session.deleted:
if isinstance(obj, Question):
obj.user.question_count -= 1
To combine the "aggregate" approach of
the "before_flush" method with the
"emit the SQL yourself" approach of
the after_insert() method, you can
also use SessionExtension.after_flush,
to count everything up and emit a
single mass UPDATE statement with many
parameters. We're likely well in the
realm of overkill for this particular
situation, but I presented an example
of such a scheme at Pycon last year,
which you can see at
http://bitbucket.org/zzzeek/pycon2010/src/tip/chap5/sessionextension.py
.
And, as I tried, I found we should update the user.question_count in after_flush

Answer 2

user, being I assume a RelationshipProperty, is only populated after the flush (as it is only this point the ORM knows how to relate the two rows).
It looks like question_count is actually a derived property, being the number of Question rows for that user. If performance is not a concern, you could use a read-only property and let the mapper do the work:
#property
def question_count(self):
return len(self.questions)
Otherwise you're looking at implementing a trigger, either at the database-level or in python (which modifies the flush plan so is more complicated).