- Notifications
You must be signed in to change notification settings - Fork 1.6k
Description
We have just been bitten by this using the latest 0.5.0.
We had the following piece of code, run in two different threads:
with dataset.transaction as t: entity = dataset.get(entity_key) archived_entity = _move_to_archive_key(entity) t.delete(entity.key) t.put(archived_entity) return archived_entityWe've seen two copies of the archived_entity for the same entity. We've managed to reproduce it with two threads performing this concurrently (running separate Connections each, because httplib2 is not thread safe).
Having used Datastore's underlying technology before, I expected to get a ConcurrentModificationException on delete, and one of the threads throwing it, similar to what the Java API for AppEngine describes:
https://cloud.google.com/appengine/docs/java/datastore/transactions#Java_Uses_for_transactions
However, both of them succeed creating two copies of archived_entity, and there's no ConcurrentModificationException anywhere in gcloud-python code.
We've tracked down to the following:
Transaction#commitcalls theBatch#commithttps://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/datastore/transaction.py#L177Batch#commitis unaware of theTransaction#id, callsConnection#commitwithout thetransaction_idparameter https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/datastore/batch.py#L214Connection#commitgetstransaction_id, and sets the REST request's mode todatastore_pb.CommitRequest.NON_TRANSACTIONALin https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/datastore/connection.py#L321
Impact: This means that all transactions made through python-gcloud are non-transactional. Please treat it as a P0 bug.