Skip to content

Transactions are commited in CommitRequest.NON_TRANSACTIONAL mode #903

@mwitkow

Description

@mwitkow

We have just been bitten by this using the latest 0.5.0.

We had the following piece of code, run in two different threads:

with dataset.transaction as t: entity = dataset.get(entity_key) archived_entity = _move_to_archive_key(entity) t.delete(entity.key) t.put(archived_entity) return archived_entity

We've seen two copies of the archived_entity for the same entity. We've managed to reproduce it with two threads performing this concurrently (running separate Connections each, because httplib2 is not thread safe).

Having used Datastore's underlying technology before, I expected to get a ConcurrentModificationException on delete, and one of the threads throwing it, similar to what the Java API for AppEngine describes:
https://cloud.google.com/appengine/docs/java/datastore/transactions#Java_Uses_for_transactions

However, both of them succeed creating two copies of archived_entity, and there's no ConcurrentModificationException anywhere in gcloud-python code.

We've tracked down to the following:

Impact: This means that all transactions made through python-gcloud are non-transactional. Please treat it as a P0 bug.

Metadata

Metadata

Assignees

Labels

api: datastoreIssues related to the Datastore API.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions