Skip to content

Datastore: projection queries do not return the same number of records. #9600

@klausmyrseth

Description

@klausmyrseth

OS: ubuntu 19.04
Python: 3.7.3
Client library: google-cloud-datastore 1.10.0

Steps to reproduce

  1. Import data into datastore (fairly big records with many fields, actually datex 2 compatible data for situations)
  2. Query all data and count the records.
  3. Make a projection query for 3-4 of the fields
  4. Create the suggested index.
  5. Run the projection query and count the records.

I seem to get only 25% of the records using the projection query, which I cant understand, for the same filter and options only by using the exact same settings as the first query.

Code example

 query = client.query(kind='TestData', namespace='TestNamespace1') data = query.fetch() run1 = len(list(data)) query = client.query(kind='TestData', namespace='TestNamespace1') query.projection = ['field1', 'field2', 'field3', 'field4'] data = query.fetch() run2 = len(list(data)) print('%s should be like %s' % (run1, run2))

Stack trace

You dont get stacktrace outside of the initial one that gives you the hint for the correct index to create in the database. I created the suggested one.

All fields outside of 1 is string or number one is lat long position using the datastore type.

Considering none projection query runs in 6 seconds and returns a result set of 18Mb I would prefer to get this working with the projection query coupled with gzip which run in 200ms where the result set should be 54Kb :)

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions