Skip to content

Conversation

@ericapywang
Copy link
Contributor

@ericapywang ericapywang commented Jul 16, 2024

Problem

describe_index_stats was not implemented.

Solution

I implemented a new function describe_index_stats for Index, which optionally takes in a MetadataFilter struct. It calls tonic's generated describe_index_stats function.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update
  • Infrastructure change (CI configs, etc)
  • Non-code change (docs, etc)
  • None of the above: (explain here)

Test Plan

New and existing test cases should pass.

Value {
kind: Some(Kind::BoolValue(false)),
},
);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, this is how the input is defined for tonic, but not sure if this is the most user friendly definition.

DescribeIndexStatsError {
/// Source error
source: tonic::Status,
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emily and I are likely going to merge these errors into a more general error type in the future, instead of having one for each operation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used DataPlaneError in #27 instead of defining ListVectorsError

/// ```
pub async fn describe_index_stats(
&mut self,
filter: Option<BTreeMap<String, Value>>,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how much I like this type definition, but it matches how it is defined in the generated code. Thoughts?

Copy link

@haruska haruska Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are optional metadata filters in a few places. might want to define a type and encapsulate the BTreeMap<String, Value>. You could also define From/Into on that type to translate to/from the codegen definition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So something like this?

struct MetadataFilter { filter: BTreeMap<String, Value> } impl From<MetadataFilter> for Struct { // impl here } 

In this case, would it just be better to use the Struct that is defined by tonic and encapsulate filter inside, like how tonic's generated function has it?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to you. It might make sense to define MetadataFilter. The data shape is more strict than tonic's Struct. You don't have to define that strictness here but having your own type might make the SDK surface area to the user better.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reference see https://docs.pinecone.io/guides/data/filter-with-metadata with particular attention to "Supported metadata types"

/// ```
pub async fn describe_index_stats(
&mut self,
filter: Option<BTreeMap<String, Value>>,
Copy link

@haruska haruska Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are optional metadata filters in a few places. might want to define a type and encapsulate the BTreeMap<String, Value>. You could also define From/Into on that type to translate to/from the codegen definition.

let filter = match filter {
Some(filter) => Some(Struct { fields: filter }),
None => None,
};
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find Option#map to be more elegant for these transformations.

let mut index = pinecone
.index(host.as_str())
.await
.expect("Failed to target index");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could look into adding a test fixture for these things since we'll need it often for data plane integration tests - for another PR

@ericapywang ericapywang marked this pull request as ready for review July 16, 2024 20:14
@ericapywang ericapywang merged commit 3f31a40 into main Jul 16, 2024
@ericapywang ericapywang deleted the erica/describe-index-stats branch July 16, 2024 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants