Getting started with Elasticsearch and .NET

GETTING STARTED WITH ELASTICSEARCH ON WINDOWS AND .NET WITH NEST A short introduction Oslo/NNUG Meetup Tomas Jansson 29/01/2014

THIS IS ME Tomas Jansson Manager & Group Lead .NET BEKK Oslo @TomasJansson tomas.jansson@bekk.no github.com/mastoj blog.tomasjansson.com

TL;DR; https://github.com/mastoj/NestDemo

AUDIENCE N00b Expert N00b Expert

BACKGROUND This is the data and we need this new application

WHAT I WANT TO SHOW YOU IS... Elasticsearch is awesome Indexing using NEST Querying using NEST ... not about advanced elasticsearch hosting

INSTALLATION Great news, install as a service added in 0.90.5 Powershell to the rescue

NEST Abstraction over Elasticsearch There is an low level abstraction as well called RawElasticClient

NEST Abstraction Fluent & Strongly over Elasticsearch typed

FUNC DEMO C:Devgit> scriptcs scriptcs (ctrl-c or blank to exit) > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 Func  executable

SIMPLE EXPRESSION DEMO > using System.Linq.Expressions; > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) Expression  ”function description” (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var binExpr = addExpr.Body as BinaryExpression; > Console.WriteLine(binExpr); (x + y) > var add2 = addExpr.Compile(); > add2(3, 1); 4

MORE COMPLEX EXPRESSION DEMO > public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString

MORE COMPLEX EXPRESSION DEMO > public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; Enables us to translate from one domain to another in an ”easy” manner > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString

ELASTICSEARCH CONNECTION public class ElasticClientWrapper : ElasticClient { private static string _connectionString = Settings.ElasticSearchServer; private static ConnectionSettings _settings = new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200 .SetDefaultIndex(Settings.Alias) //"customer_product_mapping" .UsePrettyResponses(); public ElasticClientWrapper() : base(_settings) { } } //usage var client = new ElasticClientWrapper();

MAPPING public class Product { public double UnitPrice { get; set; } public int TotalQuantity { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string ProductName { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string CategoryName { get; set; } } public class Customer { public string CustomerID { get; set; } public string CompanyName { get; set; } public string Address { get; set; } public string City { get; set; } public string Country { get; set; } [ElasticProperty(Type = FieldType.nested)] public Product[] Products { get; set; } }

MAPPING & INDEXING _client = new ElasticClientWrapper(); Mapping created from attributes _client.CreateIndex("indexName", s => s.AddMapping<Customer>(m => m.MapFromAttributes())); var customers = _customerRepo.GetCustomers(); _client.IndexMany(customers, "indexName"); Indexing will use the mapping for the specified index There is async versions of the methods

ALIAS _client = new ElasticClientWrapper(); _client.Alias("indexName", "aliasName"); Alias Index_01

SWAPPING _client = new ElasticClientWrapper(); _client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } ); Alias 1. Index_02 Swap 3. Index_01 Create new index 2. Alias Delete old index

MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES) public class SearchModel { private int? _numberToTake; public string Query { get; set; } public Dictionary<string, IEnumerable<string>> Filter { get; set; } public int? NumberToTake { get { return _numberToTake.HasValue ? _numberToTake.Value : 25; } set { _numberToTake = value; } } }

QUERYING Elasticsearch NEST { _client.Search<Customer>(sd => sd .QueryString(Input.Query)); "query": { "query_string": { "query": "tomas" } } }

FUZZY Elasticsearch NEST { _client.Search<Customer>(sd => sd .Query(q => q .Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query)))); "query": { "fuzzy": { "_all": { "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } } } } Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»

FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH { "query": { "bool": { "should": [{ "match": { "_all": { "query": "tomas" } } }, { "fuzzy": { "_all": { "boost": 0.1, "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } } }] } } }

FUZZY IMPROVED (USING BOOL QUERY) - NEST _client.Search<Customer>(sd => sd .Query(q => q .Bool(b => b .Should(new Func<QueryDescriptor<Customer>, BaseQuery>[] { _ => _.Match(m => m .OnField("_all") .QueryString(Input.Query)), _ => _.Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query) .Boost(0.1)) }))));

HIGHLIGHT RESULT - ELASTICSEARCH { "query": { // see previous example }, "highlight": { "pre_tags": [ "<span class='highlight'>" ], "post_tags": [ "</span>" ], "fields": { "companyName": { "fragment_size": 100, "number_of_fragments": 1 } } } }

HIGHLIGHT RESULT - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight(h => h .PreTags("<span class='highlight'>") .PostTags("</span>") .OnFields(new Action<HighlightFieldDescriptor<Customer>>[] { _ => _ .OnField(c => c.CompanyName) .NumberOfFragments(1) .FragmentSize(100) })));

FACETS - ELASTICSEARCH { "query": { /* See previous example */ }, "highlight": { /* See previous example */ }, "facets": { "products.productName": { "nested": "products", "terms": { "field": "products.productName", "size": 1000 } }, "products.categoryName": { "nested": "products", "terms": { "field": "products.categoryName", "size": 1000 } }, "country": { "terms": { "field": "country", "size": 1000 } } } }

FACETS - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight( /* See previous example */ ) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].ProductName) .Size(1000)) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].CategoryName) .Size(1000)) .FacetTerm(f => f .OnField(c => c.Country) .Size(1000)));

http://go-gaga-over-testing.blogspot.no/2011/09/solution-to-warning-in-quality-center.html

FILTERS - ELASTICSEARCH { "query": { "filtered": { "query": { /* See previous example */ }, "filter": { "bool": { "must": [ { "terms": { "country": ["usa"] } }, { "nested": { "query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } }, "path": "products" } }, { "nested": { "query": { "terms": { "products.productName": ["Chai"] } }, "path": "products" } } ] } } } }, "facets": { /* See previous example */}, "highlight": { /* See previous example */ } }

FILTERS – NEST – PART 1, THE CUSTOMERS FILTER private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); }

FILTERS – NEST – PART 1, THE PRODUCTS FILTER private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); }

FILTERS – NEST – PART 1, THE MAGIC DICTIONARY public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };

FILTERS – NEST – PART 1, ALL THE HELPERS private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); } private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); } public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };

FILTERS – NEST – PART 2, THE QUERY _client.Search<Customer>(sd => sd .Query(q => q .Filtered(fq => { fq.Query(qs => { if (!string.IsNullOrEmpty(Input.Query)) { qs.Bool( /* See previous example */ )); } else { qs.MatchAll(); } return qs; }); if (Input.Filter.Count > 0) { var filters = Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray(); fq.Filter(fs => fs.Bool(bf => bf.Must(filters))); } })) .Highlight( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ );

SUMMARY Elasticsearch NEST Easy installation Strongly typed client Awesome search engine Fluent Abstraction over Elasticsearch

RESOURCES Demo code: https://github.com/mastoj/NestDemo Nest documentation: http://nest.azurewebsites.net/ Nest source code: https://github.com/Mpdreamz/NEST Slideshare: http://www.slideshare.net/mastoj/getting-started-with-elasticsearch-and-net Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense

Getting started with Elasticsearch and .NET

More Related Content

What's hot

Viewers also liked

Similar to Getting started with Elasticsearch and .NET

More from Tomas Jansson

Recently uploaded

Getting started with Elasticsearch and .NET

Editor's Notes