Logging, tracing and metrics: Instrumentation in .NET 5 and Azure

Logging, tracing and metrics Overview of instrumentation in .NET 5 and Azure Alex Thissen @alexthissen Cloud architect at Xpirit, The Netherlands

DevOps needs monitoring https://docs.microsoft.com/en-us/azure/architecture/checklist/dev-ops#monitoring

Application performance monitoring Cloud applications are complex Mostly distributed applications Using multiple cloud resources Collect and correlate instrumentation data Events, usage, metrics, performance, telemetry, logs, traces, health Sentry.io Raygun.io Runscope NewRelic AlertSite DataDog AppMetrics Azure Monitor

Cloud Application Azure Resources Azure Monitor Azure Subscription Azure Tenant Azure Monitor Logs Applications in Azure Azure Monitor Metrics

Collect Monitor Application instrumentation Logging Tracing Metrics Health Dashboards Alerting Analytics Profiling Metrics streaming

Choosing correct type of instrumentation Log • What happened? • Errors and warnings and state changes • Important state changes worth registering • Always logged Trace • How did it happen? • Circumstantial data for following flow • Available on request • Publish/ subscribe model mostly • High volume Metric • How much is happening? • Numerical information of events • Suitable for aggregation and trends Health • How is it doing? • Information indicating health status • Internally determined by component Audit • Who made it happen? • Data regarding actions performed by someone • Always • Security and identity related • Proof for non- repudiability

Logging in .NET 5 Leveraging CoreFX libraries for logging

Logging Persisting important events in a log Built-in since .NET Core 1.0 Used internally by CoreFX libraries Bootstrapping changed from .NET Core 3.0 forward

Logging providers Host.CreateDefaultBuilder(args) .ConfigureLogging((context, builder) => { // Log providers builder.AddApplicationInsights(options => { options.IncludeScopes = true; options.TrackExceptionsAsExceptionTelemetry = true; }); builder.AddConsole(options => { options.Format = ConsoleLoggerFormat.Systemd; }); builder.AddDebug(); builder.AddTraceSource(source.Switch, new ApplicationInsightsTraceListener()); .NET 5 * Added by default NullLoggerProvider BatchingLoggerProvider ConsoleLoggerProvider * DebugLoggerProvider * EventLogLoggerProvider EventSourceLoggerProvider * TraceSourceLoggerProvider ASP .NET Core ApplicationInsightsLoggerProvider AzureAppServicesFile AzureAppServicesBlob Third party NLogLogger Seq Log4Net Loggr

LoggerFactory and logger instances LoggerFactory LogCritical LogError LogWarning LogDebug LogTrace LogInformation ILogger<T>

Log severity levels and categories Hierarchical structure Much like namespaces in C#, separated by dots Microsoft Microsoft.Hosting.Lifetime Determined from T in ILogger<T> Except when creating ILogger directly with LoggerFactory.CreateLogger public enum LogLevel { Trace = 0, Debug = 1, Information = 2, Warning = 3, Error = 4, Critical = 5, None = 6 } ILogger<LeaderboardController> namespace LeaderboardWebApi.Controllers { public class LeaderboardController : Controller { … } } "LeaderboardWebApi.Controllers.LeaderboardController"

Separate configuration for specific providers General configuration for all providers Log filters Filter log messages By provider and per log category based on severity Can read filters from configuration (appsettings.json) { "Logging": { "LogLevel": { "Default": "Information", "Microsoft": "Warning", "Microsoft.Hosting.Lifetime": "Information" }, "Console": { "LogLevel": { … } "IncludeScopes": true } } ILogger<T>

Quantum physics applied to logging Duality of log messages

Semantic logging (aka structured logging) Message templates Strings with named placeholders for semantics Avoid string interpolation in messages Popular log providers supporting semantic logging // Placeholders in template become queryable custom data logger.LogInformation("Searching with {SearchLimit} results.", limit);

Demo Structured logging using Seq

Scopes Groups a set of logical operations Requires setting options to include scope for provider logging.AddConsole(options => options.IncludeScopes = true); using (logger.BeginScope("Message attached to logs created in the using block")) { logger.LogInformation(LoggingEvents.NewHighScore, “New high score {score}", highScore); var game = gameRepository.Find(gameId); if (game == null) { logger.LogWarning(LoggingEvents.GameNotFound, "GetById({game}) not found", gameId); return NotFound(); } }

Logging guidance Separation of concerns “A twelve-factor app never concerns itself with routing or storage of its output stream." Choose your log severity level carefully You might get paged during the night Use event IDs when possible Log messages are not for UI No need to localize your messages When in doubt, be generous on logging Tooling will help you filter and aggregate

Alternative logging solutions Serilog Popular open source logging framework Many log providers available Replaces built-in logging system Can also log to .NET log providers More functionality Log appenders (aka enrichers) for additional data Large set of providers Serilog

Tracing Familiar concepts and API from .NET Framework

Diagnostics Trace High volume noise to follow flow Not a log of things that happened Publish/subscribe Requires a listener to receive information Information not persisted unless subscribed System.Diagnostics namespace Trace and TraceSource entrypoints Needs compilation constant #TRACE Uses activities under the cover

Tracing infrastructure Trace “Singleton” with static methods Traditional tracing TraceData TraceEvent TraceInformation TraceTransfer TraceError TraceWarning TraceInformation Write(If) WriteLine(If) TraceSource

Activities Ambient contextual data Accessible from Activity.Current Thread-safe and async aware Used for correlating events Even across network calls New W3C tracing format Application Insights uses activities Parent/Child relationship var activity = new Activity("SearchEngine.Run"); activity.SetStartTime(DateTime.Now); activity.Start(); activity.Track...();

Available trace listeners Namespace Class .NET version System.Diagnostics DefaultTraceListener Core 1.0+ TextWriterTraceListener Core 1.0+ EventLogTraceListener Core 3.0 ConsoleTraceListener Core 3.0 DelimitedListTraceListener Core 1.0+ XmlWriterTraceListener Core 3.0 EventSchemaTraceListener .NET FX 3.5+ System.Diagnostics.Eventing EventProviderTraceListener .NET FX 3.5+ System.Web IisTraceListener .NET FX 2.0+ WebPageTraceListener .NET FX 2.0+ Microsoft.VisualBasic.Logging FileLogTraceListener .NET FX 2.0+ From .NET Framework 2.0 to .NET Core 3.0 to .NET 5

Health checks Indicating health status from .NET 5

Health monitoring services .AddHealthChecks() .AddCheck("sync", () => … ) .AddAsyncCheck("async", async () => … ) .AddCheck<SqlConnectionHealthCheck>("SQL") .AddCheck<UrlHealthCheck>("URL"); ASP .NET Core application /health DefaultHealthCheckService

Health check publishers Pushes out health info periodically Options ASP .NET Core application DefaultHealthCheckService HealthCheckPublisher HostedService IEnumerable<IHealthCheckPublisher> services.AddHealthChecks() .AddApplicationInsightsPublisher() .AddPrometheusGatewayPublisher( "http://pushgateway:9091/metrics", "pushgateway") IHealthCheckPublisher

Probing containers to check for availability and health Readiness and liveness Kubernetes node Kubernetes node Kubernetes nodes Containers Readiness Liveliness

Implementing readiness and liveliness 1. Add health checks with tags 2. Register multiple endpoints with filter using Options predicate /api/v1/… /health /health/ready /health/lively app.UseHealthChecks("/health/ready", new HealthCheckOptions() { Predicate = reg => reg.Tags.Contains("ready") }); services.AddHealthChecks() .AddCheck<CircuitBreakerHealthCheck>( "circuitbreakers", tags: new string[] { "ready" }); app.UseHealthChecks("/health/lively", new HealthCheckOptions() { Predicate = _ => true });

Demo Health checks and monitoring in ASP .NET Core

Application Insights Metrics, monitoring, querying and analyzing your application

Azure Application Insights Extensible Application Performance Monitor

Application Insights in .NET 5 Logging and tracing Regular logging provider Special TraceListener builder.AddApplicationInsights(options => { options.TrackExceptionsAsExceptionTelemetry = true; options.IncludeScopes = true; }); Telemetry TelemetryClient StartOperation TrackEvent Trace.Listeners.Add(new ApplicationInsightsTraceListener()); services.AddApplicationInsightsTelemetry(options => { options.DeveloperMode = true; });

Resources https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-5.0 https://docs.microsoft.com/en-us/azure/architecture/patterns/health-endpoint-monitoring https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks https://dev.applicationinsights.io/apiexplorer/metrics https://github.com/alexthissen/instrumentation

Logging, tracing and metrics: Instrumentation in .NET 5 and Azure

More Related Content

What's hot

Similar to Logging, tracing and metrics: Instrumentation in .NET 5 and Azure

More from Alex Thissen

Recently uploaded

Logging, tracing and metrics: Instrumentation in .NET 5 and Azure

Editor's Notes