In this post, I'm describing how to replicate the LambdaSharp app capability to log to CloudWatch Logs using an Amazon API Gateway REST API.
Observability is a critical building block for developers. Therefore, it is an integral part of the LambdaSharp developer experience. For reference, this is how a Blazor WebAssembly app is created and automatically wired for CloudWatch logging in LambdaSharp.
Module: Sample.BlazorWebAssembly Items: - App: MyBlazorApp Yes, that is really it! Nothing additional is needed, but there are plenty of additional capabilities. However, non-LambdaSharp developers may want to achieve the same capability for their apps using their preferred framework. That is the purpose of this post. It shows how to build the CloudWatch logging capability for any frontend app using any framework.
Overview
This implementation does not use any Lambda functions. Instead, we enable logging to CloudWatch by directly integrating the API Gateway REST API with the CloudWatch Logs API using Apache Velocity templates. This design means there is only minimal code involved, no Lambda cold-start latencies, and no Lambda invocation costs.
The implementation is described in terms of CloudFormation resources using the YAML notation, but the same outcome can be achieved by using AWS Console instead.
Logging REST API
First, we need to create a new API Gateway resource. We define the top-level .app resource to anchor our API. In LambdaSharp, this top-level resource name is configurable via CloudFormation parameters, which are omitted here.
RestApi: Type: AWS::ApiGateway::RestApi Properties: Name: !Sub "${AWS::StackName} App API" RestApiAppResource: Type: AWS::ApiGateway::Resource Properties: RestApiId: !Ref RestApi ParentId: !GetAtt RestApi.RootResourceId PathPart: .app CloudWatch Log Group
The CloudWatch Log Group should be created explicitly for each app to make it is easy to distinguish them across apps. In addition, a log retention policy should be set to limit the amount of storage the log group uses to avoid being billed indefinitely for it.
LogGroup: Type: AWS::Logs::LogGroup Properties: RetentionInDays: 90 API Gateway IAM Role
API Gateway needs permission to create log streams in the log group and write to them. This is achieved by the following IAM role definition.
RestApiRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Sid: ApiGatewayPrincipal Effect: Allow Principal: Service: apigateway.amazonaws.com Action: sts:AssumeRole Policies: - PolicyName: ApiLogsPolicy PolicyDocument: Version: 2012-10-17 Statement: - Sid: LogGroupPermission Effect: Allow Action: - logs:CreateLogStream - logs:PutLogEvents Resource: - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${LogGroup}" - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${LogGroup}:log-stream:*" API Validation
A neat feature of API Gateway REST API is that it can validate requests against a JSON schema model. This capability prevents unnecessary invocations of the API when the incoming payload is not valid. Validation is enabled by associating each API Gateway method with the following validator declaration.
RestApiValidator: Type: AWS::ApiGateway::RequestValidator Properties: RestApiId: !Ref RestApi ValidateRequestBody: true ValidateRequestParameters: true REST API
This next section is a bit heavy, because how API Gateway resources, methods, and integrations are built.
First, we create the logs resource associated with the API methods.
RestApiAppLogsResource: Type: AWS::ApiGateway::Resource Properties: RestApiId: !Ref RestApi ParentId: !Ref RestApiAppResource PathPart: "logs" Next, we need to create the OPTIONS method to handle CORS requests. Note this implementation uses Allow-Origin: '*', which should be replaced with the actual host scheme and name from which the application is served. LambdaSharp uses CloudFormation parameters to make it configurable, but these were omitted for brevity.
RestApiAppLogsResourceOPTIONS: Type: AWS::ApiGateway::Method Properties: AuthorizationType: NONE RestApiId: !Ref RestApi ResourceId: !Ref RestApiAppLogsResource HttpMethod: OPTIONS Integration: IntegrationResponses: - StatusCode: 204 ResponseParameters: method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" method.response.header.Access-Control-Allow-Methods: "'OPTIONS,POST,PUT'" method.response.header.Access-Control-Allow-Origin: "'*'" method.response.header.Access-Control-Max-Age: "'600'" ResponseTemplates: application/json: '' PassthroughBehavior: WHEN_NO_MATCH RequestTemplates: application/json: '{"statusCode": 200}' Type: MOCK MethodResponses: - StatusCode: 204 ResponseModels: application/json: 'Empty' ResponseParameters: method.response.header.Access-Control-Allow-Headers: false method.response.header.Access-Control-Allow-Methods: false method.response.header.Access-Control-Allow-Origin: false method.response.header.Access-Control-Max-Age: false Once the browser is authorized via the OPTIONS request, we need to provide two additional endpoints: one for creating a new log stream using POST and another for writing to a log stream using PUT. In addition, we define a JSON schema model for each endpoint to validate requests before they are executed.
Note that the app is responsible for creating a new log stream. For single page apps (SPA), a new log stream should be created each time the app loads. This is also the behavior for Blazor WebAssembly apps built with LambdaSharp.
Create LogStream - POST:/.app/logs
The POST method creates a new log stream in the associated log group. Some of the response handling relates to how errors are returned to the calling application. Emphasis of the handling is on providing useful feedback without revealing too many internal details.
Similar to the OPTIONS method, this configurations uses Allow-Origin: '*', which should be replaced with the actual host scheme and name from which the application is served. LambdaSharp uses CloudFormation parameters to make it configurable, but these were omitted for brevity.
RestApiAppLogsResourcePOST: Type: AWS::ApiGateway::Method Properties: OperationName: CreateLogStream ApiKeyRequired: true RestApiId: !Ref RestApi ResourceId: !Ref RestApiAppLogsResource AuthorizationType: NONE HttpMethod: POST RequestModels: application/json: !Ref RestApiAppLogsResourcePOSTRequestModel RequestValidatorId: !Ref RestApiValidator Integration: Type: AWS IntegrationHttpMethod: POST Uri: !Sub "arn:${AWS::Partition}:apigateway:${AWS::Region}:logs:action/CreateLogStream" Credentials: !GetAtt RestApiRole.Arn PassthroughBehavior: WHEN_NO_TEMPLATES RequestParameters: integration.request.header.Content-Type: "'application/x-amz-json-1.1'" integration.request.header.X-Amz-Target: "'Logs_20140328.CreateLogStream'" RequestTemplates: application/json: !Sub |- #set($body = $input.path('$')) { "logGroupName": "${LogGroup}", "logStreamName": "$body.logStreamName" } IntegrationResponses: - SelectionPattern: "200" StatusCode: 200 ResponseParameters: method.response.header.Access-Control-Allow-Origin: "'*'" ResponseTemplates: application/x-amz-json-1.1: |- { } - SelectionPattern: "400" StatusCode: 400 ResponseParameters: method.response.header.Access-Control-Allow-Origin: "'*'" ResponseTemplates: application/x-amz-json-1.1: |- #set($body = $input.path('$')) { #if($body.message.isEmpty()) "error": "Unknown error" #else "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")" #end } - StatusCode: 500 ResponseParameters: method.response.header.Access-Control-Allow-Origin: "'*'" ResponseTemplates: application/x-amz-json-1.1: |- { "error": "Unexpected response from service." } MethodResponses: - StatusCode: 200 ResponseModels: application/json: Empty ResponseParameters: method.response.header.Access-Control-Allow-Origin: false - StatusCode: 400 ResponseModels: application/json: Empty ResponseParameters: method.response.header.Access-Control-Allow-Origin: false - StatusCode: 500 ResponseModels: application/json: Empty ResponseParameters: method.response.header.Access-Control-Allow-Origin: false RestApiAppLogsResourcePOSTRequestModel: Type: AWS::ApiGateway::Model Properties: Description: CreateLogStream ContentType: application/json RestApiId: !Ref RestApi Schema: $schema: http://json-schema.org/draft-04/schema# type: object properties: logStreamName: type: string required: - logStreamName Append to Log Stream - POST .app/logs
Similar to the POST method, the PUT method validates incoming requests and limits what internal details are exposed when errors occur.
Similar to the OPTIONS method, this configurations uses Allow-Origin: '*', which should be replaced with the actual host scheme and name from which the application is served. LambdaSharp uses CloudFormation parameters to make it configurable, but these were omitted for brevity.
RestApiAppLogsResourcePUT: Type: AWS::ApiGateway::Method Properties: OperationName: PutLogEvents ApiKeyRequired: true RestApiId: !Ref RestApi ResourceId: !Ref RestApiAppLogsResource AuthorizationType: NONE HttpMethod: PUT RequestModels: application/json: !Ref RestApiAppLogsResourcePUTRequestModel RequestValidatorId: !Ref RestApiValidator Integration: Type: AWS IntegrationHttpMethod: POST Uri: !Sub "arn:${AWS::Partition}:apigateway:${AWS::Region}:logs:action/PutLogEvents" Credentials: !GetAtt RestApiRole.Arn PassthroughBehavior: WHEN_NO_TEMPLATES RequestParameters: integration.request.header.Content-Type: "'application/x-amz-json-1.1'" integration.request.header.X-Amz-Target: "'Logs_20140328.PutLogEvents'" integration.request.header.X-Amzn-Logs-Format: "'json/emf'" RequestTemplates: application/json: !Sub |- #set($body = $input.path('$')) { "logEvents": [ #foreach($logEvent in $body.logEvents) { "message": "$util.escapeJavaScript($logEvent.message).replaceAll("\\'","'")", "timestamp": $logEvent.timestamp }#if($foreach.hasNext),#end #end ], "logGroupName": "${LogGroup}", "logStreamName": "$body.logStreamName", "sequenceToken": #if($body.sequenceToken.isEmpty()) null#else "$body.sequenceToken"#end } IntegrationResponses: - SelectionPattern: "200" StatusCode: 200 ResponseParameters: method.response.header.Access-Control-Allow-Origin: "'*'" ResponseTemplates: application/x-amz-json-1.1: |- { "nextSequenceToken": "$input.path('$.nextSequenceToken')" } - SelectionPattern: "400" StatusCode: 400 ResponseParameters: method.response.header.Access-Control-Allow-Origin: "'*'" ResponseTemplates: application/x-amz-json-1.1: |- #set($body = $input.path('$')) #if($body.expectedSequenceToken.isEmpty()) { #if($body.message.isEmpty()) "error": "Unknown error" #else "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")" #end } #else { #if($body.message.isEmpty()) "error": "unknown error", #else "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")", #end "nextSequenceToken": "$body.expectedSequenceToken" } #end - StatusCode: 500 ResponseParameters: method.response.header.Access-Control-Allow-Origin: "'*'" ResponseTemplates: application/x-amz-json-1.1: |- { "error": "Unexpected response from service." } MethodResponses: - StatusCode: 200 ResponseModels: application/json: Empty ResponseParameters: method.response.header.Access-Control-Allow-Origin: false - StatusCode: 400 ResponseModels: application/json: Empty ResponseParameters: method.response.header.Access-Control-Allow-Origin: false - StatusCode: 500 ResponseModels: application/json: Empty ResponseParameters: method.response.header.Access-Control-Allow-Origin: false RestApiAppLogsResourcePUTRequestModel: Type: AWS::ApiGateway::Model Properties: Description: PutLogEvents ContentType: application/json RestApiId: !Ref RestApi Schema: $schema: http://json-schema.org/draft-04/schema# type: object properties: logEvents: type: array items: - type: object properties: message: type: string timestamp: type: integer required: - message - timestamp logStreamName: type: string sequenceToken: type: - string - "null" required: - logEvents - logStreamName API Key & Usage Plan
The following resources declare an API key and usage plan. The API key is set by default to the Base64 value of the CloudFormation stack GUID. It is recommended to explicitly set the API key since the frontend app will need access to it to use the logging REST API. The API key can be further obfuscated by combining it with an internal value of the app. In LambdaSharp, the API key is generated by combining the CloudFormation stack GUID and the compiled .NET Core assembly identifier GUID.
RestApiKey: Type: AWS::ApiGateway::ApiKey Properties: Description: !Sub "${AWS::StackName} App API Key" Enabled: true StageKeys: - RestApiId: !Ref RestApi StageName: !Ref RestApiStage Value: Fn::Base64: !Select [ 2, !Split [ "/", !Ref AWS::StackId ]] RestApiUsagePlan: Type: AWS::ApiGateway::UsagePlan Properties: ApiStages: - ApiId: !Ref RestApi Stage: !Ref RestApiStage Description: !Sub "${AWS::StackName} App API Usage Plan" Throttle: BurstLimit: 200 RateLimit: 100 RestApiUsagePlanKey: Type: AWS::ApiGateway::UsagePlanKey Properties: KeyId: !Ref RestApiKey KeyType: API_KEY UsagePlanId: !Ref RestApiUsagePlan API Deployment
Finally, we define a stage called LATEST, which is used by the deployment resource. Note that CloudFormation only runs the deployment once. Subsequent CloudFormation stack updates need to be manually deployed when the REST API changes. LambdaSharp uses the Finalizer construct to allow for configuration changes to be always applied automatically.
RestApiStage: Type: AWS::ApiGateway::Stage Properties: DeploymentId: !Ref RestApiDeployment Description: App API LATEST Stage RestApiId: !Ref RestApi StageName: LATEST RestApiDeployment: Type: AWS::ApiGateway::Deployment Properties: Description: !Sub "${AWS::StackName} App API" RestApiId: !Ref RestApi DependsOn: - RestApiAppLogsResource - RestApiAppLogsResourcePOST - RestApiAppLogsResourcePOSTRequestModel - RestApiAppLogsResourcePUT - RestApiAppLogsResourcePUTRequestModel Conclusion - To be continued...
In this post, we created the resources required to enable a frontend apps to log to CloudWatch directly. In the next post, we will cover the protocol for logging via this REST API.
Happy Hacking!
Top comments (0)