A foreign key (FK) can refer to a primary key in the same table in MySQL. This is known as a self-referential foreign key or a recursive foreign key. It allows you to create hierarchical relationships within the same table, which can be useful for representing structures like organizational charts, bill of materials, or nested… Continue reading Relation DB – Foreign key
Author: A
Basic – API Request
Path Parameter vs. Request Body Parameter Path Parameter: Use Case: Typically, path parameters are used to identify specific resources. For instance, if you want to add a child node to a specific parent node, using the parent’s node_id as a path parameter (e.g., POST /nodes/{parent_id}/children) clearly indicates the relationship and hierarchy. Best Practice: This aligns with REST principles… Continue reading Basic – API Request
KEDA (Kubernetes-based Event Driven Autoscaling)
KEDA (Kubernetes-based Event Driven Autoscaling) is a CNCF (Cloud Native Computing Foundation) project that enables event-driven and queue depth-based autoscaling for workloads running in Kubernetes. It allows Kubernetes deployments to scale up or down based on the rate of incoming messages or the depth of various message queues, such as Kafka, RabbitMQ, or Azure Service… Continue reading KEDA (Kubernetes-based Event Driven Autoscaling)
Kafka – incremental cooperative rebalance protocol
The incremental cooperative rebalance protocol was introduced in Kafka 2.4.0, which supports KIP-429. Starting with this version, Kafka clients can opt to use this protocol for consumer groups to achieve more efficient and less disruptive rebalancing.To enable the incremental cooperative rebalancing protocol, the consumer's partition.assignment.strategy configuration should be set to use one of the following… Continue reading Kafka – incremental cooperative rebalance protocol
Kafka Parallel Processing
Uber - https://www.uber.com/en-IN/blog/reliable-reprocessing/https://www.uber.com/en-IN/blog/kafka-async-queuing-with-consumer-proxy/ Confluent - https://www.confluent.io/blog/introducing-confluent-parallel-message-processing-client/https://github.com/confluentinc/parallel-consumer?tab=readme-ov-file 21 min in video to understand offset management in kafka parallel https://play.vidyard.com/5MLb1Xh7joEQ7phxPxiyPK.jpg
RPCs over HTTP
XML-RPC is an example of an RPC system that is simple and uses XML for encoding, but it can be inefficient compared to modern RPC systems like gRPC that use more compact binary serialization formats like Protobuf.Thrift (Apache Thrift): A flexible cross-language RPC framework that supports a variety of serialization formats (including both text and… Continue reading RPCs over HTTP
storing data as binary
The JSON and binary representation of the string "John" would store the actual characters as 4 bytes (1 byte per character), as each character like J, o, h, n is typically represented in ASCII or UTF-8 encoding, which takes 1 byte per character. The key distinction between JSON and binary representation in this case lies… Continue reading storing data as binary
DynamoDB
When you perform an UpdateItem operation using the DAX client, the update is processed as a write-through operation. This means that the item is updated in both DynamoDB and the DAX cache simultaneously. How DAX Handles Updates Immediate Update in DAX: When you execute an UpdateItem via the DAX client, DAX first updates the item in the underlying DynamoDB table. Once… Continue reading DynamoDB
Cursor vs Offset Based Pagination
CriteriaCursor-Based PaginationOffset-Based Pagination PerformanceExcellent for large datasets, as it avoids the overhead of calculating offsets for deep pagination.Degrades with large datasets or high offsets, as scanning through previous pages becomes expensive. Consistency in Dynamic DataHighly consistent in environments with frequent data changes; avoids skipping or duplicating items.Prone to skipping or duplicating items when the dataset… Continue reading Cursor vs Offset Based Pagination
Adaptive rate limiting ES
Creating a dynamic rate limiter for Elasticsearch (ES) requires monitoring the current state of the ES cluster and adjusting the rate limit based on its load. Here are some steps and factors to consider when designing the dynamic rate-limiting logic: Key Factors for Dynamic Rate Limiting: Cluster Health: Green: Increase the rate limit, as the… Continue reading Adaptive rate limiting ES
You must be logged in to post a comment.