Elasticache bandwidth usage is low, but bandwidth allowance exceeded

Question

We recently switched to using Elasticache, and we're seeing some blips on the charts for Network Bandwidth In Allowance Exceeded, and for Network Bandwidth Out Allowance Exceeded:

This is weird because we're not using much bandwidth:

We are using a cache.t4g.small instance with cluster mode off and one replica. According to the pricing page, we should be getting "Up to 5 Gigabit" network performance. The charts above are nowhere near that.

So why is our bandwidth getting exceeded? Do I just need to suck it up and pay for a larger instance? If so, that's going to be about 4× more expensive, since the next instance with more bandwidth is a cache.m6g.large. Something seems off? Or maybe I can ignore these charts?

Answer 1

This is over 6 months ago, but I just stumbled on the same thing. For anyone coming here later, the answer is on ElastiCache Troubleshooting Connections page and is summed up by:

NetworkBandwidthInAllowanceExceeded / NetworkBandwidthOutAllowanceExceeded: Network packets shaped because the throughput exceeded the aggregated bandwidth limit.

It is important to note that every byte written to the primary node will be replicated to N replicas, N being the number of replicas. Clusters with small node types, multiple replicas, and intensive write requests may not be able to cope with the replication backlog. For such cases, it's a best practice to scale-up (change node type), scale-out (add shards in cluster-mode enabled clusters), reduce the number of replicas, or minimize the number of writes.

Also note that the "up to 5 Gigabit" is in bits (not bytes) and is the "burst bandwidth", (not sure how long a burst can be sustained on a t4g). The baseline bandwidth of a t4g.small is 0.128 Gbps (source).

Both you and I are gonna need larger instances.