AWS S3 Storage and Request Cost Optimization

The client was facing an unexpectedly high AWS bill of nearly $25,000 per month, primarily driven by Amazon S3 storage and request charges. The increase was not aligned with application growth, indicating inefficiencies in how log data was being stored and managed.

• Sector: Banking
• Scale: High-volume log ingestion across multiple S3 buckets
• Goal: Bring AWS S3 costs under control while maintaining required log retention and auditability.

The platform relied heavily on S3 for centralized log storage:

1. Log Sources: Application logs, load balancer access logs, and service logs continuously written to S3.
2. Storage Layer: Multiple large S3 buckets with server access logging enabled.
3. Governance: No effective lifecycle or retention enforcement.

• Runaway Costs: Monthly S3 spend had grown to ~$25k due to unbounded log retention.
• Request Amplification: Misconfigured S3 server access logging caused recursive logging, dramatically increasing PUT and GET requests.
• Low Visibility: Cost drivers were spread across multiple buckets and prefixes, making root-cause analysis non-trivial.

A focused S3 cost-reduction initiative was executed.

• Lifecycle Enforcement: Verified and corrected lifecycle policies to transition older logs to Glacier and expire data beyond compliance needs.
• Logging Optimization: Identified and fixed bucket logging configurations that were logging high-traffic buckets into themselves or other hot buckets.
• Cost Monitoring: Introduced bucket-level cost visibility to prevent future regressions.

Results:

• Monthly Cost: Reduced AWS S3 spend from $25,000/month to $3,000/month (~88% reduction).
• Storage: Eliminated large volumes of redundant and stale log data.
• Requests: Massive drop in S3 PUT/GET request volume after correcting logging behavior.
• Reliability: No impact to production workloads, compliance, or observability.