Billing Adjustment Request: Anomalous Data Transfer Charges

EC2 Data Transfer Out — Bot/Crawler Abuse of Misconfigured Web Server
Account ID
459395819858
Instance
i-0a10dbfcb309c390d
Region
eu-west-1
Instance Type
t3a.medium
Incident Period
2026-03-19 – 2026-04-02
Status
Resolved
Request: One-time billing credit for anomalous data transfer charges caused by automated bot/crawler traffic exploiting a misconfigured directory listing. This was not legitimate usage. The issue has been identified, resolved, and safeguards are being put in place to prevent recurrence.
Anomalous charges: Approximately $549 in data transfer across March and April 2026, compared to a normal monthly bill of ~$38.50. Total billed: ~$689 (including tax).
What Happened

I run a single t3a.medium EC2 instance as a personal development environment. A Caddy web server on the instance serves small static sites (total content: 88 KB).

On March 19, 2026, after a server reboot, Caddy started with the file_server browse directive enabled, which exposes directory listings as HTML pages. Automated bots discovered these listings and began scraping them continuously, generating ~440 GB/day of outbound data transfer.

The actual content being served was only 88 KB. The bots were recursively requesting directory listing pages, not transferring real data. This sustained traffic ran for 14 days before I identified and resolved it on April 2, 2026.

Cost Comparison: Normal vs. Incident
Expected Monthly Cost
$38.50 /month
t3a.medium compute$27.00
80 GB gp3 EBS$6.40
Public IPv4$3.60
VPC, SSM, misc$1.50
Actual March 2026 Bill
$634.30
EC2 (incl. data transfer)$508.22
Tax$110.08
EC2 Other (EBS, IPs)$14.40
VPC$1.60
Daily Cost — Data Transfer Out (March 19 – April 3)

Each bar represents one day. Data transfer charges at $0.09/GB. Normal daily data transfer cost: ~$0.

Mar 19
$17.78
Mar 20
$39.59
Mar 21
$40.61
Mar 22
$40.27
Mar 23
$38.57
Mar 24
$36.69
Mar 25
$39.38
Mar 26
$40.46
Mar 27
$39.50
Mar 28
$42.16
Mar 29
$39.63
Mar 30
$40.78
Mar 31
$39.75
Apr 1
$30.66
Apr 2
$10.19
Apr 3
$0.00
Period Data Transfer Cost Volume Total Bill
March 19–31 (13 days) ~$475 ~5,280 GB $634.30
April 1–2 (2 days) $40.85 553.87 GB $55.48
April 3+ (post-fix) $0.00 0.05 GB $1.08/day
Evidence: Bot Traffic, Not Legitimate Usage
  1. Content served totals 88 KB. The entire sites/ directory on this instance is 88 KB. The 5,800+ GB transferred was bots repeatedly requesting auto-generated directory listing HTML pages, not actual content.
  2. Constant rate, 24/7. System monitoring (sar) shows a perfectly flat ~5,400 kB/s outbound, around the clock, with no human usage patterns. This is characteristic of automated scraping.
  3. Traffic dropped to zero instantly on fix. The moment directory listings were disabled (April 2, 06:24 UTC), outbound traffic dropped from 5,400 kB/s to <3 kB/s within 6 minutes. No gradual decline — bots stopped getting responses.
  4. No legitimate visitors. This is a personal development server. The sites hosted are small prototypes with no public audience.
  5. February bill was $0. This account had no charges before the instance was created in March. There is no history of data transfer usage.
Network throughput (sar) — April 2, showing exact moment of fix
Time         txkB/s    Notes
06:10:06     5407.80   Sustained bot traffic
06:20:06     4247.67   Starting to drop (Caddy reload in progress)
06:30:02       15.27   Fix applied at 06:24 — traffic collapses
06:40:06        2.74   Normal idle
...
23:50:02        0.70   Remained idle
AWS Cost Explorer — Usage type, April 1 (representative day)
$30.66  (440.67 GB)  EU-DataTransfer-Out-Bytes    97% of cost
$ 0.97  ( 19.33 hr)  EU-CPUCredits:t3a
$ 0.96  ( 23.57 hr)  EU-BoxUsage:t3a.medium
$ 0.23  (  2.67 GB)  EU-EBS:VolumeUsage.gp3
$ 0.12  ( 24.00 hr)  EU-PublicIPv4:InUseAddress
AWS Cost Explorer — April 3 (post-fix, normal)
$ 0.80  (19.69 hr)  EU-BoxUsage:t3a.medium
$ 0.17  ( 1.89 GB)  EU-EBS:VolumeUsage.gp3
$ 0.10  (21.00 hr)  EU-PublicIPv4:InUseAddress
$ 0.00  ( 0.05 GB)  EU-DataTransfer-Out-Bytes    back to zero
Root Cause & Resolution

Cause

The Caddy web server configuration included the file_server browse directive on two virtual hosts, which generates HTML directory listing pages for any path. Internet bots discovered these endpoints and scraped them continuously, generating massive outbound traffic from auto-generated HTML.

Problematic configuration
dev.liztem.com {
    root * /home/ubuntu/projects/personal-os/sites
    file_server browse    # generates directory listings
}

Fix Applied — April 2, 2026 at 06:24 UTC

sudo sed -i 's/file_server browse/file_server/' /etc/caddy/Caddyfile
sudo systemctl reload caddy

Removing browse disables directory listing generation. The web server now returns 404 for directories without an explicit index.html file. Traffic dropped to near-zero immediately.

Preventive Measures Taken

  • Directory listing permanently disabled on all virtual hosts
  • HTTPS auto-enabled with TLS certificates (was HTTP-only during incident)
  • Setting up CloudWatch billing alarm for early anomaly detection
  • Implementing outbound traffic monitoring via system metrics
Requested Adjustment
Item Amount Notes
March 2026 anomalous data transfer ~$475 EU-DataTransfer-Out-Bytes, Mar 19–31
March 2026 associated tax on data transfer ~$100 Pro-rated from $110.08 total tax
April 2026 anomalous data transfer $40.85 EU-DataTransfer-Out-Bytes, Apr 1–2
April 2026 associated tax ~$8 Pro-rated
CPU credit overage (serving bot traffic) ~$15 EU-CPUCredits:t3a, above baseline
Total requested credit ~$639 Total bill ($689) minus expected normal cost (~$50 for the period)

This is my first time requesting a billing adjustment. The issue was caused by a web server misconfiguration on my end, but the resulting charges were entirely from automated bot traffic, not legitimate data transfer. The total static content on the server is 88 KB — the 5,800 GB transferred represents bots requesting auto-generated directory listing pages millions of times.

I have resolved the issue, confirmed that data transfer has returned to normal levels (~0 GB/day), and am implementing monitoring to prevent this from happening again.

I appreciate any credit AWS is able to provide. Thank you for your time.