This content originally appeared on DEV Community and was authored by Wendy Wong
A Very Expensive Lesson
I received a huge surprise this month, a forecast for a July month end bill of $586 linked to my AWS root account. As best practice I never use my AWS root account and always build projects with the AWS IAM account login.
The AWS Management Console provides a summary of the main AWS services used forecasted costs, in particular indicating that Amazon Sagemaker had an end of month $578.57 forecast and a cost increasing by over 300% in the current month. I have not used Amazon Sagemaker recently 🙄 and this raised concerns for my unexpected charges.
Amazon Cost Explorer
By navigating to Amazon Cost Explorer, AWS Cost Management provides a summary view of costs, commentary on the right hand side and also a graph outlining daily usage had increased by 270% in the region US-East-1 (N.Virginia).
I navigated each of the panes to investigate further information regarding the costs incurred such as Region, Instance Type and Linked Account.
Instance Type
I checked under Type to see if there were any active EC2 instances. There were no active instances after 11 July 2022.
Usage Type
In this pane, I could identify that after 11 July 2022, someone had created an Amazon Sagemaker Canvas session from 11-17 July (indicated in purple)
Linked Account
I used the AWS root account login details to check any active sessions of Amazon Sagemaker Canvas.
Reduce Costs - Clean up Active AWS resources
The previous week on the 11 July, I cleaned up my AWS account for active resources such as:
a) Deleting EC2 instances in the regions Sydney, Ohio and N.Virginia
b) Shutting down any active Cloud9 sessions
c) Deleting any active Cloud Formation stacks
d) Deleting any AWS Sagemaker processing jobs
e) Emptying S3 buckets for completed AWS Glue jobs
Reduce Costs - Create a Cost Anomaly Detection Report
As an extra measure I created a Cost Anomaly Report that could be emailed to me to identify any suspicious activity to my AWS account over a threshold of $15. You may create a Cost Anomaly Detection Report from this link
You may also download as a csv file the Cost Anomaly Detection Report to view the details of the surprise charges:
Reduce Costs - Delete if not in use - Amazon Sagemaker Canvas
Navigate to Amazon Sagemaker Canvas and click on Canvas and then click Getting Started. I located active sessions of an unauthorized user 'michael-c' who has been generating workloads $40 per day from 13-18 July on my account. I followed the instructions to remediate and prevent unauthorized user from this link
I deleted the app from Amazon Sagemaker Canvas and then deleted the user.
From the diagram you may see an unauthorized user 'michael-c'.
AWS Cloud Watch
I further explored AWS Cloud Watch to confirm the usage levels of Amazon Sagemaker Canvas used by user 'michael-c' in US East - region (N.Virginia) to understand how many days I was being charged from the Canvas instance.
Cloud Watch was able to monitor the metrics of an AWS Service and provide further evidence for my investigation of the surprise charges.
Delete and Block Public Access - Amazon S3 bucket
I wanted to review my Amazon S3 buckets to see if I could further reduce my monthly bill. This is what I discovered:
a) User 'michael-c' had created an S3 bucket with my AWS account number and included a prefix with region US- East-1 (N.Virginia)
I emptied the files within the bucket.
b) This S3 bucket was Public and not private. I updated the settings to Block Public S3 buckets.
Security Layers - AWS IAM
To further secure my AWS Root account I also implemented Multi-Factor Authentication (MFA) as recommended on the AWS IAM dashboard.
Best practice guidelines to enable MFA include downloading Twilio Authy for an iPhone or Android phone and link it to your mobile device. You may find the steps here.
MFA provides an additional layer of security to identify the user during the login process of your AWS account.
Final Steps - Change passwords
As an extra measure to prevent unauthorized access to my AWS account, I also reset the passwords to the AWS IAM and Root user accounts.
I hope you won't have to learn the hard way like me. Until next time, happy learning! 😁
This content originally appeared on DEV Community and was authored by Wendy Wong
Wendy Wong | Sciencx (2022-07-18T16:41:32+00:00) AWS Cost Explorer – Cost Anomaly Detection Report identifed an unauthorized Amazon Sagemaker Canvas user. Retrieved from https://www.scien.cx/2022/07/18/aws-cost-explorer-cost-anomaly-detection-report-identifed-an-unauthorized-amazon-sagemaker-canvas-user/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.