Export log data to Amazon S3 using the console

1. Step 1: Create an Amazon S3 bucket. We recommend that you use a bucket that was created specifically for CloudWatch Logs.
2. Step 2: Create an IAM user with full access to Amazon S3 and CloudWatch Logs.
3. Step 3: Set permissions on an Amazon S3 bucket.
4. Step 4: Create an export task.

Monitor bucket storage using CloudWatch, which collects and processes storage data from Amazon S3 into readable, daily metrics. These storage metrics for Amazon S3 are reported once per day and are provided to all customers at no additional cost.

Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/. In the navigation pane, choose Insights. Choose one or more log groups and run a query. Choose Actions, Copy query results.

There are four recommended methods for retrieving log data from CloudWatch Logs:

1. Use subscription filters to stream log data to another receiving source in real time.
2. Run a query with CloudWatch Logs Insights.
3. Export log data to Amazon Simple Storage Service (Amazon S3) for batch use cases.

Logging Automation action output with CloudWatch Logs

1. In the navigation pane, choose Automation.
2. Choose the Preferences tab, and then choose Edit.
3. Select the check box next to Send output to CloudWatch Logs.
4. (Recommended) Select the check box next to Encrypt log data.

Go to the AWS CloudWatch console and click on Logs at the left most; select the CloudTrail Log group that we just created earlier, and click on Actions and select Stream to Amazon Elasticsearch Service.

Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ .

1. In the title bar, choose your AWS Identity and Access Management (IAM) user name, and then choose My Billing Dashboard.
2. In the navigation pane, choose AWS Cost & Usage Reports.

To find out size of S3 bucket using AWS Console:

1. Click the S3 bucket name.
2. Select “Management” tab.
3. Click “Metrics” navigation button.
4. By default you should see Storage metric of the bucket.

Once you configure the agent, logs start streaming from the EC2 instances and are sent to CloudWatch for analysis. You’ll want to analyze the EC2 error logs when troubleshooting issues with instances. The logs are sent to CloudWatch for analysis and can be stored in S3 buckets for archiving.

Export log data to Amazon S3 using the AWS CLI

1. Step 1: Create an Amazon S3 bucket.
2. Step 2: Create an IAM user with full access to Amazon S3 and CloudWatch Logs.
3. Step 3: Set permissions on an Amazon S3 bucket.
4. Step 4: Create an export task.
5. Step 5: Describe export tasks.
6. Step 6: Cancel an export task.

Amazon CloudWatch is a monitoring and management service that provides data and actionable insights for AWS, hybrid, and on-premises applications and infrastructure resources. With CloudWatch, you can collect and access all your performance and operational data in form of logs and metrics from a single platform.

For near real-time analysis of log data, see Analyzing log data with CloudWatch Logs Insights or Real-time processing of log data with subscriptions instead. Starting on February 15, 2019, the export to Amazon S3 feature requires callers to have s3:PutObject access to the destination bucket.

Using CloudWatch Logs, you can monitor log data in real time, search and filter log data by creating one or more metric filters, and archive and retrieve historical data when you need it. For more information about CloudWatch Logs, see the Amazon CloudWatch Logs User Guide .

To log data events for an S3 bucket to AWS CloudTrail and CloudWatch Events, create a trail. A trail captures API calls and related events in your account and delivers the log files to an S3 bucket that you specify. You can update an existing trail or create a new one.

If the existing bucket already has one or more policies attached to it, add the statements for CloudWatch Logs access to that policy or policies. We recommend that you evaluate the resulting set of permissions to be sure that they’re appropriate for the users who will access the bucket.