S3 Logging bucket controls in error or alarm

AWS imposes throttle limits which can impact resource creation and setup, namely S3 logging buckets in various regions. This can include buckets not being created, DynamoDB entries not existing, and bucket or role policy misconfiguration. 

In order to perform the fix, access to the Turbot Master DynamoDB table is recommended, as is AWS/SuperUser in the affected account. Utilizing SuperUser will negate the potential authorization errors when attempting to make changes.

To fix Turbot bucket logging-related issues:

  1. Verify that the region with a S3 logging bucket error does in fact have a bucket that is named in the form $turbotAccountName-$region-<random string>, i.e. if the AWS account is named aab in Turbot and the region US East 2 is having troubles, the bucket will be named similar to aab-usea2-h5gqico31eax. If it does not exist, the bucket name can be found in the next steps.
  2. If the bucket does not already exist, copy the bucket name and create a bucket using that name. The bucket policy can be default. The bucket name can be found in a control in error.
  3. Check the Turbot Master account DynamoDB table TurbotAccounts. Search for the account with an error - if there is an entry for the region with an issue (using the example from above, look for the key aws•logs•us-east-2), delete it. (This step is optional)
  4. At this point, there should be a bucket in the region that is having an issue and no DynamoDB entry. Within the account that has the errors, navigate to CloudFormation.
  5. Double check that the region selected is correct then delete the account stack. Using the example from step 1, that stack will be titled aab.
  6. The stack should run and complete in a few minutes, but if a new stack is not being created, go to the account in Turbot. Click on the controls tab and find an alarm with the title CloudFormation Account Configuration in $region. Select the control, run a check and then apply. This will kick off the CloudFormation stack. Both a new bucket and the DynamoDB entry will be created.
  7. The old bucket can either be deleted or renamed depending on the amount of data being stored. If logging was misconfigured, it is likely that the bucket is empty, or close to it, and can be deleted without losing audit logs. However, confirm that is the case by viewing the contents of the old bucket.
  8. Ensure that the turbot_config IAM role has a correctly configured policy titled turbot_config_s3_log_access. The action s3:PutObject* and s3:GetBucketAcl must have the new s3 bucket resource urn. An example is the following:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject*"
],
"Resource": [
"arn:aws:s3:::$turbotAccountID-$region-<random string>/AWSLogs/$awsAccountID/*"
],
"Condition": {
"StringLike": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketAcl"
],
"Resource": [
"arn:aws:s3:::$turbotAccountID-$region-<random string>"
]
}
]
}

Ensure that each region is present in this policy. Once all the above steps have been completed, Turbot logging alarms should clear within 24 hours, though they can be cleared manually from within Turbot.

If issues arise or there is any concern with any of the steps, please do not hesistate to reach out to Turbot support at help@turbot.com

Was this article helpful?
0 out of 0 found this helpful