FAQ

1 . A Service Linked Recorder 

1. • is used to automatically record configuration changes for a specific AWS service.

   • You can NOT define which resource types to record. Limited to the specific service it is linked to.

1. Custom Recorder:

   • A custom recorder is a Config Recorder that you manually configure to record changes for specific resources or all resources in your AWS account.

   • You can define which resource types to record and which regions to include. You have full control over the configuration

How Many Service Linked Recorders Can You Have with One Custom Recorder?

2. • If you enable a Service Linked Recorder for a specific service, it will use the same Config Recorder that you might have configured manually (the custom recorder).

   • In other words, you cannot have multiple Config Recorders in the same region. The Service Linked Recorder and custom recorder share the same Config Recorder instance.

Example:

2. • If you enable a Service Linked Recorder for AWS EC2, it will automatically start recording configuration changes for EC2 instances.

   • If you also have a custom recorder configured to record changes for all resources, the Service Linked Recorder will integrate with the same Config Recorder.

Summary

• You can have only one Config Recorder per region, which can be used for both Service Linked Recorders and custom recorders.

• A Service Linked Recorder is specific to an AWS service and automates the recording of configuration changes for that service.

• A custom recorder allows you to manually configure which resources and regions to record.

• Both types of recorders share the same Config Recorder instance in a region.

The scenario:

• You have a Service Linked Recorder enabled for RDS.

  • This means the Config Recorder is automatically configured to record changes for RDS resource types.

• Your custom recorder is not configured for RDS.

  • This implies that if you manually configured the custom recorder, you did not explicitly include RDS resource types.

Will You Get Change Events for RDS?

Yes you will receive change events for RDS because the Service Linked Recorder ensures that RDS resource types are recorded, even if your custom recorder is not explicitly configured for RDS. The Service Linked Recorder takes precedence for the service it is linked to :

1. The Service Linked Recorder for RDS automatically enables recording for RDS resource types in the Config Recorder.

2. Even if your custom recorder is not explicitly configured for RDS, the Service Linked Recorder overrides this and ensures RDS changes are recorded.

Key Points to Remember

1. Service Linked Recorder Takes Precedence:

   • If a Service Linked Recorder is enabled for a service (e.g., RDS), it will ensure that changes for that service are recorded, regardless of how the custom recorder is configured.

2. Single Config Recorder:

   • The Service Linked Recorder effectively modifies the configuration of the shared recorder to include RDS.

3. Custom Recorder Configuration:

   • If you manually configure the custom recorder to exclude RDS, the Service Linked Recorder will override this exclusion for RDS resource types.

Example

• Suppose you have a custom recorder configured to record only EC2 and S3 resource types.

• You then enable a Service Linked Recorder for RDS.

• Result: The Config Recorder will record changes for EC2, S3, and RDS because the Service Linked Recorder ensures RDS is included.

How to Verify

1. Go to the AWS Config Console.

2. Navigate to the Config Recorder settings for the region.

3. Check the resource types being recorded. You should see RDS listed if the Service Linked Recorder is enabled.

The choice between using a custom Config Recorder or Service Linked Recorder depends on your specific use case, requirements, and preferences. Here’s a breakdown of the pros and cons of each approach to help you decide what’s better for your scenario:

Custom Config Recorder

Pros:

1. Full Control:

   • You can explicitly specify which resource types and regions to record.

   • You have granular control over the configuration.

2. Flexibility:

   • You can include or exclude specific resource types based on your needs.

   • Useful if you only want to monitor a subset of resources (e.g., EC2, S3, etc.).

3. Cost Optimization:

   • By recording only the resources you care about, you can reduce the volume of configuration items and lower AWS Config costs.

4. Consistency:

   • You can apply the same configuration across multiple accounts and regions using automation (e.g., AWS CloudFormation, Terraform).

Cons:

1. Manual Setup:

   • Requires manual configuration, which can be time-consuming, especially in large environments with many resource types.

2. Maintenance Overhead:

   • If new resource types are introduced by AWS, you need to manually update the Config Recorder to include them.

3. Risk of Missing Resources:

   • If you forget to include a resource type, changes to those resources won’t be recorded.

Service Linked Recorder

Pros:

1. Automatic Setup:

   • Automatically configures the Config Recorder to record changes for the specific AWS service it’s linked to (e.g., RDS, S3).

   • No manual intervention is required.

2. Simplified Management:

   • Ideal if you want to ensure that all changes for a specific service are recorded without worrying about configuration.

3. Always Up-to-Date:

   • Automatically includes new resource types or features introduced by the linked service.

Cons:

1. Limited Control:

   • You cannot customize which resource types or regions are recorded for the linked service.

   • The Service Linked Recorder records all resource types for the linked service.

2. Potential Overhead:

   • May record more resources than you actually need, leading to higher AWS Config costs.

3. Service-Specific:

   • Each Service Linked Recorder is tied to a specific AWS service, so you may need multiple Service Linked Recorders for different services.

When to Use Custom Config Recorder

• You need granular control over which resource types and regions are recorded.

• You want to optimize costs by recording only the resources you care about.

• You have a well-defined set of resources to monitor and want consistency across accounts and regions.

When to Use Service Linked Recorder

• You want to simplify setup and management for specific AWS services.

• You need to ensure that all changes for a specific service (e.g., RDS, S3) are recorded without manual configuration.

• You are okay with recording all resource types for the linked service, even if some are not needed.

Hybrid Approach

In many cases, a hybrid approach works best:

• Use a custom Config Recorder for resources where you need granular control (e.g., EC2, IAM).

• Use Service Linked Recorders for services where you want automatic and comprehensive recording (e.g., RDS, S3).

This way, you get the benefits of both approaches:

• Flexibility and cost optimization from the custom recorder.

• Simplicity and completeness from the Service Linked Recorder.

Recommendation

• If you have a small set of resources to monitor and want full control, use a custom Config Recorder.

• If you want to reduce management overhead and ensure comprehensive recording for specific services, use Service Linked Recorders.

• For most production environments, a hybrid approach is the best balance of control, simplicity, and cost efficiency.

• With a custom Config Recorder, you have the flexibility to selectively include or exclude specific resource types.

• For example, you can choose to record ( include ) changes only for AWS::RDS::DBInstance and exclude:

  • AWS::RDS::DBSnapshot

  • AWS::RDS::DBSubnetGroup

  • Any other RDS resource types you don’t need.

This granular control allows you to:

• Optimize costs by recording only the resources you care about. i.e  Events for AWS::RDS::DBSnapshot, AWS::RDS::DBSubnetGroup, etc., will not be recorded.

• Reduce noise by excluding irrelevant resource types.

Service Linked Recorder

• With a Service Linked Recorder, you do not get this flexibility.

• It will automatically record changes for all RDS resource types, including:

  • AWS::RDS::DBInstance

  • AWS::RDS::DBSnapshot

  • AWS::RDS::DBSubnetGroup

  • And any other RDS-related resource types.

This means:

• You cannot exclude specific RDS resource types. 

• You may end up recording more data than you need, which could increase costs and storage requirements.

Key Difference

• Custom Config Recorder:

  • You can pick and choose specific resource types (e.g., only AWS::RDS::DBInstance).

  • Provides fine-grained control over what is recorded.

• Service Linked Recorder:

  • Records all resource types for the linked service (e.g., all RDS resource types).

  • Offers no flexibility to exclude specific resource types.

Which Should You Use?

• Use a custom Config Recorder if:

  • You want fine-grained control over which resource types are recorded.

  • You want to exclude specific resource types (e.g., AWS::RDS::DBSnapshot).

  • You want to optimize costs by recording only what you need.

• Use a Service Linked Recorder if:

  • You want to simplify setup and don’t mind recording all resource types for the service.

  • You need to ensure comprehensive coverage of all changes for the linked service this may be for compliance reasons 

Topic rephrased as question. : config rules check for resouce compliance so when a change happens that triggers config rule that identifies non compliance at the same time this event will also trigger config event that is published to sns topic that has sqs or lambda that process that same event and will need to same check for compliance and then either remediate or do nothing if compliant so will this not duplicate the event and why shoud i set up config rule when i have a config recorder and lambda setup

1. Config Rules:

   • Continuously evaluate the configuration of your AWS resources against desired rules (e.g., "RDS instances must have encryption enabled").

   • When a resource is non-compliant, Config Rules generate a compliance change event.

   • These events can trigger automated remediation (e.g., using AWS Systems Manager Automation or Lambda).

2. Config Recorder:

   • Records configuration changes for your resources.

   • Publishes these changes as configuration change events to an SNS topic, which can trigger Lambda or SQS for further processing.

The way Lambda Setup Works 

1. Config Recorder:

   • Records configuration changes and publishes them to an SNS topic.

2. Lambda Function:

   • Processes the event, checks for compliance, and performs remediation if necessary.

Potential Overlap

• If you have both Config Rules and a Lambda-based setup, there is a potential for duplication:

  • Config Rules will evaluate compliance and generate compliance change events.

  • Your Lambda function will also evaluate compliance and potentially perform remediation.

This means:

• The same resource change could trigger two compliance checks:

  1. By the Config Rule.

  2. By your Lambda function.

• This could lead to redundant processing and unnecessary costs.

Why You Might Still Want Config Rules

Despite the potential overlap, there are reasons to use Config Rules alongside your Lambda setup:

1. Built-in Compliance Dashboard:

   • AWS Config provides a centralized compliance dashboard that shows the compliance status of all your resources.

   • This is useful for auditing and reporting purposes.

2. Out-of-the-Box Rules:

   • AWS Config provides managed rules (e.g., "rds-storage-encrypted") that you can use without writing custom logic.

   • These rules are maintained by AWS and updated as new compliance requirements are introduced.

3. Compliance History:

   • Config Rules maintain a history of compliance changes, which can be useful for troubleshooting and auditing.

4. Integration with Other AWS Services:

   • Config Rules can integrate with AWS Security Hub and other services to provide a unified view of your security and compliance posture.

Why You Might Not Need Config Rules

If your Lambda-based setup already handles compliance checks and remediation, you might not need Config Rules because:

1. Custom Logic:

   • Your Lambda function can implement custom compliance logic tailored to your specific requirements.

2. Cost Optimization:

   • Running both Config Rules and a Lambda function for the same checks could lead to unnecessary costs.

3. Simplified Architecture:

   • Relying on a single system (your Lambda function) for compliance checks and remediation can simplify your architecture.

Best Approach

To avoid duplication and optimize your setup, consider the following:

Option 1: Use Config Rules and Disable Lambda Compliance Checks

• Let Config Rules handle compliance evaluation and remediation.

• Use the Config compliance dashboard for reporting and auditing.

• Disable the compliance check logic in your Lambda function and use it only for non-compliance remediation (if Config Rules cannot handle the remediation directly).

Option 2: Use Lambda and Disable Config Rules

• Use your Lambda function for both compliance checks and remediation.

• Disable Config Rules to avoid redundant checks.

• Implement your own compliance dashboard (e.g., using CloudWatch Metrics or a custom solution) if needed.

Option 3: Hybrid Approach

• Use Config Rules for high-level compliance monitoring and reporting.

• Use your Lambda function for custom compliance checks and remediation that Config Rules cannot handle.

• Ensure there is no overlap in the checks performed by Config Rules and your Lambda function.

Key Considerations

1. Cost:

   • Config Rules and Lambda both incur costs. Evaluate whether the duplication is justified based on your use case.

2. Complexity:

   • A hybrid approach might increase complexity, so weigh the benefits against the added complexity.

3. Compliance Requirements:

   • If you need a centralized compliance dashboard or out-of-the-box rules, Config Rules might be worth the duplication.

Conclusion

• Yes, there is potential duplication if you use both Config Rules and a Lambda-based setup for compliance checks.

• You do not necessarily need both. If your Lambda function already handles compliance checks and remediation, you can skip Config Rules.

• However, Config Rules provide additional benefits like a centralized compliance dashboard, out-of-the-box rules, and compliance history, which might justify their use in some scenarios.

Choose the approach that best aligns with your cost, complexity, and compliance requirements!

Scenario: Custom Tagging Compliance with Dynamic Business Rules

Use Case:

Your organization has a dynamic tagging policy for AWS resources, where:

1. Resources must be tagged with a ProjectID and Environment (e.g., ProjectID: 123, Environment: Production).

2. The valid values for Environment are dynamically fetched from an external system (e.g., a database or API).

3. If a resource is non-compliant, it must be automatically tagged with the correct values or notified to the responsible team.

Why Config Rules Cannot Handle This

1. Dynamic Validation:

   • Config Rules are designed to evaluate compliance against static rules (e.g., "Is the Environment tag present?").

   • They cannot dynamically fetch valid values from an external system (e.g., a database or API) to validate tags.

2. Custom Remediation:

   • Config Rules can trigger automated remediation (e.g., using AWS Systems Manager Automation), but the remediation logic is limited to predefined actions.

   • They cannot implement custom logic like fetching valid tag values from an external system and applying them to non-compliant resources.

3. Complex Business Logic:

   • Config Rules are not designed to handle complex business logic (e.g., "If the resource is in the Production environment, it must also have a CostCenter tag").

How a Lambda Function Can Handle This

A custom Lambda function can be used to:

1. Fetch Valid Tag Values:

   • Query an external system (e.g., a database or API) to fetch the valid values for the Environment tag.

2. Evaluate Compliance:

   • Check if the resource has the required tags (ProjectID and Environment).

   • Validate that the Environment tag value is one of the valid values fetched from the external system.

3. Perform Custom Remediation:

   • If the resource is non-compliant, the Lambda function can:

     • Automatically apply the correct tags.

     • Send a notification to the responsible team (e.g., via SNS or Slack).

4. Handle Complex Logic:

   • Implement additional business rules (e.g., "If the resource is in the Production environment, ensure it also has a CostCenter tag").

Example Workflow

1. Config Recorder:

   • Detects a configuration change (e.g., an EC2 instance is launched).

   • Publishes the event to an SNS topic.

2. Lambda Function:

   • Subscribes to the SNS topic and processes the event.

   • Fetches valid tag values from an external system.

   • Checks if the EC2 instance is compliant with the tagging policy.

   • If non-compliant:

     • Applies the correct tags.

     • Sends a notification to the responsible team.

Why This Cannot Be Done with Config Rules

1. Config Rules:

   • Can check if a resource has the required tags (ProjectID and Environment).

   • Cannot dynamically validate the Environment tag against an external system.

   • Cannot implement custom remediation logic (e.g., fetching valid tag values and applying them).

2. Lambda Function:

   • Can handle dynamic validation and custom remediation.

   • Can implement complex business logic that Config Rules cannot support.

Another Example: Cost Optimization for Unused Resources

Use Case:

You want to identify and remediate unused resources (e.g., unattached EBS volumes, idle RDS instances) based on custom criteria:

1. An EBS volume is considered unused if it has been unattached for more than 30 days.

2. An RDS instance is considered idle if it has had no connections for more than 7 days.

Why Config Rules Cannot Handle This

1. Custom Metrics:

   • Config Rules cannot evaluate custom metrics like "time since last attachment" or "time since last connection."

2. Complex Logic:

   • Config Rules cannot implement logic like "if unattached for more than 30 days, delete the volume."

How a Lambda Function Can Handle This

1. Custom Metrics:

   • The Lambda function can query CloudWatch metrics or other sources to determine if a resource is unused.

2. Complex Logic:

   • The Lambda function can implement custom logic like:

     • "If an EBS volume has been unattached for more than 30 days, delete it."

     • "If an RDS instance has had no connections for more than 7 days, stop it."

Conclusion

In scenarios like dynamic tagging compliance or cost optimization for unused resources, Config Rules are limited by their inability to:

• Fetch data from external systems.

• Implement complex or dynamic business logic.

• Perform custom remediation actions.

A Lambda function, on the other hand, can handle these use cases effectively, making it the better choice for custom compliance checks and remediation. This is why a hybrid approach (Option 3) can be beneficial: use Config Rules for standard compliance checks and Lambda for custom or complex scenarios.

AWS Systems Manager (SSM) Automation is a powerful tool for automating operational tasks, such as patching instances, creating AMIs, or remediating non-compliant resources. However, it does have some limitations that might make it less suitable for certain use cases compared to custom Lambda functions. Let’s explore these limitations in detail, along with examples.

1. Limited Custom Logic and Flexibility

Limitation:

SSM Automation is designed for predefined, structured workflows. While it supports some conditional logic (e.g., branching based on step outputs), it is not as flexible as writing custom code in a Lambda function.

Example:

• You want to implement a remediation workflow that:

  • Checks if an EC2 instance is running an outdated AMI.

  • If yes, creates a new AMI, launches a new instance, and validates the new instance before terminating the old one.

  • If the validation fails, roll back the changes.

• SSM Automation:

  • You can create an automation document with steps for each task, but implementing complex rollback logic or dynamic decision-making is challenging.

• Lambda Function:

  • You can write custom logic to handle the entire workflow, including rollback and dynamic decision-making.

2. Limited Integration with External Systems

Limitation:

SSM Automation has limited ability to interact with external systems (e.g., APIs, databases) or fetch data dynamically.

Example:

• You want to fetch a list of approved AMI IDs from an external API and ensure that all EC2 instances are using one of these AMIs.

• SSM Automation:

  • Cannot directly call an external API to fetch the list of approved AMIs.

• Lambda Function:

  • Can call the external API, fetch the list, and compare it with the AMIs used by EC2 instances.

3. Limited Error Handling and Retry Logic

Limitation:

SSM Automation provides basic error handling (e.g., stopping the workflow if a step fails), but it lacks advanced error handling and retry mechanisms.

Example:

• You want to retry a step (e.g., creating an AMI) up to 3 times if it fails due to transient issues (e.g., API throttling).

• SSM Automation:

  • Does not natively support retry logic for individual steps.

• Lambda Function:

  • Can implement custom retry logic with exponential backoff.

4. Limited Support for Complex Data Processing

Limitation:

SSM Automation is not designed for complex data processing or transformations.

Example:

• You want to parse a JSON response from an API, extract specific fields, and use them in subsequent steps.

• SSM Automation:

  • Cannot easily parse JSON or perform complex data transformations.

• Lambda Function:

  • Can parse JSON, extract fields, and perform any required transformations.

5. Limited Support for Long-Running Workflows

Limitation:

SSM Automation has a maximum execution time of 12 hours. For longer-running workflows, you need to break them into smaller automations or use a different tool.

Example:

• You want to perform a multi-day data migration or backup process.

• SSM Automation:

  • Cannot handle workflows that run longer than 12 hours.

• Lambda Function:

  • Can orchestrate long-running workflows by breaking them into smaller tasks and using Step Functions or other tools.

6. Limited Support for Custom Notifications

Limitation:

SSM Automation can send notifications via SNS, but it lacks flexibility in customizing the notification content or integrating with third-party systems (e.g., Slack, PagerDuty).

Example:

• You want to send a detailed notification to Slack with the results of a remediation workflow.

• SSM Automation:

  • Can send a basic SNS notification but cannot format the message for Slack or include dynamic content.

• Lambda Function:

  • Can format the message for Slack, include dynamic content, and send it via a webhook.

7. Limited Support for Cross-Account Operations

Limitation:

SSM Automation can perform cross-account operations, but it requires additional setup (e.g., IAM roles) and has limited flexibility compared to Lambda.

Example:

• You want to remediate non-compliant resources in multiple AWS accounts.

• SSM Automation:

  • Requires setting up IAM roles and permissions for each account.

  • Cannot easily switch between accounts dynamically.

• Lambda Function:

  • Can assume roles in multiple accounts and perform actions across them dynamically.

8. Limited Support for Non-EC2 Resources

Limitation:

SSM Automation is primarily designed for EC2 instances and related resources. While it can interact with other AWS services, its capabilities are limited compared to Lambda.

Example:

• You want to remediate non-compliant S3 buckets (e.g., enabling encryption or disabling public access).

• SSM Automation:

  • Can perform basic S3 operations but lacks the flexibility to implement complex remediation logic.

• Lambda Function:

  • Can implement any required logic for S3 remediation.

9. Limited Debugging and Logging

Limitation:

SSM Automation provides basic logs for each step, but debugging complex workflows can be challenging.

Example:

• You want to debug a workflow that failed due to an unexpected condition.

• SSM Automation:

  • Provides limited logs and debugging capabilities.

• Lambda Function:

  • Can log detailed information to CloudWatch Logs for easier debugging.

When to Use SSM Automation vs. Lambda

Use SSM Automation:

• For predefined, structured workflows (e.g., patching instances, creating AMIs).

• When you need out-of-the-box integrations with AWS services (e.g., EC2, RDS).

• For simple error handling and notifications.

Use Lambda:

• For custom logic or complex workflows.

• When you need to integrate with external systems or process complex data.

• For advanced error handling, retry logic, or long-running workflows.

Conclusion

While SSM Automation is a powerful tool for automating operational tasks, it has limitations in terms of custom logic, external integrations, error handling, and long-running workflows. For more complex or dynamic use cases, a Lambda function is often a better choice due to its flexibility and ability to integrate with external systems. Choose the tool that best fits your specific requirements! 

AWS Config Recorder Limits:

1. Per AWS Account:

• You can create only ONE AWS Config recorder per region

• This means 1 recorder per account per region

Central Management Approach

1. AWS Organizations Integration

• You can centrally manage AWS Config across accounts using AWS Organizations

• Enables delegated administrator and central configuration

Recommended Architecture:

1. Management Account (Delegated Administrator)

   • Create organization-wide configuration rules

   • Aggregate config data from member accounts

   • Use AWS Config organization features

Key Steps

1. Enable AWS Config in management account

2. Enable AWS Organizations integration

3. Use AWS Config organization features

   • Organization Config Rules

   • Centralized configuration

   • Consolidated view of compliance

Configuration Options

• AWS Config Aggregator

• AWS Config Organization Settings

• AWS Config Multi-account Multi-region Data Aggregation

Best Practices

• Use delegated administrator

• Standardize configuration rules

• Implement central monitoring

• Leverage AWS Organizations for governance