AWS cloud cost optimization By AWS Lambda - Identifying Stale Resources

In this example, we'll create a Lambda function that identifies EBS snapshots that are no longer associated with any active EC2 instance and deletes them to save on storage costs.

The Lambda function fetches all EBS snapshots owned by the same account ('self') and also retrieves a list of active EC2 instances (running and stopped). For each snapshot, it checks if the associated volume (if exists) is not associated with any active instance. If it finds a stale snapshot, it deletes it, effectively optimizing storage costs.

  1. create a t2micro ubuntu EC2 instance

  2. create a snapshot of volume of ec2 instance.

  3. create a lambda function - runtime is python and create.

  4. in the code section of lambda function, add this file - and deploy : -

    1. importing the boto3 Library:

       import boto3
      

      This line imports the boto3 library, which is the AWS SDK for Python. It provides the necessary functionality to interact with AWS services, including EC2 (Elastic Compute Cloud) for snapshot management.

    2. Lambda Handler Function:

       def lambda_handler(event, context):
      

      The lambda_handler function is the entry point for the AWS Lambda function. AWS Lambda expects this function to be present in your code, and it is automatically executed when the Lambda function is triggered.

    3. Initializing the EC2 Client:

       ec2 = boto3.client('ec2')
      

      In this line, an EC2 client is created using boto3. This client allows the Lambda function to interact with EC2 services, including snapshots, instances, and volumes.

    4. Getting All EBS Snapshots:

       response = ec2.describe_snapshots(OwnerIds=['self'])
      

      This code retrieves a list of all EBS snapshots that belong to the AWS account specified by 'self'. These are snapshots owned by your AWS account.

    5. Getting Active EC2 Instance IDs:

       instances_response = ec2.describe_instances(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
       active_instance_ids = set()
      

      The script fetches information about all currently running EC2 instances. It uses a filter to select only instances that are in the "running" state, and it collects the IDs of these running instances in a set named active_instance_ids.

    6. Iterating Through EBS Snapshots:

       for snapshot in response['Snapshots']:
      

      This loop iterates through each EBS snapshot retrieved in the response.

    7. Extracting Snapshot and Volume Information:

       snapshot_id = snapshot['SnapshotId']
       volume_id = snapshot.get('VolumeId')
      

      For each snapshot, it extracts the SnapshotId and attempts to retrieve the associated VolumeId.

    8. Deleting Unnecessary Snapshots:

      • If a snapshot is not associated with any volume (i.e., volume_id is None), the script deletes it, as it's not in use.

      • If a snapshot is associated with a volume but the volume is not attached to any running instance, the snapshot is also deleted.

    9. Error Handling:

       except ec2.exceptions.ClientError as e:
           if e.response['Error']['Code'] == 'InvalidVolume.NotFound':
      

      The code includes error handling to check if an associated volume no longer exists. If the associated volume is not found (with an error code of 'InvalidVolume.NotFound'), the script deletes the snapshot as its associated volume may have been deleted.

In summary, this AWS Lambda function is designed to automatically manage and clean up EBS snapshots. It retrieves all EBS snapshots owned by your AWS account, identifies snapshots that are no longer in use (not attached to any volume or not attached to a running instance), and deletes them. This script can be scheduled to run periodically, ensuring that your AWS environment's storage is efficiently managed.

whole file is -

  1.    import boto3
    
       def lambda_handler(event, context):
           # Initialize an EC2 client
           ec2 = boto3.client('ec2')
    
           # Get all EBS snapshots
           response = ec2.describe_snapshots(OwnerIds=['self'])
    
           # Get all active EC2 instance IDs
           instances_response = ec2.describe_instances(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
           active_instance_ids = set()
    
           # Collect the IDs of running instances
           for reservation in instances_response['Reservations']:
               for instance in reservation['Instances']:
                   active_instance_ids.add(instance['InstanceId'])
    
           # Iterate through each snapshot and delete if it's not attached to any volume or the volume is not attached to a running instance
           for snapshot in response['Snapshots']:
               snapshot_id = snapshot['SnapshotId']
               volume_id = snapshot.get('VolumeId')
    
               if not volume_id:
                   # Delete the snapshot if it's not attached to any volume
                   ec2.delete_snapshot(SnapshotId=snapshot_id)
                   print(f"Deleted EBS snapshot {snapshot_id} as it was not attached to any volume.")
               else:
                   # Check if the volume still exists
                   try:
                       volume_response = ec2.describe_volumes(VolumeIds=[volume_id])
                       if not volume_response['Volumes'][0]['Attachments']:
                           ec2.delete_snapshot(SnapshotId=snapshot_id)
                           print(f"Deleted EBS snapshot {snapshot_id} as it was taken from a volume not attached to any running instance.")
                   except ec2.exceptions.ClientError as e:
                       if e.response['Error']['Code'] == 'InvalidVolume.NotFound':
                           # The volume associated with the snapshot is not found (it might have been deleted)
                           ec2.delete_snapshot(SnapshotId=snapshot_id)
                           print(f"Deleted EBS snapshot {snapshot_id} as its associated volume was not found.")
    

    2nd time deleted snap because it is not attached to ec2

    when volume not found -

    thanks for reading :)