0

So I have created a lambda function (the purpose of which doesn't matter anymore) and have tested that it works when run from my laptop. However the bigger problem is that I cannot get it to run off of a test event or on a schedule in AWS.

When I try to run it from AWS I get a 300s timeout error.

The following is included for your consideration:

  • Function
  • Logs
  • Trigger Event
  • Policy
  • VPC Related Configuration

If anyone can tell me what the issue might be, I would appreciate it as I have been searching for the solution for about 3 days.

FUNCTION:

from __future__ import print_function

def lambda_handler(event, context):
    if event["account"] == "123456789012":
        print("Passed!!")
    return event['time']


import boto3
import datetime

def find_auto_scaling_instances():
    """
    Find Auto-scaling Instances
    """
    client=boto3.client("autoscaling")
    auto_scaling=client.describe_auto_scaling_groups()
    dict={}
    for group in auto_scaling["AutoScalingGroups"]:
        print("hello")
        auto_scaling_group_name=group["AutoScalingGroupName"]
        number_of_instances=len(group["Instances"])
        if number_of_instances == 0:
            continue
        else:
            for i in range(number_of_instances):
                if group["Instances"][i]["LifecycleState"] == "InService":
                    instance_id=group["Instances"][i]["InstanceId"]
                    dict[auto_scaling_group_name]=instance_id
                    break
    return dict

def find_staging_instances():
    """
    Find Static Instances
    """
    stg_list=[]
    tag_list=["Y-qunie-stepsrv-1a","S-StepSvr"]
    for i in range(num_of_instances):
        print("hello2")
        target=instances[i]["Instances"][0]
        number_of_tags=len(target["Tags"])
        for tag in range(number_of_tags):
            if target["Tags"][tag]["Value"] in tag_list:
                stg_list+=[target["InstanceId"]]

    return stg_list

def volumes_per_instance():
    """
    Find the EBS associated with the Instances
    """
    instance_disk={}
    for i in range(num_of_instances):
        print("hello3")
        target=instances[i]["Instances"][0]
        if target["InstanceId"] in instance_list:
            instance_disk[target["InstanceId"]]=[]
            for disk in range(len(target["BlockDeviceMappings"])):
                print("hello4")
                instance_disk[target["InstanceId"]]+=\
                target["BlockDeviceMappings"][disk]["Ebs"]["VolumeId"]]

    return instance_disk

#Group instances together and prepare to process
instance_in_asgroup_dict=find_auto_scaling_instances()

as_instance_list=[]
for group in instance_in_asgroup_dict:
    print("hello5")
    as_instance_list+=[instance_in_asgroup_dict[group]]

client=boto3.client("ec2")
instances=client.describe_instances()["Reservations"]
num_of_instances=len(instances)

staging_instances=find_staging_instances()

instance_list=[]
instance_list+=as_instance_list
instance_list+=staging_instances

#Gather Disk Information
inst_disk_set=volumes_per_instance()

date=str(datetime.datetime.now()).replace(" \
","_").replace(":","").replace(".","")

#Create Disk Images
as_image={}
image=[]
for instance in instance_list:
    print("hello6")
    if instance in as_instance_list:
        as_image[instance]=client.create_image(
            InstanceId=instance,
            Name=instance+"-"+date+"-AMI",
            Description="AMI for instance "+instance+".",
            NoReboot=True
        )
    else:
        image+=[client.create_image(
            InstanceId=instance,
            Name=instance+"-"+date+"-AMI",
            Description="AMI for instance "+instance+".",
            NoReboot=True
        )]

LOGS:

18:03:30
START RequestId: 0ca9e0a3-7f11-11e6-be11-6974d9213102 Version: $LATEST

18:08:30
END RequestId: 0ca9e0a3-7f11-11e6-be11-6974d9213102

18:08:30
REPORT RequestId: 0ca9e0a3-7f11-11e6-be11-6974d9213102  Duration: 300001.99 ms  Billed Duration: 300000 ms Memory Size: 128 MB  Max Memory Used: 24 MB

18:08:30
2016-09-20T09:08:30.544Z 0ca9e0a3-7f11-11e6-be11-6974d9213102 Task timed out after 300.00 seconds

TRIGGER_EVENT:

{
  "account": "123456789012",
  "region": "us-east-1",
  "detail": {},
  "detail-type": "Scheduled Event",
  "source": "aws.events",
  "time": "1970-01-01T00:00:00Z",
  "id": "cdc73f9d-aea9-11e3-9d5a-835b769c0d9c",
  "resources": [
    "arn:aws:events:us-east-1:123456789012:rule/my-schedule"
  ]
}

EDIT-1

IAM POLICY:

From my understanding all I need to allow VPC Access to my function is to add the following privilages to the lambda function's assigned policy.

  • ec2:CreateNetworkInterface
  • ec2:DeleteNetworkInterface
  • ec2:DescribeNetworkInterfaces
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "ec2:CreateNetworkInterface",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DeleteNetworkInterface"
            ],
            "Resource": "*"
        },
        {
            "Action": [
                "ec2:DescribeInstances",
                "ec2:CreateImage",
                "autoscaling:DescribeAutoScalingGroups"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        }
    ] }

EDIT 2

CONFIGURATION:

The subnets, security groups and VPC attached to the function.

Configurations

EDIT 3 (CONCLUSION)

Mark Gave an excellent answer, informing me that I had set my function up to run inside a VPC yet I was not accessing resources within the VPC. Rather, I was accessing the Amazon API endpoint which required that I have access to the internet or the transaction would timeout.

As such, there were two options available to fix this situation.

  • Remove my VPC settings
  • Create a NAT Gateway inside my VPC

I chose the one that costs the least money.

4
  • Did you enable VPC access for your Lambda function? Commented Sep 20, 2016 at 12:56
  • Unless I'm missing something, according to the documentation I believe I have everything correct. I updated my post with my IAM Policy. Commented Sep 20, 2016 at 23:54
  • By the way thanks for your comment! I also added my configuration. Commented Sep 21, 2016 at 0:05
  • IAM policy is for AWS API access, and is unrelated to the issue at hand. Commented Sep 21, 2016 at 0:36

1 Answer 1

3

You enabled VPC access for your Lambda function. So now it only has access to resources inside your VPC. Note that the AWS API exists outside your VPC. Any attempt to access something outside your VPC is going to result in a network timeout. That's why you are seeing the timeout issues.

To fix this, you can move the Lambda function outside the VPC, or you can add a NAT Gateway to your VPC. I'm not seeing anything in your code that is accessing anything inside your VPC, so it's probably cheapest and easiest to just remove the VPC settings from the Lambda function.

Sign up to request clarification or add additional context in comments.

1 Comment

hahaha, true, I think this is one of those Homer Simpson "D'oh!" moments... Worked like a charm... thanks!!!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.