Getting the response results from an asynchronous call to AWS lambda in python

Question

I have an AWS lambda function which I can call synchronously and get results back alright with below code

response = lambda_client.invoke(
        FunctionName=FUNCTION_NAME,
        InvocationType='RequestResponse',
        LogType='Tail',
        Payload=payload,
        Qualifier=$LATEST
    )

The response Payload is of type <botocore.response.StreamingBody object at 0x115fb3160> So I use below code to extract the payload which works fine.

response_body = response['Payload']
response_str = response_body.read().decode('utf-8')
response_dict = eval(response_str)

Now, I need to call my lambda asynchronously, so I change the invocation type with InvocationType='Event'

It gives me a response with payload of the same type as before, botocore.response.StreamingBody object but I am getting error with this line - response_dict = eval(response_str)

The error message says

    response_dict = eval(response_str)
  File "<string>", line 0

    ^
SyntaxError: unexpected EOF while parsing

What am I missing? If the response payload is same type as synchronous call, why is this parsing error? Any suggestion?

EDIT

For clarity, I understand that if the InvocationType='Event', then we only get the status of the invoke call, not the lambda function result. In my case though, I need both - launch the lambda async and get the result back when done. How do I do that? Is writing the result back to s3 and periodically checking that the only option?

I've answered why this doesn't work as you expected, below, but what I couldn't anticipate was what you were actually trying to accomplish by making the invocation asynchronous, since you still wanted the result returned. Please advise, and perhaps I or someone else can help you solve the original issue, if one remains. — Michael - sqlbot
– Michael - sqlbot, Commented Oct 1, 2019 at 10:53
There's no architected way to do what you want in Lambda. You need to design a solution. One option would be for the client invoking the initial Lambda to send a UUID along with its request, then the async Lambda does its work and writes the results somewhere (DynamoDB or S3, for example, using the client-supplied UUID as a key), and the client can then poll a second Lambda passing that same UUID in order to retrieve the results when they become available. Or the original Lambda sends a notification via SNS and your client polls SNS. — jarmod
– jarmod, Commented Oct 1, 2019 at 21:38
Currently it can be achieved using AWS Step functions: aws.amazon.com/step-functions — pafede2
– pafede2, Commented Feb 26, 2024 at 15:41

Michael - sqlbot · Accepted Answer · 2019-10-01 00:39:25Z

8

InvocationType='Event' means you aren't getting a response. An asynchronous Lambda invocation means you just want to invoke the function, not wait for the response. The response payload from the function is discarded by the service.

When you invoke a function asynchronously, Lambda sends the event to a queue. A separate process reads events from the queue and runs your function. When the event is added to the queue, Lambda returns a success response without additional information. (emphasis added)

https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html

Note that the queue mentioned here is a queue inside the Lambda service, not to be confused with Amazon Simple Queue Service (SQS).

answered Oct 1, 2019 at 0:39

Michael - sqlbot

180k28 gold badges388 silver badges458 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Matthew Pope Over a year ago

In addition, if you do need the results from an asynchronous Lambda execution, you should consider having the Lambda function write its results to S3 (or something to that effect) at a known location where clients of the Lambda function can periodically check for the existence of the result.

Maile Cupo Over a year ago

@MatthewPope It seems like you would need a separate s3 object for each invocation of the worker lambda right? Ideally I could track the state of all the asynchronous invocations in a single s3 file, but I'm struggling to see how they could all access the same file without overwriting one another's results. Any ideas?

Matthew Pope Over a year ago

@MaileCupo, my point is that if you need the results of an asynchronous invocation, you can write the results to some known location and the requestor can look for them there. I don’t have any particular suggestions for how you would implement your use case. You should consider asking a new question on SO for it.

Collectives™ on Stack Overflow

Getting the response results from an asynchronous call to AWS lambda in python

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related