How to handle timeouts in a python lambda?

Here's an example from cloudformation-custom-resources/lambda/python · GitHub showing how an AWS Lambda function written in Python can realise that it is about to timeout.

(I've edited out the other stuff, here's the relevant bits):

import signal

def handler(event, context):

    # Setup alarm for remaining runtime minus a second
    signal.alarm((context.get_remaining_time_in_millis() / 1000) - 1)

    # Do other stuff
    ...

def timeout_handler(_signal, _frame):
    '''Handle SIGALRM'''
    raise Exception('Time exceeded')

signal.signal(signal.SIGALRM, timeout_handler)

I want to update on @John Rotenstein answer which worked for me yet resulted in the following errors populating the cloudwatch logs:

START RequestId: ********* Version: $LATEST
Traceback (most recent call last):
    File "/var/runtime/bootstrap", line 9, in <module>
    main()
    File "/var/runtime/bootstrap.py", line 350, in main
    event_request = lambda_runtime_client.wait_next_invocation()
    File "/var/runtime/lambda_runtime_client.py", line 57, in wait_next_invocation
    response = self.runtime_connection.getresponse()
    File "/var/lang/lib/python3.7/http/client.py", line 1369, in getresponse
    response.begin()
    File "/var/lang/lib/python3.7/http/client.py", line 310, in begin
    version, status, reason = self._read_status()
    File "/var/lang/lib/python3.7/http/client.py", line 271, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    File "/var/lang/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
    File "/var/task/lambda_function.py", line 6, in timeout_handler
    raise Exception('Time limit exceeded')
Exception: Time limit exceeded
END RequestId

So I just had to reset the signals alarm before returning each response:

import logging
import signal


def timeout_handler(_signal, _frame):
    raise Exception('Time limit exceeded')


signal.signal(signal.SIGALRM, timeout_handler)


def lambda_handler(event, context):
    try:
        signal.alarm(int(context.get_remaining_time_in_millis() / 1000) - 1)

        logging.info('Testing stuff')
        # Do work

    except Exception as e:
        logging.error(f'Exception:\n{e}')

    signal.alarm(0)# This line fixed the issue above!
    return {'statusCode': 200, 'body': 'Complete'}

Two options I can think of, the first is quick and dirty, but also less ideal:

1. run it in a step function (check out step functions in AWS) which has the capability to retry on timeouts/errors

2. a better way would be to re-architect your code to be idempotent. In this example, the process that triggers the lambda checks a condition, and as long as this condition is true, trigger the lambda. That condition needs to remain true unless the lambda finished executing the logic successfully. This can be obtained by persisting the parameters sent to the lambda in a table in the DB, for example, and have an extra field called "processed" which will be modified to "true" only once the lambda finished running successfully for that event.

Using method #2 will make your code more resilient, easy to re-run on errors, and also easy to monitor: basically all you have to do is check how many such records do you have which are not processed, and what's their create/update timestamp on the DB.

If you care not only to identify the timeout, but to give your Lambdas an option of a "healthy" shutdown and pass the remaining payload to another execution automatically, you may have a look at the Siblings components of the sosw package.

Here is an example use-case where you call the sibling when the time is running out. You pass a pointer to where you have left the job to the Sibling. For example you may store the remaining payload in S3 and the cursor will show where you have stopped processing.

You will have to grant the Role of this Lambda permission to lambda:InvokeFunction on itself.

import logging
import time
from sosw import Processor as SoswProcessor
from sosw.app import LambdaGlobals, get_lambda_handler
from sosw.components.siblings import SiblingsManager


logger = logging.getLogger()
logger.setLevel(logging.INFO)


class Processor(SoswProcessor):

    DEFAULT_CONFIG = {
        'init_clients':    ['Siblings'],    # Automatically initialize Siblings Manager
        'shutdown_period': 10,  # Some time to shutdown in a healthy manner.
    }

    siblings_client: SiblingsManager = None


    def __call__(self, event):

        cursor = event.get('cursor', 0)

        while self.sufficient_execution_time_left:
            self.process_data(cursor)
            cursor += 1
            if cursor == 20:
                return f"Reached the end of data"

        else:
            # Spawning another sibling to continue the processing
            payload = {'cursor': cursor}

            self.siblings_client.spawn_sibling(global_vars.lambda_context, payload=payload, force=True)
            self.stats['siblings_spawned'] += 1


    def process_data(self, cursor):
        """ Your custom logic respecting current cursor. """
        logger.info(f"Processing data at cursor: {cursor}")
        time.sleep(1)


    @property
    def sufficient_execution_time_left(self) -> bool:
        """ Return if there is a sufficient execution time for processing ('shutdown period' is in seconds). """
        return global_vars.lambda_context.get_remaining_time_in_millis() > self.config['shutdown_period'] * 1000


global_vars = LambdaGlobals()
lambda_handler = get_lambda_handler(Processor, global_vars)