aws-lambda-sqs-lambda - FIFO Support

[dextel2]

I was wondering if we have support for FIFO File processing, say.

I have first lambda function which is trigged if an object is dropped in a S3 bucket, which further on goes to processing in the second lambda function and meanwhile I upload another object in the same S3 bucket but this time the second object is completed in process and then the first object is processed.

So I was wondering if there is any way to process file sequentially i.e. FIFO.

[biffgaut]

I'm pretty sure there is no guarantee (nor even expectation) that S3 events will come out in any particular order. On this page of the docs it says it can take up to a minute for an event to fire. So if the events coming out of S3 are in an unreliable order there's no Lambda configuration that can overcome that. If your Use Case requires that objects be processed in the exact order they are stored, I'd suggest having the process that stores the object to S3 then put a message with a link to the object in a FIFO queue, and trigger a Lambda function off that queue.

[dextel2]

@biffgaut

I'd suggest having the process that stores the object to S3 then put a message with a link to the object in a FIFO queue, and trigger a Lambd

Could you please elaborate on this line?

[biffgaut]

My suggestion assumes that you control the code writing objects to the S3 bucket, so you could add a step that sends a message to a FIFO queue with a URL to the object you just wrote to the S3 bucket. Then your client code would ignore S3 events and trigger off events from the FIFO queue. Again, this is only possible if you are writing the objects to the S3 bucket.