[Question] - Use it inside a lambda function

[jledesma-opsguru]

Hi all.

I would like to know if I can import and use this tool from an AWS lambda function. If possible, could you give me a small example of how to do it?

Thanks

[jledesma-opsguru]

@CoaxVex @okummer any thoughts on this one?

[okummer]

I have not used s3-pit-restore in a lambda, but I don't see why you cannot get the basic algorithm to work in that context. I guess that you want to transfer data between S3 buckets, because file system access is unusual in lambdas and might have some limitations. I don't have an example ready, but I'll try to given some hints.

Your biggest problem is likely that the current code weaves command line parsing and execution, so that the actual algorithm repeatedly accesses the args object. I suspect that you will want to create a dummy args object, so that you do not have to change the the core algorithm. In general, the code might benefit from a clearer split between command line parsing and execution, e.g., using an executor class with parameters as fields, but that would require some refactoring.

Note that the execution time of a lambda is seriously limited and the script can take a long time for execute. With millions of objects to check of tens of thousand of object to restore you can easily exceed the maximum lambda timeout.

The imports include dateutil, which you have to bring yourself, if I am not mistaken. The other modules should be available.

[jledesma-opsguru]

Thanks for your comments @okummer.
I am experiencing varying behaviour between executing the code directly and using the s3-pit-restore tool installed by Brew, and I am unsure of the issue. However, when using the s3-pit-restore (brew installed), I am able to successfully restore files.

s3-pit-restore \
 -b my-test-bucket \
 -B my-test-bucket \
 -d folder1/subfolder1 \
 -p folder1/subfolder1 \
 -t "25-04-2023 23:59:50 +2"

When I execute the code with the same parameters, I do not receive any errors, but the restoring process does not occur.:

 python3 s3_pit_restore.py -b my-test-bucket \
 -B my-test-bucket \
 -d folder1/subfolder1 \
 -p folder1/subfolder1 \
 -t "25-04-2023 23:59:50 +2"

Any thoughts on why?
Thanks

[okummer]

Sorry, no. Maybe someone else can chime in.