Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Lambda function error
#1
First of all: I am not familiar with Python, so I am asking for help to fix the existing script. There is a Python script (Lambda function) for deleting files having time stamps from S3 bucket:
https://gist.github.com/pwilken/b548b589...9ebadae59f

However it is failing at:

Error:
[ERROR] IndexError: list index out of range Traceback (most recent call last): File "/var/task/lambda_function.py", line 34, in lambda_handler hour = dateArray[1].split(":")[0]

time stamp in S3 looks like: Jan 27, 2020 4:36:02 PM GMT+0000

so it is probably not expecting year here:

 # Start Policy for everything older then one week date = obj["Key"].replace('prefix_before_time', '') dateArray = date.split("T") date = dateArray[0] hour = dateArray[1].split(":")[0] print(f' {date} - {hour}')
Do anyone know how to fix it?
Reply
#2
(Jan-28-2020, 09:03 PM)localsystemuser Wrote: time stamp in S3 looks like: Jan 27, 2020 4:36:02 PM GMT+0000
Are you sure? That is not what the procedure is expecting. The date is split on the "T" sign. The part before that is considered a date. The part after is considered time.
dateArray = date.split("T") date = dateArray[0] hour = dateArray[1]
So I think the program expects a date in ISO 8601 format. Like this: "2020-01-28T20:32:25+00:00".
Reply
#3
(Jan-28-2020, 09:42 PM)ibreeden Wrote:
(Jan-28-2020, 09:03 PM)localsystemuser Wrote: time stamp in S3 looks like: Jan 27, 2020 4:36:02 PM GMT+0000
Are you sure? That is not what the procedure is expecting. The date is split on the "T" sign. The part before that is considered a date. The part after is considered time.
dateArray = date.split("T") date = dateArray[0] hour = dateArray[1]
So I think the program expects a date in ISO 8601 format. Like this: "2020-01-28T20:32:25+00:00".

So changing T to blank space should fix the problem? I am asking because my Python knowledge is 0.

Well,

In aws cli, timestamp looks differently:

aws s3 ls bucketname PRE folder/ 2019-02-27 10:24:00 271 file 2019-06-06 14:41:47 2353 file2

I have replaced T with blank space, but still getting error:

Error:
{ "errorMessage": "list index out of range", "errorType": "IndexError", "stackTrace": [ " File \"/var/task/lambda_function.py\", line 34, in lambda_handler\n hour = dateArray[1].split(\":\")[0]\n" ] }
Reply
#4
If this is the format: Jan 27, 2020 4:36:02 PM GMT+0000
This will be the format string for datetime: "%b %d, %Y %I:%M:%S %p %Z%z"
  • %b: Month as locale’s abbreviated name.
  • %d: Day of the month as a zero-padded decimal number.
  • %Y: Year with century as a decimal number.
  • %I: Hour (12-hour clock) as a zero-padded decimal number.
  • %M: Minute as a zero-padded decimal number.
  • %S: Second as a zero-padded decimal number.
  • %p: Locale’s equivalent of either AM or PM.
  • %Z: Time zone name (empty string if the object is naive).
  • %z: UTC offset in the form ±HHMM[SS[.ffffff]] (empty string if the object is naive).

The method datetime.strptime will convert the string into a datetime object. The format is documented.

If the format is iso8601, it's much easier, because there is a method datetime.datetime.fromisoformat which parses this format specification.

from datetime import datetime as dt my_format = "%b %d, %Y %I:%M:%S %p %Z%z" my_timestamp = "Jan 27, 2020 4:36:02 PM GMT+0000" # now use strptime, which is the abbreviation for "string put time", # which is used to create a datetime object from str # datetime.datetime.strptime(your_date, your_format) # strftime means "string format time" # which is used to convert datetime to a str my_date = dt.strptime(my_timestamp, my_format)
ISO8601 example:
from datetime import datetime as dt my_timestamp = "2020-01-29T14:12:53.439493+01:00" my_date = dt.fromisoformat(my_timestamp)
In both cases the offset from utc is known. The resulting datetime-object has the timezone offset included.

Paring the ISO8601 timestamp manually is also possible, but not required.

from datetime import datetime as dt iso8601_fmt = '%Y-%m-%dT%H:%M:%S%z' ts = "2020-01-28T20:32:25+10:00" my_date = dt.strptime(ts, iso8601_fmt)
Don't forget, that there are two different types datetime objects.
  • datetime with timezone information
  • naive datetime without timezone information
You can't do operations between a naive datetime and a datetime object.
For example you have a datetime object created, to compare it with your timestamps:

from datetime import datetime as dt not_before = dt(2020, 1, 10) # no timezone my_timestamp = dt.fromisoformat("2020-01-28T20:32:25+10:00") if my_timestamp < not_before: print(f'{my_timestamp} is before {not_before}') else: print(f'{my_timestamp} is not before {not_before}')
Error:
TypeError: can't compare offset-naive and offset-aware datetimes
To fix this problem with external dependencies:
from datetime import datetime as dt # pip install pendulum import pendulum my_timezone = pendulum.timezone('Europe/Berlin') not_before = my_timezone.datetime(2020, 1, 10) my_timestamp = dt.fromisoformat("2020-01-28T20:32:25+10:00") if my_timestamp < not_before: print(f'{my_timestamp} is before {not_before}') else: print(f'{my_timestamp} is not before {not_before}')
To handle timezone direct with datetime is a bit annoying.
Depending on you input, you can decide what do to.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Add two resultant fields from python lambda function klllmmm 4 2,529 Jun-06-2023, 05:11 PM
Last Post: rajeshgk
  Writing a lambda function that sorts dictionary GJG 1 3,087 Mar-09-2021, 06:44 PM
Last Post: buran
  Why does lambda throw 'name value_o is not defined' error? karabakh 3 4,410 Dec-14-2020, 05:45 PM
Last Post: karabakh
  translating lambda in function fabs 1 3,075 Apr-28-2020, 05:18 AM
Last Post: deanhystad
  Lambda function recursion error DeadlySocks 1 3,495 Apr-13-2020, 05:09 PM
Last Post: deanhystad
  Help with AWS Lambda function faws2019 0 2,310 Aug-27-2019, 01:54 PM
Last Post: faws2019
  Lambda function Uchikago 3 4,191 Jul-16-2019, 08:56 AM
Last Post: perfringo
  eval lambda function with restricted context Olivier 7 8,227 Mar-04-2019, 10:45 PM
Last Post: Olivier
  Using a lambda function within another function JChapman 8 8,318 Jan-08-2019, 01:54 PM
Last Post: JChapman
  Write lambda function in pyhhon to coy data from multiple JSON into a single JSON fil anandmn85 2 5,787 Apr-19-2018, 05:56 AM
Last Post: anandmn85

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020
This forum uses Lukasz Tkacz MyBB addons.
Forum use Krzysztof "Supryk" Supryczynski addons.