TypeError: a bytes-like object is required, not 'str' - Help Please.

[IanJ]

Hi,

I have inherited a number of python scripts for AWS Lambda that need upgrading from python 2.7 to v3. The following script has been through the 2to3 python script. However I'm getting ' TypeError: a bytes-like object is required, not 'str' '.

Sorry but I'm new to python and no idea how I should fix this.

Thanks in advance.

This is my code -

import os import json import urllib.request, urllib.parse, urllib.error import boto3 import botocore import datetime import json import csv print('Loading function') region_list = {'eu-west-1','eu-west-2','eu-central-1'} date = datetime.date.today().strftime("%Y_%m_%d") #file path filepath ='/tmp/' #filename filename = os.environ['filename'] + date + '.csv' tempfile = filepath + filename mybucket = os.environ['bucket'] account = os.environ['account'] #print ('filename "%s"' % str(filename)) session = boto3.session.Session() s3_client = session.client('s3', config= boto3.session.Config(signature_version='s3v4')) def lambda_handler(event, context): with open(tempfile, 'wb+') as out: dict_writer = csv.DictWriter(out, fieldnames=['Region','InstanceId','Name','Status','Instance_Size','Launch_Time','Service','CPU.Average','CPU.Max','CPUCreditBalance.Max','CPUCreditBalance.Min']) dict_writer.writeheader() for region in region_list: client = session.client('ec2', region_name=region) cwclient = session.client('cloudwatch', region_name=region) #filters=[{'Name': 'instance-state-name', 'Values': ['running']}] #response = client.describe_instances(Filters=filters) response = client.describe_instances() instance_list = [] for instances in response['Reservations']: for instance in instances['Instances']: inst = {} inst['Region'] = region inst['InstanceId'] = instance['InstanceId'] inst['Name'] = instance['InstanceId'] inst['Instance_Size'] = instance['InstanceType'] state = instance['State'] inst['Status'] = state['Name'] launch = instance['LaunchTime'] inst['Launch_Time'] = launch.strftime("%Y_%m_%d %H:%M") inst['Service'] = '' inst['Name'] = '' inst['CPU.Average'] = '' inst['CPU.Max'] = '' try: if len(instance['Tags']) > 0: for tag in instance['Tags']: if tag['Key'] == 'Service': inst['Service'] = tag['Value'] if tag['Key'] == 'Name': inst['Name'] = tag['Value'] except KeyError as e: print(('I got a KeyError - reason "%s"' % str(e))) cwresponse=cwclient.get_metric_statistics( Period = 900, StartTime = datetime.datetime.utcnow() - datetime.timedelta(days=4), EndTime = datetime.datetime.utcnow(), Namespace = 'AWS/EC2', MetricName = 'CPUUtilization', Statistics = ['Average','Maximum'], Dimensions = [{'Name' : 'InstanceId', 'Value' : instance['InstanceId']}] ) average = 0 maxcpu = 0 if cwresponse['Datapoints']: for data in cwresponse['Datapoints']: cpu = data['Maximum'] ave = data['Average'] if cpu > maxcpu: maxcpu = cpu if ave > average: average = ave inst['CPU.Max'] = maxcpu inst['CPU.Average'] = average cwresponse2=cwclient.get_metric_statistics( Period = 900, StartTime = datetime.datetime.utcnow() - datetime.timedelta(days=4), EndTime = datetime.datetime.utcnow(), Namespace = 'AWS/EC2', MetricName = 'CPUCreditBalance', Statistics = ['Minimum','Maximum'], Dimensions = [{'Name' : 'InstanceId', 'Value' : instance['InstanceId']}] ) minbal = 999999 maxbal = 0 if cwresponse2['Datapoints']: for data in cwresponse2['Datapoints']: minimum = data['Minimum'] maximum = data['Maximum'] if minimum < minbal: minbal = minimum if maximum > maxbal: maxbal = maximum inst['CPUCreditBalance.Max'] = maxbal inst['CPUCreditBalance.Min'] = minbal dict_writer.writerow(inst) instance_list.append(inst) out.close() # Upload the file to S3 s3_client.upload_file(tempfile, mybucket, account + filename, ExtraArgs={'ACL': 'public-read'})

Error:
[ERROR] TypeError: a bytes-like object is required, not 'str' Traceback (most recent call last): File "/var/task/lambda_function.py", line 35, in lambda_handler dict_writer.writeheader() File "/var/lang/lib/python3.8/csv.py", line 143, in writeheader return self.writerow(header) File "/var/lang/lib/python3.8/csv.py", line 154, in writerow return self.writer.writerow(self._dict_to_list(rowdict))

[Larz60+]

The error occurs in writerow, probably caused by line 35,
but you don't show writerow, so hard to say

[malcomjarr]

Python makes a clear distinction between bytes and strings . Bytes objects contain raw data — a sequence of octets — whereas strings are Unicode sequences . Conversion between these two types is explicit: you encode a string to get bytes, specifying an encoding (which defaults to UTF-8); and you decode bytes to get a string. Clients of these functions should be aware that such conversions may fail, and should consider how failures are handled.

We can convert bytes to string using bytes class decode() instance method, So you need to decode the bytes object to produce a string. In Python 3 , the default encoding is "utf-8" , so you can use directly:

b"python byte to string".decode("utf-8")

[deanhystad]

The problem is that you open the file as binary

with open(tempfile, 'wb+') as out:

And then you try to write str objects which use unicode characters, not bytes.

You can either convert all the str to bytes, or not open the file as binary. Since this is a csv file I don't know why you would want it to be binary.