Reading and manipulating csv

[Prince_Bhatia]

hi,


I have a csv which has two columns and on the other hand i have multiple lists in my code.

I am looping through second column of each data in csv and checking if it exists in the already defined lists.

if it exists create a new csv and add them. But what has happened till now it has added data but it is separating them into different columns and i want it data of each category into one column but comma separated.

below is my code:

import csv from itertools import groupby,chain from operator import itemgetter Facility1 = ["club_house", "gymnasium", "children’s_play_area", "24/7_water_supply", "power_back-up", "inter-com","lift"] Facility2 = ["jogging_track", "swimming_pool", "aerobics_center", "badminton_court", "basketball_court", "beach_volleyball_court", "football_court", "golf_course", "lawn_tennis_court","yoga/meditation_center"] Facility3 = ["crèche", "ATM", "car_wash_area", "community_hall", "banquet_hall", "garbage_disposal", "grocery_shop","library"] Facility4 = ["fountains","landscape_gardens"] Facility5 = ["gated_society"] Facility6 = ["car_parking"] Facility7 = ["CCTV_cameras"] Facility8 = ["security_guards"] Facility9 = ["pool_table", "carrom_room", "chess_room", "dart_board", "squash_court", "table_tennis_room", "skating_rink"] def main(): prj_structure = {} f = open("./facility_data.csv","r",encoding="utf-8") data = f.read() f.close() lst = data.split("\n") prj = "" for i in range(1, len(lst)): val = lst[i].split(",") if len(val)>0: prj = val[0] if prj!="": if prj not in prj_structure.keys(): prj_structure[prj] = [] prj_structure[prj].append(val[1]) return prj_structure final_data = [] def readingfiles(): global final_data data = main() for k,v in data.items(): for i in v: sublist = [] sublist.append(k) if i in Facility1: sublist.append(i+"-facility1") if i in Facility2: sublist.append(i+"-facility2") if i in Facility3: sublist.append(i+"-facility3") if i in Facility4: sublist.append(i+"-facility4") if i in Facility5: sublist.append(i+"-facility5") if i in Facility6: sublist.append(i+"-facility6") if i in Facility7: sublist.append(i+"-facility7") if i in Facility8: sublist.append(i+"-facility8") if i in Facility9: sublist.append(i+"-facility9") final_data.append(sublist) return final_data def writefile(filename, alldata): global final_data with open(filename,"w", encoding="utf-8")as csvfile: csvfile = csv.writer(csvfile, delimiter=",") csvfile.writerow("") for i in range(0, len(alldata)): csvfile.writerow(alldata[i]) def get_new(): readingfiles() writefile("new_facilities.csv",[[k, *chain.from_iterable(r for _, *r in g)] for k, g in groupby(final_data, key=itemgetter(0))]) get_new()

attached is the sample csv that i am using to create a new csv which is by name facility_data.csv

and attached is the output csv how i want data to be which is by name "new_facilities.csv"

Please help!

[Larz60+]

The Facility_data.csv file has non standard encoding what is the codec?

[Prince_Bhatia]

Codec is UTF-8 only

[Larz60+]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 58: invalid start byte

[Prince_Bhatia]

i made a change in the code , please use that

[Larz60+]

I already tried that, the error I show is from modified code.
I've seen this before, are you on MS windows?
I believe the character in question is a '

[perfringo]

0x92 problem will 'go away' if instead of utf-8 to use either ISO-8859-1 or windows-1252 encoding.

[Larz60+]

perfringo's codec is correct.
this is much easier done by using a dictionary:
example:

import csv from itertools import groupby,chain from operator import itemgetter import os facilities = { 'Facility1': ["club_house", "gymnasium", "children’s_play_area", "24/7_water_supply", "power_back-up", "inter-com","lift"], 'Facility2': ["jogging_track", "swimming_pool", "aerobics_center", "badminton_court", "basketball_court", "beach_volleyball_court", "football_court", "golf_course", "lawn_tennis_court","yoga/meditation_center"], 'Facility3': ["crèche", "ATM", "car_wash_area", "community_hall", "banquet_hall", "garbage_disposal", "grocery_shop","library"], 'Facility4': ["fountains","landscape_gardens"], 'Facility5': ["gated_society"], 'Facility6': ["car_parking"], 'Facility7': ["CCTV_cameras"], 'Facility8': ["security_guards"], 'Facility9': ["pool_table", "carrom_room", "chess_room", "dart_board", "squash_court", "table_tennis_room", "skating_rink"] } def main(): # Make sure in src directory to start os.chdir(os.path.abspath(os.path.dirname(__file__))) with open("facility_data.csv", encoding="ISO-8859-1") as fp: crdr = csv.reader(fp, delimiter=',') for n, row in enumerate(crdr): for facid, values in facilities.items(): if row[1] in values: print(f'found: from row: {n}, {row[1]} ({row[0]}) in {facid}') if __name__ == '__main__': main()

partial output:

Output:
found: from row: 1, swimming_pool (R3) in Facility2 found: from row: 2, gymnasium (R3) in Facility1 found: from row: 6, swimming_pool (R5) in Facility2 found: from row: 7, gymnasium (R5) in Facility1 found: from row: 11, swimming_pool (R6) in Facility2 found: from row: 12, gymnasium (R6) in Facility1 found: from row: 16, swimming_pool (R9) in Facility2 found: from row: 17, gymnasium (R9) in Facility1 found: from row: 20, swimming_pool (R10) in Facility2 found: from row: 21, gymnasium (R10) in Facility1 found: from row: 22, swimming_pool (R703) in Facility2 found: from row: 23, gymnasium (R703) in Facility1 found: from row: 26, swimming_pool (R17) in Facility2 found: from row: 27, gymnasium (R17) in Facility1 found: from row: 30, swimming_pool (R18) in Facility2

Also (which I haven't done), write youor output incrementally as you read input

[Prince_Bhatia]

Yes i am using windows

[Larz60+]

Please respond to post 8. Do you understand the concept here?