Posts: 49 Threads: 19 Joined: Jul 2020 Jun-28-2021, 09:47 AM (This post was last modified: Jun-29-2021, 09:18 AM by Laplace12.) Hey! I have a text file that I want to sort out. I've coded this and tried dataframe, but that only prints the last line. The code I have now is this, producing the txt file: with open(output) as file, open(out, 'w') as file_out: for line in file: if '2101' in line and found: a = line.split() print(a[1], file=file_out) elif 'Lifetimes' in line and found: b = line.split() print(b[3], b[4], b[5], file=file_out) elif 'Std deviations' in line and found: # print(c[3:6]) c = line print(deviations(c), file=file_out) elif 'Intensities' in line and found: d = line.split() print(d[3], d[4], d[5], file=file_out) elif 'Time-zero' in line and found: e = line.split() print(e[4], file=file_out) else: found = True #This is what I tried so far with open(out) as a: cpt = 0 for line in a: cpt += 1 if cpt == 8: print(line) cpt = 0 The 'out' file is like this: Number Value1 Deviation Value2 Deviation Value3 Deviation Number Value1 Deviation ... So basically the file is now a list I want to sort so that Lifetimes are all in one column, Value1 in the next, then Deviation, Value2, its deviation so on; I want every 8th value in the same column, and I'm guessing this could somehow be done by creating a loop that prints, skips 7 values and prints the next so that the start number could be changed from 1-7. I need to save the results in another file, so perhaps it'd be easier to code the columns in the 'out' file already without creating so many files, but for now it's enough to get the data sorted properly, so even the simplest code to produce columns from the txt file works! Posts: 7,398 Threads: 123 Joined: Sep 2016 Jun-28-2021, 10:54 AM (This post was last modified: Jun-28-2021, 10:54 AM by snippsat.) How dos the original file look it's properly a better way,but can not advice anything without a sample of original file. Posts: 49 Threads: 19 Joined: Jul 2020 (Jun-28-2021, 10:54 AM)snippsat Wrote: How dos the original file look it's properly a better way,but can not advice anything without a sample of original file. Hey, it looks like this: #0 0.4000 0.1250 2.0446 ['Fixed', 'Fixed', '0.0339'] 69.2721 9.6726 21.0553 ['1.0359', '0.8128', '0.4063'] 41.5603 ['0.0588', ' ', ' '] #1 0.4000 0.1250 2.0714 ['Fixed', 'Fixed', '0.0344'] 70.0338 9.0952 20.8710 ['1.0308', '0.8135', '0.4009'] 41.5853 ['0.0593', ' ', ' '] #2 0.4000 0.1250 2.0568 ['Fixed', 'Fixed', '0.0333'] 69.5963 8.7445 21.6592 ['1.0411', '0.8177', '0.4072'] 41.5541 ['0.0603', ' ', ' '] #3 0.4000 0.1250 2.0321 ['Fixed', 'Fixed', '0.0329'] ... With 490 lines in total. Posts: 7,398 Threads: 123 Joined: Sep 2016 Jun-28-2021, 01:20 PM (This post was last modified: Jun-28-2021, 01:20 PM by snippsat.) What is the output you want from this? I can not see why you look for Lifetime,Std deviations...ect in this. Are you making this file? When put a Python list ['Fixed', 'Fixed', '0.0339'] in a text file it lose all it's meaning. Have to parse it back or could done something else like taken out values(eg CSV way) then save a list to a test file. If you no control of the text file then have to parse it to what you want. Posts: 49 Threads: 19 Joined: Jul 2020 Jun-28-2021, 06:47 PM (This post was last modified: Jun-28-2021, 06:47 PM by Laplace12.) (Jun-28-2021, 01:20 PM)snippsat Wrote: What is the output you want from this?
I can not see why you look for Lifetime,Std deviations...ect in this.
Are you making this file?
When put a Python list ['Fixed', 'Fixed', '0.0339'] in a text file it lose all it's meaning.
Have to parse it back or could done something else like taken out values(eg CSV way) then save a list to a test file.
If you no control of the text file then have to parse it to what you want. Alright, I must've explained this quite badly, let me try again! The first part of the code (picking Lifetimes etc.) is just sorting out a file (called output) that looks like this: CA50_40_ref_data2101_E04_spec0-70 #0 Lifetimes (ns) : 0.4000 0.1250 2.0446 Std deviations : Fixed Fixed 0.0339 Intensities (%) : 69.2721 9.6726 21.0553 Std deviations : 1.0359 0.8128 0.4063 Time-zero Channel number : 41.5603 Std deviations : 0.0588 CA50_40_ref_data2101_E04_spec0-70 #1 Lifetimes (ns) : 0.4000 0.1250 2.0714 Std deviations : Fixed Fixed 0.0344 Intensities (%) : 70.0338 9.0952 20.8710 Std deviations : 1.0308 0.8135 0.4009 Time-zero Channel number : 41.5853 Std deviations : 0.0593 CA50_40_ref_data2101_E04_spec0-70 #2 Lifetimes (ns) : 0.4000 0.1250 2.0568 Std deviations : Fixed Fixed 0.0333 Intensities (%) : 69.5963 8.7445 21.6592 Std deviations : 1.0411 0.8177 0.4072 Time-zero Channel number : 41.5541 Std deviations : 0.0603 CA50_40_ref_data2101_E04_spec0-70 #3 Lifetimes (ns) : 0.4000 0.1250 2.0321 Std deviations : Fixed Fixed 0.0329 Intensities (%) : 70.4228 8.0614 21.5158 Std deviations : 1.0497 0.8219 0.4105 Time-zero Channel number : 41.4507 Std deviations : 0.0604 CA50_40_ref_data2101_E04_spec0-70 #4 Lifetimes (ns) : 0.4000 0.1250 2.0513 Std deviations : Fixed Fixed 0.0331 Intensities (%) : 67.2025 11.0731 21.7244 Std deviations : 1.0204 0.7976 0.4057 Time-zero Channel number : 41.6253 Std deviations : 0.0579 CA50_40_ref_data2101_E04_spec0-70 #5 ... into this (file called out): #0 0.4000 0.1250 2.0446 ['Fixed', 'Fixed', '0.0339'] 69.2721 9.6726 21.0553 ['1.0359', '0.8128', '0.4063'] 41.5603 ['0.0588', ' ', ' '] #1 0.4000 0.1250 2.0714 ['Fixed', 'Fixed', '0.0344'] 70.0338 9.0952 20.8710 ['1.0308', '0.8135', '0.4009'] 41.5853 ['0.0593', ' ', ' '] #2 0.4000 0.1250 2.0568 ['Fixed', 'Fixed', '0.0333'] 69.5963 8.7445 21.6592 ['1.0411', '0.8177', '0.4072'] 41.5541 ['0.0603', ' ', ' '] #3 0.4000 0.1250 2.0321 ['Fixed', 'Fixed', '0.0329'] 70.4228 8.0614 21.5158 ['1.0497', '0.8219', '0.4105'] 41.4507 ['0.0604', ' ', ' '] #4 0.4000 0.1250 2.0513 ['Fixed', 'Fixed', '0.0331'] 67.2025 11.0731 21.7244 ['1.0204', '0.7976', '0.4057'] So the first loop was needed to extract the necessary information from the first file, and now I am trying to get the 'out' file above in this form for easier comparison: Dataset Lifetimes Std deviations Intensities Std deviations Time-zero Std deviation #0 0.4000 0.1250 2.0446 ['Fixed', 'Fixed', '0.0339'] 69.2721 9.6726 21.0553 ['1.0359', '0.8128', '0.4063'] 41.5603 ['0.0588', ' ', ' '] #1 0.4000 0.1250 2.0714 ['Fixed', 'Fixed', '0.0344'] 70.0338 9.0952 20.8710 ['1.0308', '0.8135', '0.4009'] 41.5853 ['0.0593', ' ', ' '] ... So basically I'm just trying to sort out the 'out' file into columns with every eight value in the same column. Posts: 2,167 Threads: 35 Joined: Sep 2016 Jun-28-2021, 11:20 PM (This post was last modified: Jun-28-2021, 11:32 PM by Yoriz.) output Output: CA50_40_ref_data2101_E04_spec0-70 #0 Lifetimes (ns) : 0.4000 0.1250 2.0446 Std deviations : Fixed Fixed 0.0339 Intensities (%) : 69.2721 9.6726 21.0553 Std deviations : 1.0359 0.8128 0.4063 Time-zero Channel number : 41.5603 Std deviations : 0.0588 CA50_40_ref_data2101_E04_spec0-70 #1 Lifetimes (ns) : 0.4000 0.1250 2.0714 Std deviations : Fixed Fixed 0.0344 Intensities (%) : 70.0338 9.0952 20.8710 Std deviations : 1.0308 0.8135 0.4009 Time-zero Channel number : 41.5853 Std deviations : 0.0593 CA50_40_ref_data2101_E04_spec0-70 #2 Lifetimes (ns) : 0.4000 0.1250 2.0568 Std deviations : Fixed Fixed 0.0333 Intensities (%) : 69.5963 8.7445 21.6592 Std deviations : 1.0411 0.8177 0.4072 Time-zero Channel number : 41.5541 Std deviations : 0.0603 CA50_40_ref_data2101_E04_spec0-70 #3 Lifetimes (ns) : 0.4000 0.1250 2.0321 Std deviations : Fixed Fixed 0.0329 Intensities (%) : 70.4228 8.0614 21.5158 Std deviations : 1.0497 0.8219 0.4105 Time-zero Channel number : 41.4507 Std deviations : 0.0604 CA50_40_ref_data2101_E04_spec0-70 #4 Lifetimes (ns) : 0.4000 0.1250 2.0513 Std deviations : Fixed Fixed 0.0331 Intensities (%) : 67.2025 11.0731 21.7244 Std deviations : 1.0204 0.7976 0.4057 Time-zero Channel number : 41.6253 Std deviations : 0.0579
You may need to add some error checking from itertools import zip_longest HEADER = ('Dataset Lifetimes Std deviations ' 'Intensities Std deviations ' 'Time-zero Std deviation\n') def grouper(iterable, n, fillvalue=''): "Collect data into fixed-length chunks or blocks" # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx" args = [iter(iterable)] * n return zip_longest(*args, fillvalue=fillvalue) class Dataset: def __init__(self, line: str) -> None: self.data = line.strip().split()[1] def __repr__(self) -> str: return f'{self.data:7}' class LifeTimes: def __init__(self, line: str) -> None: self.data = line.strip().split()[3:6] def __repr__(self) -> str: return ' '.join(self.data) class StdDeviations: def __init__(self, line: str) -> None: split_line = line.strip().split() self.data = [split_line[num] if split_line[num:] else '' for num in range(3, 6)] def __repr__(self) -> str: return f"['{self.data[0]:^7}', '{self.data[1]:^7}', '{self.data[2]:^7}']" class Intensities: def __init__(self, line: str) -> None: self.data = line.strip().split()[3:6] def __repr__(self) -> str: return f'{self.data[0]:>7} {self.data[1]:>7} {self.data[2]:>7}' class TimeZero: def __init__(self, line: str) -> None: self.data = line.strip().split()[4] def __repr__(self) -> str: return f'{self.data:9}' class Block: def __init__(self, dataset, lifetimes, std_deviations, intensities, std_deviations2, time_zero, std_deviations3) -> None: self.dataset = Dataset(dataset) self.lifetimes = LifeTimes(lifetimes) self.std_deviations = StdDeviations(std_deviations) self.intensities = Intensities(intensities) self.std_deviations2 = StdDeviations(std_deviations2) self.time_zero = TimeZero(time_zero) self.std_deviations3 = StdDeviations(std_deviations3) def __repr__(self) -> str: return (f'{self.dataset} {self.lifetimes} {self.std_deviations}' f' {self.intensities} {self.std_deviations2}' f' {self.time_zero} {self.std_deviations3}\n') with open('output') as file_in, open('out', 'w') as file_out: file_out.write(HEADER) for group in grouper(file_in, 7): file_out.write(str(Block(*group)))out Output: Dataset Lifetimes Std deviations Intensities Std deviations Time-zero Std deviation #0 0.4000 0.1250 2.0446 [' Fixed ', ' Fixed ', '0.0339 '] 69.2721 9.6726 21.0553 ['1.0359 ', '0.8128 ', '0.4063 '] 41.5603 ['0.0588 ', ' ', ' '] #1 0.4000 0.1250 2.0714 [' Fixed ', ' Fixed ', '0.0344 '] 70.0338 9.0952 20.8710 ['1.0308 ', '0.8135 ', '0.4009 '] 41.5853 ['0.0593 ', ' ', ' '] #2 0.4000 0.1250 2.0568 [' Fixed ', ' Fixed ', '0.0333 '] 69.5963 8.7445 21.6592 ['1.0411 ', '0.8177 ', '0.4072 '] 41.5541 ['0.0603 ', ' ', ' '] #3 0.4000 0.1250 2.0321 [' Fixed ', ' Fixed ', '0.0329 '] 70.4228 8.0614 21.5158 ['1.0497 ', '0.8219 ', '0.4105 '] 41.4507 ['0.0604 ', ' ', ' '] #4 0.4000 0.1250 2.0513 [' Fixed ', ' Fixed ', '0.0331 '] 67.2025 11.0731 21.7244 ['1.0204 ', '0.7976 ', '0.4057 '] 41.6253 ['0.0579 ', ' ', ' ']
Laplace12 likes this post Posts: 7,398 Threads: 123 Joined: Sep 2016 If look at data so should it be turn around this is called transpose() if want data into Pandas for calculation, plot..ect. If just want display data then can Yoriz method work. To give example,just using first record. record = {} with open('ca_data.txt') as f: header = next(f) for line in f: line = line.strip() line = line.replace('Time-zero ', '') line = line.split(':') line_1 = line[0].strip() line_2 = ''.join(line[1:]) record[line_1] = line_2.split() # Read like this so it fill in empty values with None df = pd.DataFrame.from_dict(record, orient='index') print(df) Output: Lifetimes (ns) 0.4000 0.1250 2.0446 Std deviations 0.0588 None None Intensities (%) 69.2721 9.6726 21.0553 Channel number 41.5603 None None
No can use transpose(),then it will a useful DataFrame. >>> df = df.transpose() >>> df Lifetimes (ns) Std deviations Intensities (%) Channel number 0 0.4000 0.0588 69.2721 41.5603 1 0.1250 None 9.6726 None 2 2.0446 None 21.0553 None Laplace12 likes this post Posts: 49 Threads: 19 Joined: Jul 2020 (Jun-28-2021, 11:20 PM)Yoriz Wrote: output
Output: CA50_40_ref_data2101_E04_spec0-70 #0 Lifetimes (ns) : 0.4000 0.1250 2.0446 Std deviations : Fixed Fixed 0.0339 Intensities (%) : 69.2721 9.6726 21.0553 Std deviations : 1.0359 0.8128 0.4063 Time-zero Channel number : 41.5603 Std deviations : 0.0588 CA50_40_ref_data2101_E04_spec0-70 #1 Lifetimes (ns) : 0.4000 0.1250 2.0714 Std deviations : Fixed Fixed 0.0344 Intensities (%) : 70.0338 9.0952 20.8710 Std deviations : 1.0308 0.8135 0.4009 Time-zero Channel number : 41.5853 Std deviations : 0.0593 CA50_40_ref_data2101_E04_spec0-70 #2 Lifetimes (ns) : 0.4000 0.1250 2.0568 Std deviations : Fixed Fixed 0.0333 Intensities (%) : 69.5963 8.7445 21.6592 Std deviations : 1.0411 0.8177 0.4072 Time-zero Channel number : 41.5541 Std deviations : 0.0603 CA50_40_ref_data2101_E04_spec0-70 #3 Lifetimes (ns) : 0.4000 0.1250 2.0321 Std deviations : Fixed Fixed 0.0329 Intensities (%) : 70.4228 8.0614 21.5158 Std deviations : 1.0497 0.8219 0.4105 Time-zero Channel number : 41.4507 Std deviations : 0.0604 CA50_40_ref_data2101_E04_spec0-70 #4 Lifetimes (ns) : 0.4000 0.1250 2.0513 Std deviations : Fixed Fixed 0.0331 Intensities (%) : 67.2025 11.0731 21.7244 Std deviations : 1.0204 0.7976 0.4057 Time-zero Channel number : 41.6253 Std deviations : 0.0579
You may need to add some error checking
from itertools import zip_longest HEADER = ('Dataset Lifetimes Std deviations ' 'Intensities Std deviations ' 'Time-zero Std deviation\n') def grouper(iterable, n, fillvalue=''): "Collect data into fixed-length chunks or blocks" # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx" args = [iter(iterable)] * n return zip_longest(*args, fillvalue=fillvalue) class Dataset: def __init__(self, line: str) -> None: self.data = line.strip().split()[1] def __repr__(self) -> str: return f'{self.data:7}' class LifeTimes: def __init__(self, line: str) -> None: self.data = line.strip().split()[3:6] def __repr__(self) -> str: return ' '.join(self.data) class StdDeviations: def __init__(self, line: str) -> None: split_line = line.strip().split() self.data = [split_line[num] if split_line[num:] else '' for num in range(3, 6)] def __repr__(self) -> str: return f"['{self.data[0]:^7}', '{self.data[1]:^7}', '{self.data[2]:^7}']" class Intensities: def __init__(self, line: str) -> None: self.data = line.strip().split()[3:6] def __repr__(self) -> str: return f'{self.data[0]:>7} {self.data[1]:>7} {self.data[2]:>7}' class TimeZero: def __init__(self, line: str) -> None: self.data = line.strip().split()[4] def __repr__(self) -> str: return f'{self.data:9}' class Block: def __init__(self, dataset, lifetimes, std_deviations, intensities, std_deviations2, time_zero, std_deviations3) -> None: self.dataset = Dataset(dataset) self.lifetimes = LifeTimes(lifetimes) self.std_deviations = StdDeviations(std_deviations) self.intensities = Intensities(intensities) self.std_deviations2 = StdDeviations(std_deviations2) self.time_zero = TimeZero(time_zero) self.std_deviations3 = StdDeviations(std_deviations3) def __repr__(self) -> str: return (f'{self.dataset} {self.lifetimes} {self.std_deviations}' f' {self.intensities} {self.std_deviations2}' f' {self.time_zero} {self.std_deviations3}\n') with open('output') as file_in, open('out', 'w') as file_out: file_out.write(HEADER) for group in grouper(file_in, 7): file_out.write(str(Block(*group)))out
Output: Dataset Lifetimes Std deviations Intensities Std deviations Time-zero Std deviation #0 0.4000 0.1250 2.0446 [' Fixed ', ' Fixed ', '0.0339 '] 69.2721 9.6726 21.0553 ['1.0359 ', '0.8128 ', '0.4063 '] 41.5603 ['0.0588 ', ' ', ' '] #1 0.4000 0.1250 2.0714 [' Fixed ', ' Fixed ', '0.0344 '] 70.0338 9.0952 20.8710 ['1.0308 ', '0.8135 ', '0.4009 '] 41.5853 ['0.0593 ', ' ', ' '] #2 0.4000 0.1250 2.0568 [' Fixed ', ' Fixed ', '0.0333 '] 69.5963 8.7445 21.6592 ['1.0411 ', '0.8177 ', '0.4072 '] 41.5541 ['0.0603 ', ' ', ' '] #3 0.4000 0.1250 2.0321 [' Fixed ', ' Fixed ', '0.0329 '] 70.4228 8.0614 21.5158 ['1.0497 ', '0.8219 ', '0.4105 '] 41.4507 ['0.0604 ', ' ', ' '] #4 0.4000 0.1250 2.0513 [' Fixed ', ' Fixed ', '0.0331 '] 67.2025 11.0731 21.7244 ['1.0204 ', '0.7976 ', '0.4057 '] 41.6253 ['0.0579 ', ' ', ' ']
Brilliant, this works perfectly! Using def/class commands is not familiar to me at all but I have to take a much closer look at that, seems very useful for my purposes - and it's a huge plus having reduced the amount of files the code creates. Definitely something to try and learn better. Thank you! (Jun-29-2021, 12:39 AM)snippsat Wrote: If look at data so should it be turn around this is called transpose() if want data into Pandas for calculation, plot..ect.
If just want display data then can Yoriz method work.
To give example,just using first record.
record = {} with open('ca_data.txt') as f: header = next(f) for line in f: line = line.strip() line = line.replace('Time-zero ', '') line = line.split(':') line_1 = line[0].strip() line_2 = ''.join(line[1:]) record[line_1] = line_2.split() # Read like this so it fill in empty values with None df = pd.DataFrame.from_dict(record, orient='index') print(df) Output: Lifetimes (ns) 0.4000 0.1250 2.0446 Std deviations 0.0588 None None Intensities (%) 69.2721 9.6726 21.0553 Channel number 41.5603 None None
No can use transpose(),then it will a useful DataFrame.
>>> df = df.transpose() >>> df Lifetimes (ns) Std deviations Intensities (%) Channel number 0 0.4000 0.0588 69.2721 41.5603 1 0.1250 None 9.6726 None 2 2.0446 None 21.0553 None Big thanks to you as well, I'll take a look at this method! |
