python3 regex support

[anna]

Hi All,

lines in files are as below

"11-03-2019_18:43:14,User-name=4841605613,NAS-IP-Address=10.124.113.246,Framed-IP-Address=,Calling-Station-Id=14:a7:2b:03:5f:23,Reply-Mesasge=Authentication Failed due to Invalid Password,NAS-Port-Id=slot=3;subslot=2;port=100;vlanid=2160;vlanid2=1284;"

I want to find

11-03-2019 , 18:43:14 , 4841605613 , 10.124.113.246, 14:a7:2b:03:5f:23 , Authentication Failed due to Invalid Password

My regular expression is as below

r = re.compile(r"^(\d{2}\D\d{2}\D\d{4})\D(\d{2}\D\d{2}\D\d{2})\D\w+\D\w+\D([0-9]{10})\D\w+\D\w+\D\w+\D(\d+\.\d+\.\d+\.\d+)\D\w+\D\w+\D\w+\D*([0-9a-f]{2}:[0-9a-f]{2}:[0-9a-f]{2}:[0-9a-f]{2}:[0-9a-f]{2}:[0-9a-f]{2})\D\w+\D\w+\D(\w*\s\w.*)",re.MULTILINE)

Output is

Output:
('11-03-2019', '18:43:14', '4841605613', '10.124.113.246', '14:a7:2b:03:5f:23', 'Authentication Failed due to Invalid Password,NAS-Port-Id=slot=3;subslot=2;port=100;vlanid=2160;vlanid2=1284;')

however unwanted lines are printing.

like :- ,NAS-Port-Id=slot=3;subslot=2;port=100;vlanid=2160;vlanid2=1284;')[/output]

[perfringo]

If you want use this data later on, maybe it is useful to read into list of dictionaries for easy analysing?

Processing every row something like this:

>>> s = "11-03-2019_18:43:14,User-name=4841605613,NAS-IP-Address=10.124.113.246,Framed-IP-Address=,Calling-Station-Id=14:a7:2b:03:5f:23,Reply-Mesasge=Authentication Failed due to Invalid Password,NAS-Port-Id=slot=3;subslot=2;port=100;vlanid=2160;vlanid2=1284;" >>> d = dict() >>> for i, field in enumerate(s.split(',')): ... if i == 0: ... d['date'], d['time'] = field.split('_') ... else: ... try: ... name, value = field.split('=') ... if name in ['User-name', 'NAS-IP-Address', 'Calling-Station-Id', 'Reply-Mesasge']: ... d.update({name: value}) ... except ValueError: ... continue ... >>> d {'date': '11-03-2019', 'time': '18:43:14', 'User-name': '4841605613', 'NAS-IP-Address': '10.124.113.246', 'Calling-Station-Id': '14:a7:2b:03:5f:23', 'Reply-Mesasge': 'Authentication Failed due to Invalid Password'}

[DeaD_EyE]

Don't use reqex for this task.

A solution without using regex and relying that the output format does not change.

def parse(line): date, user, nas_ip, frame_ip, station, reply, port = line.split(',') date = datetime.datetime.strptime(date, '%d-%m-%Y_%H:%M:%S') user = user.replace('User-name=', '') nas_ip = nas_ip.replace('NAS-IP-Address=', '') frame_ip = frame_ip.replace('Framed-IP-Address=', '') station = station.replace('Calling-Station-Id=', '') reply = reply.replace('Reply-Mesasge=', '') port = port.replace('NAS-Port-Id=', '').rstrip(';') port_attr = {} for item in port.split(';'): key, value = item.split('=') port_attr[key] = int(value) return { 'date': date, 'user': user, 'nas_ip': nas_ip, 'frame_ip': frame_ip, 'station': station, 'reply': reply, 'port': port_attr, }

Output:
{'date': datetime.datetime(2019, 3, 11, 18, 43, 14), 'user': '4841605613', 'nas_ip': '10.124.113.246', 'frame_ip': '', 'station': '14:a7:2b:03:5f:23', 'reply': 'Authentication Failed due to Invalid Password', 'port': {'slot': 3, 'subslot': 2, 'port': 100, 'vlanid': 2160, 'vlanid2': 1284}}

Maybe the parsing of datetime is not right. Check the format. Maybe month and day is swapped.

[anna]

Hi All,

My Code as below, input lines are not same always, -rbash-4.1$ in line is creating problem. I am breaking loop if -rbash-4.1$ found. However missing something.

Input lines

11-03-2019_16:04:10,User-name=0201407256@ttml,NAS-IP-Address=10.124.117.180,Framed-IP-Address=,Calling-Station-Id=00:17:7c:92:a9:d7,Reply-Mesasge=Authentication Success,NAS-Port-Id=pppoe lag-50:3984.2000
12-03-2019_04:05:13,User-name=0201407256@ttml,NAS-IP-Address=10.124.117.180,Framed-IP-Address=,Calling-Station-Id=00:17:7c:92:a9:d7,Reply-Mesasge=Authentication Success,NAS-Port-Id=pppoe lag-50:3984.2000
-rbash-4.1$

import time import re import os import sys dn_details = sys.argv[1]+'@ttml' host = '10.xx.113.xx' username = 'xxxops' password = 'abcd@1234' port = 22 #print(dn_details) remote_conn_pre = paramiko.SSHClient() remote_conn_pre remote_conn_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy()) remote_conn_pre.connect(host, username=username, password=password, look_for_keys=False, allow_agent=False) remote_conn = remote_conn_pre.invoke_shell() output = remote_conn.recv(1000) #print(output) remote_conn.send("\n") remote_conn.send("grep "+dn_details+" transaction2019-03-11_120000_881.log\n") time.sleep(5) output = remote_conn.recv(65535) remote_conn_pre.close() for line in output.decode('utf-8').split('\n')[4:]: d = dict() for i, field in enumerate(line.split(',')): if '-rbash-4.1$' in field: break else: if i == 0: d['date'], d['time'] = field.split('_') else: try: name, value = field.split('=') if name in ['User-name', 'NAS-IP-Address', 'Calling-Station-Id', 'Reply-Mesasge']: d.update({name: value}) except ValueError: continue print(d['date'],d['time'],d['User-name'],d['NAS-IP-Address'],d['Calling-Station-Id'],d['Reply-Mesasge'])

Output:
11-03-2019 16:04:10 0201407256@ttml 10.124.117.180 00:17:7c:92:a9:d7 Authentication Success 12-03-2019 04:05:13 0201407256@ttml 10.124.117.180 00:17:7c:92:a9:d7 Authentication Success Traceback (most recent call last): File "crmsessionlogs.py", line 45, in <module> d['date'], d['time'] = field.split('_') ValueError: not enough values to unpack (expected 2, got 1)

---

its working now, is this correct.

import paramiko import time import re import os import sys dn_details = sys.argv[1]+'@ttml' host = '10.124.113.16' username = 'ttmlops' password = 'Ttml@1234' port = 22 #print(dn_details) remote_conn_pre = paramiko.SSHClient() remote_conn_pre remote_conn_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy()) remote_conn_pre.connect(host, username=username, password=password, look_for_keys=False, allow_agent=False) remote_conn = remote_conn_pre.invoke_shell() output = remote_conn.recv(1000) #print(output) remote_conn.send("\n") remote_conn.send("grep "+dn_details+" transaction2019-03-11_120000_881.log\n") time.sleep(5) output = remote_conn.recv(65535) remote_conn_pre.close() for line in output.decode('utf-8').split('\n')[4:]: if '-rbash-4.1$' in line: break else: d = dict() for i, field in enumerate(line.split(',')): # if '-rbash-4.1$' in field: #break #else: if i == 0: d['date'], d['time'] = field.split('_') else: try: name, value = field.split('=') if name in ['User-name', 'NAS-IP-Address', 'Calling-Station-Id', 'Reply-Mesasge']: d.update({name: value}) except ValueError: continue print(d['date'],d['time'],d['User-name'],d['NAS-IP-Address'],d['Calling-Station-Id'],d['Reply-Mesasge'])

Output:
11-03-2019 16:04:10 0201407256@ttml 10.124.117.180 00:17:7c:92:a9:d7 Authentication Success 12-03-2019 04:05:13 0201407256@ttml 10.124.117.180 00:17:7c:92:a9:d7 Authentication Success