Script fails to execute

[tabletseeker]

System: Debian Bookworm
Dependencies have been installed.
rich via apt-get install python3-rich
file-read-backwards was built from source
Command: sudo python3 py_fw_log.py -l /var/log/kern.log -s 50

Traceback (most recent call last):
  File "/home/user/py-fw-log/py_fw_log.py", line 188, in <module>
    py_fw_log(sys.argv)
  File "/home/user/py-fw-log/py_fw_log.py", line 51, in py_fw_log
    fw_log_view(arg_log_file_name, int(
                                   ^^^^
ValueError: invalid literal for int() with base 10: ''

Traceback (most recent call last):
  File "/home/user/py-fw-log/py_fw_log.py", line 188, in <module>
    py_fw_log(sys.argv)
  File "/home/user/py-fw-log/py_fw_log.py", line 51, in py_fw_log
    fw_log_view(arg_log_file_name, int(
  File "/home/user/py-fw-log/py_fw_log.py", line 63, in __init__
    self.main()
  File "/home/user/py-fw-log/py_fw_log.py", line 118, in main
    self.data['IN'], self.data['PROTO'], self.data['SRC'],
    ~~~~~~~~~^^^^^^
KeyError: 'IN'

[dunneja]

pyfwlog script is designed to parse iptables log files, not full kernel log files. It's clear from your second traceback that it's trying to index a line item that doesn't match the regex, hence the KeyError. Filter out your iptables to a seperate log file, and specifiy this with the -l flag.

[tabletseeker]

Hi, sorry you're right and thanks for the quick reply.

Is there a way I can pipe the output of watch -n 1 "cat /var/log/kern.log | grep -E 'DROP|ACCEPT'" directly to the script?

I would like the analysis to be ongoing and refresh by increments of 1 second.

[dunneja]

Hi, no worries!

You can use rsyslog to filter out iptables logs based on the iptables --log-prefix option to a log file.. Rsyslog in combination with logrorate.d means the files are rotated once a day for example, so the script only outputs recently dropped connections.

I've been using this on all my servers as a snapshot command of who has been knocking on the front door, so it was written to be a parser of a log file on demand, rather than a running process that constantly updates every x seconds.

I like the idea of having a flag that loops through the log file and displays new line items as they are logged / dropped in real time.

I'm actually refactoring the code for this project, due to the Ubuntu 23.04 python package changes (you can't install pip packages to the world anymore). I plan to wrap the script and it's module dependencies into an executable (i've done this roughly for my own purposes recently) and package into a *.deb and *.rpm file respectively for easy install / distribution.

Since i'm tidying this all up, i'll take a look at adding a -t flag for "tailing" the log file in real time.

Watch this space!

[tabletseeker]

Yeah exactly that's possible with the rsyslog package. I think I've filtered them like this once by --log-level but didn't like that other messages were also included like interfaces entering/exiting promiscuous mode etc. If it's possible to grab anything with a --log-prefix (possibly with a regex) instead of specific ones, that might be interesting, but I have hundreds of different prefixes on my server.
Thanks a bunch for your work man!
I've created my own little ghetto solution with a regex for SRC/DST/PROTO/SPT/DPT and a script that tails /var/log/kern.log while continuously grepping that regex from every new line that's written to it.