pfSense 2 log visualization with glTail on OSX Lion
pfSense has changed considerably since the first generation of the fantastic FreeBSD based firewall platform and with that, 3rd party utilities such as glTail require updated configs and log parsers to restore the ability to visualize pfSense 2 firewall logs since the logging system in pfSense 2 has been revised considerably.
Necessary changes:
A commit to the glTail repo by pfSense developer JimP for an updated logging parser for pfSense 2 (https://github.com/Fudge/gltail/pull/14) included a comment about an additional and mandatory updated log output command used by pfSense 2 to be added to config.yaml called /usr/local/bin/filterpaser.php
In trying out the commit for the updated pfsense2.rb logging processor, glTail output was broken since the updated pfSense 2 logging parser had been revised considerably with a new block naming convention and as well as dropping blocks that were previously used in the pfSense 1 configuration. This resulted a config.yaml revision to use the new block naming convention and depreciated block names.
Additionally updates were needed to the hostwithport value processing for IPv4 traffic in the pfsense2.rb logging processor to parse out the host and port combinations from the old xxx.xxx.xxx.xxx.zzz format from pfSense 1 to the xxx.xxx.xxx.xxx:zzz format now used by pfSense 2. The committed parser did not yet include this necessary change.
Installation of glTail on OSX Lion:
Provided here are some simple steps to take to get glTail installed on OSX. This as been tested on Lion only.
wget https://github.com/Fudge/gltail/zipball/master unzip Fudge-gltail-9d2b843.zip cd Fudge-gltail-9d2b843 sudo gem install net-ssh ruby-opengl file-tail net-ssh-gateway chipmunk -r mv config.yaml config.yaml.old
Configuration of glTail for pfSense 2:
The following updated config and logging parser include all the fixes necessary to run glTail against pfSense 2 generated log output from /usr/local/bin/filterpaser.php.
Download and save the following config.yaml to the root of your glTail install and update with the host value with the IP of your pfSense 2 server.
servers: # Example 1: Connect directly to a pfSense router pfsense1: host: 192.168.1.1 user: root password: command: /usr/sbin/clog -f /var/log/filter.log | /usr/local/bin/filterparser.php files: /var/log/filter.log parser: pfsense2 color: white # Example 2: Logs forwarded to a syslog host # pfsense2: # host: 192.168.1.2 # user: logview # password: logviewpassword # command: /usr/bin/tail -f -n0 # # Adjust this based on where you have syslog direct the output # files: /var/log/hosts/pfsense.log # parser: pfsense # color: 0.2, 1.0, 0.2, 1.0 config: dimensions: 1024x700 min_blob_size: 0.004 max_blob_size: 0.02 highlight_color: orange bounce: true left_column: size: 45 alignment: -0.99 blocks: action: order: 1 size: 5 color: red ipprotocol: order: 2 size: 5 color: magenta int: order: 3 size: 5 sourcedestination: order: 4 size: 10 color: pink right_column: size: 45 alignment: 0.99 blocks: destinationhost: order: 1 size: 15 destinationport: order: 2 size: 15 color: cyan sourcehost: order: 3 size: 15 sourceport: order: 4 size: 15 color: blue resolver: reverse_ip_lookups: true reverse_timeout: 0.5
Download and save pfsense2.rb to lib/gl_tail/parsers
# gl_tail.rb - OpenGL visualization of your server traffic # Copyright 2007 Erlend Simonsen ([email protected]) # # Licensed under the GNU General Public License v2 (see LICENSE) # # Parser for pfSense PF Logs, specifically those from pfSense (2.0) # Jim Pingle ([email protected]) # Available Blocks #action: block|pass #rule: Rule number matched #ipprotocol: carp|icmp|tcp|udp|ah|igmp|esp|gre you get the idea.. #int: This will be the actual interface (fxp0, vlan2, em1, etc) as the 'friendly' name is not put in the logs. #sourcehost: source host/IP #sourceport: source port #destinationhost: destination host/IP #destinationport: destination port #sourcedestination: source host and port > destination host and port # Use with command: /usr/sbin/clog -f /var/log/filter.log | /usr/local/bin/filterparser.php class PFSense2Parser < Parser require 'date' def getipandport(hostwithport) # Test for IPv6 if (hostwithport.count(':') > 2) if (hostwithport.count('.') == 1) thisport = hostwithport.split('.')[1].to_s thishost = hostwithport.split('.')[0].to_s else thishost = hostwithport thisport = "none" end else # IPv4 if (hostwithport.count('.') == 3 && hostwithport.count(':') == 1) thisport = hostwithport.split(':')[-1,1].to_s thishost = hostwithport.split(':')[0,1].to_s else thishost = hostwithport thisport = "none" end end if thisport.include?(':') thisport = thisport.split(':')[0] end if thisport.include?(' ') thisport = thisport.split(' ')[0] end return [thishost, thisport] end def getport(thisport) if thisport == "none" return "" else return ":" + thisport.to_s end end def parse( line ) lmonth, lday, ltod, action, int, ipprotocol, src, dst = line.split(' ') ltime = [ lmonth, lday, ltod ].join(' ') # Assume the server is in the same time zone as the viewing client. timewithoffset = ltime.to_s + DateTime.now().zone() # Alternately, just set it this way to assume UTC/GMT #timewithoffset = ltime.to_s hours,minutes,seconds,frac = Date.day_fraction_to_time(DateTime.now() - DateTime.parse(timewithoffset)) # When connecting directly, there is no way to only view the end of the log. The clog program to view # circular logs will dump the entire log to the parser, then will tail it showing new messages. # Therefore, we can run a simple time check and only view entries from the last 5 minutes, or the # "future". On some systems, I have seen the clock show negative (-1hr 59mins) instead of 0, so we # can allow "future" messages just to be safe. if ((hours == 0) and (minutes < 5)) or (hours < 0) # Debug # printf("Adding entry from %s hours, %s minutes ago\n", hours.to_s, minutes.to_s) sourcehost, sourceport = getipandport(src) destinationhost, destinationport = getipandport(dst) add_activity(:block => 'action', :name => action.to_s) add_activity(:block => 'int', :name => int.to_s) add_activity(:block => 'ipprotocol', :name => ipprotocol.to_s) add_activity(:block => 'sourcehost', :name => sourcehost.to_s) if sourceport != "none" add_activity(:block => 'sourceport', :name => sourceport.to_s) end add_activity(:block => 'destinationhost', :name => destinationhost.to_s, :type => 5) if destinationport != "none" add_activity(:block => 'destinationport', :name => destinationport.to_s, :type => 5) end add_activity(:block => 'sourcedestination', :name => sourcehost.to_s + getport(sourceport) + " > " + destinationhost.to_s + getport(destinationport) + " (" + ipprotocol.to_s + ")") else # Debug # printf("Not adding entry from %s hours, %s minutes ago\n", hours.to_s, minutes.to_s) end end end
Executing:
Executing glTail is done from the root of the glTail installation folder location.
./bin/gl_tail config.yaml
Notes:
This has not been tested against IPv6 traffic yet.