pfSense 2 log visualization with glTail on OSX Lion

pfSense has changed considerably since the first generation of the fantastic FreeBSD based firewall platform and with that, 3rd party utilities such as glTail require updated configs and log parsers to restore the ability to visualize pfSense 2 firewall logs since the logging system in pfSense 2 has been revised considerably.

glTail rendering of logs generated by pfSense 2 on OSX

Necessary changes:
A commit to the glTail repo by pfSense developer JimP for an updated logging parser for pfSense 2 ( included a comment about an additional and mandatory updated log output command used by pfSense 2 to be added to config.yaml called /usr/local/bin/filterpaser.php

In trying out the commit for the updated pfsense2.rb logging processor, glTail output was broken since the updated pfSense 2 logging parser had been revised considerably with a new block naming convention and as well as dropping blocks that were previously used in the pfSense 1 configuration. This resulted a config.yaml revision to use the new block naming convention and depreciated block names.

Additionally updates were needed to the hostwithport value processing for IPv4 traffic in the pfsense2.rb logging processor to parse out the host and port combinations from the old format from pfSense 1 to the format now used by pfSense 2. The committed parser did not yet include this necessary change.

Installation of glTail on OSX Lion:
Provided here are some simple steps to take to get glTail installed on OSX. This as been tested on Lion only.

cd Fudge-gltail-9d2b843
sudo gem install net-ssh ruby-opengl file-tail net-ssh-gateway chipmunk -r
mv config.yaml config.yaml.old

Configuration of glTail for pfSense 2:
The following updated config and logging parser include all the fixes necessary to run glTail against pfSense 2 generated log output from /usr/local/bin/filterpaser.php.

Download and save the following config.yaml to the root of your glTail install and update with the host value with the IP of your pfSense 2 server.

    # Example 1: Connect directly to a pfSense router
        user: root
        command: /usr/sbin/clog -f /var/log/filter.log | /usr/local/bin/filterparser.php
        files: /var/log/filter.log
        parser: pfsense2
        color: white
    # Example 2: Logs forwarded to a syslog host 
    # pfsense2:
    #     host:
    #     user: logview
    #     password: logviewpassword
    #     command: /usr/bin/tail -f -n0
    #     # Adjust this based on where you have syslog direct the output
    #     files: /var/log/hosts/pfsense.log
    #     parser: pfsense
    #     color: 0.2, 1.0, 0.2, 1.0

    dimensions: 1024x700
    min_blob_size: 0.004
    max_blob_size: 0.02
    highlight_color: orange
    bounce: true
        size: 45
        alignment: -0.99
                order: 1
                size: 5
                color: red
                order: 2
                size: 5
                color: magenta
                order: 3
                size: 5
                order: 4
                size: 10
                color: pink

        size: 45
        alignment: 0.99
                order: 1
                size: 15
                order: 2
                size: 15
                color: cyan
                order: 3
                size: 15
                order: 4
                size: 15
                color: blue
    reverse_ip_lookups: true
    reverse_timeout: 0.5

Download and save pfsense2.rb to lib/gl_tail/parsers

# gl_tail.rb - OpenGL visualization of your server traffic
# Copyright 2007 Erlend Simonsen ([email protected])
# Licensed under the GNU General Public License v2 (see LICENSE)
# Parser for pfSense PF Logs, specifically those from pfSense (2.0)
# Jim Pingle ([email protected])
# Available Blocks
#action: block|pass
#rule: Rule number matched
#ipprotocol: carp|icmp|tcp|udp|ah|igmp|esp|gre you get the idea..
#int: This will be the actual interface (fxp0, vlan2, em1, etc) as the 'friendly' name is not put in the logs.
#sourcehost: source host/IP
#sourceport: source port
#destinationhost: destination host/IP
#destinationport: destination port
#sourcedestination:  source host and port > destination host and port
# Use with command:  /usr/sbin/clog -f /var/log/filter.log | /usr/local/bin/filterparser.php
class PFSense2Parser < Parser require 'date' def getipandport(hostwithport) # Test for IPv6 if (hostwithport.count(':') > 2)
      if (hostwithport.count('.') == 1)
        thisport = hostwithport.split('.')[1].to_s
        thishost = hostwithport.split('.')[0].to_s
        thishost = hostwithport
        thisport = "none"
      # IPv4
      if (hostwithport.count('.') == 3 && hostwithport.count(':') == 1)
        thisport = hostwithport.split(':')[-1,1].to_s
        thishost = hostwithport.split(':')[0,1].to_s
        thishost = hostwithport
        thisport = "none"
    if thisport.include?(':')
      thisport = thisport.split(':')[0]
    if thisport.include?(' ')
      thisport = thisport.split(' ')[0]
    return [thishost, thisport]
  def getport(thisport)
    if thisport == "none"
      return ""
      return ":" + thisport.to_s
  def parse( line )
    lmonth, lday, ltod, action, int, ipprotocol, src, dst = line.split(' ')
    ltime = [ lmonth, lday, ltod ].join(' ')
    # Assume the server is in the same time zone as the viewing client.
    timewithoffset = ltime.to_s +
    # Alternately, just set it this way to assume UTC/GMT
    #timewithoffset = ltime.to_s
    hours,minutes,seconds,frac = Date.day_fraction_to_time( - DateTime.parse(timewithoffset))
    # When connecting directly, there is no way to only view the end of the log. The clog program to view
    # circular logs will dump the entire log to the parser, then will tail it showing new messages.
    # Therefore, we can run a simple time check and only view entries from the last 5 minutes, or the
    # "future". On some systems, I have seen the clock show negative (-1hr 59mins) instead of 0, so we
    # can allow "future" messages just to be safe.
    if ((hours == 0) and (minutes < 5)) or (hours < 0) # Debug # printf("Adding entry from %s hours, %s minutes ago\n", hours.to_s, minutes.to_s) sourcehost, sourceport = getipandport(src) destinationhost, destinationport = getipandport(dst) add_activity(:block => 'action',  :name => action.to_s)
      add_activity(:block => 'int',     :name => int.to_s)
      add_activity(:block => 'ipprotocol',   :name => ipprotocol.to_s)
      add_activity(:block => 'sourcehost', :name => sourcehost.to_s)
      if sourceport != "none"
        add_activity(:block => 'sourceport', :name => sourceport.to_s)
      add_activity(:block => 'destinationhost', :name => destinationhost.to_s, :type => 5)
      if destinationport != "none"
        add_activity(:block => 'destinationport', :name => destinationport.to_s, :type => 5)
      add_activity(:block => 'sourcedestination',  :name => sourcehost.to_s + getport(sourceport) + " > " + destinationhost.to_s + getport(destinationport) + " (" + ipprotocol.to_s + ")")
      # Debug
      # printf("Not adding entry from %s hours, %s minutes ago\n", hours.to_s, minutes.to_s)

Executing glTail is done from the root of the glTail installation folder location.

./bin/gl_tail config.yaml

This has not been tested against IPv6 traffic yet.