Blocking incoming hostile traffic dynamically with PF and fail2ban on OS X
Here is an improved walkthrough to install fail2ban on OS X 10.10 (it probably works on 10.9 also) based on the (somehow faulty) guide at forgetcomputers.zendisk.com.

The automated installer didn't work at all for me so I did it manually.


cd to ~/Downloads and download fail2ban-0.8.10

cd ~/Downloads
curl -O https://forgetcomputers.zendesk.com/hc/en-us/article_attachments/200287990/fail2ban-0.8.10.tar.gz

Unpack the tar package:

tar xzf fail2ban-0.8.10.tar.gz

cd to fail2ban-0.8.10 and install the software:

cd fail2ban-0.8.10/
sudo python setup.py install

Make a file for the log:

sudo touch /var/log/fail2ban.log

cd back and download the modifications package:

cd ~/Downloads/
curl -O https://forgetcomputers.zendesk.com/hc/en-us/article_attachments/200287980/install_fail2ban_mods.tar.gz

Unpack this package:

tar xzf install_fail2ban_mods.tar.gz

Run the install script from the modifications package:

sudo ./fail2ban_mods/install_fail2ban_mod.sh

Make yourself sudo and rename /etc/fail2ban/jail.local (the file jail.local is superior to jail.conf and might break everything because the installed file contains a totally useless configuration):

sudo bash
mv /etc/fail2ban/jail.local /etc/fail2ban/jail.local.bak

Add the following two lines to /etc/pf.conf with nano /etc/pf.conf:

table &lt;fail2ban&gt; persist  
block drop log quick from &lt;fail2ban&gt; to any

In /etc/fail2ban/jail.conf, modify the [ssh-pf] section at the end with nano as follows:

[ssh-pf]  

enabled  = true  
filter   = sshd  
action   = pf  
logpath  = /var/log/system.log  
maxretry = 3  


You may enter another maxretry count or define an individual bantime or findtime.
In /etc/fail2ban/action.d/pf.conf, ensure that the following values are set and modify them if necessary with nano /etc/fail2ban/action.d/pf.conf:

actionban = /sbin/pfctl -t fail2ban -T add &lt;ip&gt;  
actionunban = pfctl -t fail2ban -T delete `pfctl -t fail2ban -T show 2&gt;/dev/null | grep &lt;ip&gt;`  
[Init] 
tablename = fail2ban
localhost = 127.0.0.1

Shutdown pf, tell it to reload its configuration, and start it again:

pfctl -d
pfctl -f /etc/pf.conf
pfctl -e

Stop the fail2ban daemon if it is already running, and start it with launchctl:

fail2ban-client stop
launchctl load -w /Library/LaunchDaemons/org.fail2ban.init.plist 
launchctl load -w /Library/LaunchDaemons/org.fail2ban.redo.plist 
launchctl load -w /Library/LaunchDaemons/org.fail2ban.reset.plist



Testing the System


Open a terminal window and watch fail2ban's log (live-update):

sudo tail -f /var/log/fail2ban.log

While keeping this terminal active on the server, SSH into the server from a client and watch the server's terminal output (username is arbitrary, since we are testing what will happen when an incorrect login is attempted; replace server_ip with the IP address or hostname of the server):

 ssh username@server_ip

On the client machine, type the wrong password several times until you see a message in fail2ban's log that indicates that the client has been banned. This message will look something like this:


  2015-08-04 18:56:25,001 fail2ban.actions        [216]: NOTICE  [ssh-pf] Ban 192.168.8.15


When you see this message, the client machine's IP has been banned. At this point, any future SSH attempts from this IP (within fail2ban's bantime period) should time-out and be unsuccessful.
To stop tail just enter ctrlC


If you want to install the latest fail2ban 0.9.1


Download manually fail2ban to your ~/Downloads folder
cd to ~/Downloads and unpack the tar package:

cd ~/Downloads
tar xzf fail2ban-0.9.1.tar.gz

cd to fail2ban-0.9.1 and install the software:

cd fail2ban-0.9.1/
sudo python setup.py install

Make a file for the log:

sudo touch /var/log/fail2ban.log

cd back and download the modifications package:

cd ~/Downloads/
curl -O https://forgetcomputers.zendesk.com/hc/en-us/article_attachments/200287980/install_fail2ban_mods.tar.gz

Unpack this package:

tar xzf install_fail2ban_mods.tar.gz

Make a backup of /etc/fail2ban/filter.d/sshd.conf and copy the file from the mod_pack to the fail2ban/filter.d directory. You may copy the other filter.conf but better make a backup of the original files. I didn't test these though.

sudo bash
mv /etc/fail2ban/filter.d/sshd.conf /etc/fail2ban/filter.d/sshd.conf.old
cp ~/Downloads/fail2ban_mods/filter.d/sshd.conf /etc/fail2ban/filter.d/
cp ~/Downloads/fail2ban_mods/fail2ban_reset.sh /private/etc/fail2ban
cp ~/Downloads/fail2ban_mods/lib-launchdaemons/org.fail2ban* /Library/LaunchDaemons

Add the following two lines to /etc/pf.conf with nano /etc/pf.conf:

table &lt;fail2ban&gt; persist  
block drop log quick from &lt;fail2ban&gt; to any

In /etc/fail2ban/jail.conf, modify the [ssh-pf] section at the end with nano as follows:

[ssh-pf]  

enabled  = true  
filter   = sshd  
action   = pf  
logpath  = /var/log/system.log  
maxretry = 3  


You may enter another maxretry count or define an individual bantime or findtime.
In /etc/fail2ban/action.d/pf.conf, ensure that the following values are set and modify them if necessary with nano /etc/fail2ban/action.d/pf.conf:

actionban = /sbin/pfctl -t fail2ban -T add &lt;ip&gt;/32
actionunban = /sbin/pfctl -t fail2ban -T delete &lt;ip&gt;/32 
[Init] 
tablename = fail2ban

Shutdown pf, tell it to reload its configuration, and start it again:

pfctl -d
pfctl -f /etc/pf.conf
pfctl -e

Stop the fail2ban daemon if it is already running and start it again:

fail2ban-client stop
launchctl load -w /Library/LaunchDaemons/org.fail2ban.init.plist 
launchctl load -w /Library/LaunchDaemons/org.fail2ban.redo.plist 
launchctl load -w /Library/LaunchDaemons/org.fail2ban.reset.plist



Now test your system again like described above.

Improvements

Since the implementation of the fail2ban table and rule is non-standard and real fail2ban-blocking doesn't survive a reboot, I reworked the whole pf-mechanism, created a separate anchor (inspired by IceFloor) to remove any dependency on Apple's /etc/pf.conf file and modified the ban action.


Starting from the fail2ban 0.9.1 install described above enter:

sudo bash
nano /etc/fail2ban/action.d/pf.conf 


and change in the file /etc/fail2ban/action.d/pf.conf the lines starting with 

actionban = .... 


to 

actionban = /sbin/pfctl -a fail2ban.anchor -t fail2ban -T add &lt;ip&gt;/32 &amp;&amp; /sbin/pfctl -k &lt;ip&gt;/32 &amp;&amp; /sbin/pfctl -Ef /etc/fail2ban/pf/fail2ban.conf


and 

actionunban = ....


to

actionunban = /sbin/pfctl -a fail2ban.anchor -t fail2ban -T delete &lt;ip&gt;/32

Create a folder pf in /etc/fail2ban/

mkdir /etc/fail2ban/pf

Create three files fail2ban, fail2ban.conf and fail2ban.sh in the previously made folder with the following content; afterwards make fail2ban.sh executable with chmod:

fail2ban:

table &lt;fail2ban&gt; persist
block drop log quick from &lt;fail2ban&gt; to any


fail2ban.conf:

############### LOOPBACK ###############
#
# skip loopback (no filtering on loopback interface)
set skip on lo0

scrub-anchor "com.apple/*"

############### INBOUND ###############
#
anchor "fail2ban.anchor"
load anchor "fail2ban.anchor" from "/etc/fail2ban/pf/fail2ban"


fail2ban.sh:

#!/bin/sh

# start
#
# We need to trap on TERM signals, according to Apple's launchd docs:
#
trap 'exit 1' 15

#
# Use the "ipconfig waitall" command to wait for all the interfaces to come up:
#
ipconfig waitall
sleep 5
#
# System sysctl 
#
sysctl -w net.inet6.ip6.fw.verbose=0
sysctl -w net.inet.ip.fw.verbose=0
sysctl -w net.inet.ip.fw.verbose_limit=0

#
# interface forwarding enabled by default
#
sysctl -w net.inet.ip.forwarding=1

# enable PF and load rules from default fail2ban configuration file using tokens (apple specific PF options -E and -X)
#
/sbin/pfctl -e
/sbin/pfctl -Ef /etc/fail2ban/pf/fail2ban.conf


# Exit with a clean status
exit 0

# this file is public domain and is available to everyone with no exceptions.

Create the file org.fail2ban.plist in /Library/LaunchDaemon with the following content:

&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt;
&lt;plist version="1.0"&gt;
&lt;dict&gt;
    &lt;key&gt;Disabled&lt;/key&gt;
    &lt;false/&gt;
    &lt;key&gt;ExitTimeOut&lt;/key&gt;
    &lt;integer&gt;1&lt;/integer&gt;
    &lt;key&gt;Label&lt;/key&gt;
    &lt;string&gt;org.fail2ban&lt;/string&gt;
    &lt;key&gt;Program&lt;/key&gt;
    &lt;string&gt;/etc/fail2ban/pf/fail2ban.sh&lt;/string&gt;
    &lt;key&gt;RunAtLoad&lt;/key&gt;
    &lt;true/&gt;
&lt;/dict&gt;
&lt;/plist&gt;

Remove the following two lines from /etc/pf.conf with nano /etc/pf.conf:

table &lt;fail2ban&gt; persist  
block drop log quick from &lt;fail2ban&gt; to any

Load the file /Library/LaunchDaemons/org.fail2ban.plist with launchctl after stopping fail2ban and pf:

fail2ban-client stop
pfctl -d
pfctl -f /etc/pf.conf
launchctl load -w /Library/LaunchDaemons/org.fail2ban.plist
fail2ban-client start


or reboot your Mac after loading org.fail2ban.plist with launchctl.


Now banning works properly even after rebooting your system. If you want to add an IP-address manually to block it, just enter:

sudo /sbin/pfctl -a fail2ban.anchor -t fail2ban -T add &lt;ip&gt;/32 &amp;&amp; /sbin/pfctl -k &lt;ip&gt;/32 &amp;&amp; /sbin/pfctl -Ef /etc/fail2ban/pf/fail2ban.conf


To un-ban it (it will not automatically be un-banned after the fail2ban bantime!) enter:

sudo /sbin/pfctl -a fail2ban.anchor -t fail2ban -T delete &lt;ip&gt;/32


So to answer your question:

If you want to use the fail2ban table to ban an IP manually after applying the improvements, you have to enter the command above. The reason for the additional parts:


/sbin/pfctl -k &lt;ip&gt;/32 is needed to kill all of the state entries originating from the specified host.
/sbin/pfctl -Ef /etc/fail2ban/pf/fail2ban.conf to reload the fail2ban.conf gracefully and reflect the changes you made in the table fail2ban.


Without the above improvements you may use:

sudo /sbin/pfctl -t fail2ban -T add &lt;ip&gt;/32 &amp;&amp; /sbin/pfctl -k &lt;ip&gt;/32 &amp;&amp; /sbin/pfctl -Ef /etc/pf.conf




Installation of fail2ban 0.9.1 on El Capitan

I got it installed on El Capitan in rootless mode (sudo nvram boot-args="rootless=0" and reboot) after removing the doc-install part to /usr/share/docs/fail2ban (= the lines 140-143) in the setup.py of fail2ban 0.9.1.

Use the improved 0.9.1 install and config method. If the command pfctl -sA doesn't reveal fail2ban.anchor check for the correct double-quotes (no smart-quotes!) in the file /etc/fail2ban/pf/fail2ban.conf.