PF options: set timeout and set limit states

This might look obvious and defaults are fine, but when you start playing with PF options, take care. My SIP connections were a bit unstable, so seeking to improve this, after some digging, I came up with these in /etc/pf.conf:

# improving SIP stability
set timeout { udp.first 300, udp.single 150, udp.multiple 900 }

Shortly after, a huge overall slowdowns started to appear on the Internet connection, but only during some times of the day. A simple ping during the slowdown produced this:

# ping edqm.pheur.org
PING edqm.pheur.org (81.252.100.69): 56 data bytes
ping: sendto: Operation not permitted
ping: sendto: Operation not permitted
ping: sendto: Operation not permitted

After some searching I found that the time when it happens corresponds to following NTP pool traffic peaks:

Thanks to ALTQ integrated in PF, this should normally not be a problem since NTP traffic is placed into a lower priority queue. But still, during these peaks the Internet usage was nearly impossible. I also noticed that flushing PF states with pfctl -F state repaired the problem, but only for a short time, it looked like PF was filling it’s internal counters with something during these NTP traffic peaks, and when full, any new connection was refused, so there it was, PF state table was full:

# pfctl -s info

State Table           Total        Rate
  current entries     10005

When I increased UDP states expire time to improve SIP connection stability, this also affected the incoming NTP pool requests, which is using UDP port 123, so during the NTP traffic peaks PF filled it’s state table to 10000 entries (default setting), and then any new connections were refused, resulting in a huge slowdown of the Internet connection. A quick fix was to increase default max state table entries with following:

set limit { states 50000, frags 5000 }