title: Unable to parse RFC6587 framed syslog from PulseSecure using syslog-ng OSE 3.33
body: <p>PulseSecure devices are sending syslog conform RFC5424. When TCP is used as transport, RFC6587 framing is prepended to the syslog message (MSG_LEN SP SYSLOG_MSG).
  I also need to receive syslog on same TCP port without RFC6587 framing, so the syslog source is not an option to use as that expects the framing to be present at all times.
  What I tried is to use network driver and disable parsing using flags(no-parse), and have a rewrite rule to remove the RFC6587 framing. The rewrite rule works - framing is indeed removed.
  But when I try to parse the message with the syslog-parser it does not seem to work; e.g. hostname is not extracted (IP address is used).
  I try to use regular expression matching to recognize RFC5424 or RFC3164 syslog which does not work either; it always ends up on the &quot;else&quot; part of the if construction. Even removing the if construction does not help (forcing to use the rfc5424 parser).</p>
  <p>Test configuration is as follows (I use TCP/602 for testing):</p>
  <pre><code>source s_net_tcp602 {
    network(
      port(602)
      transport(&quot;tcp&quot;)
      flags(no-parse)           # Store entire message into MESSAGE/MSG macro
      flags(store-raw-message)  # Store original message in RAWMSG macro
    );
  };
  
  # RFC6587 framing: MSG_LEN SP SYSLOG-MSG
  # This rewrite rule removes the MSG_LEN SP completely from MESSAGE macro if present
  rewrite rw_remove_rfc6587_framing {
    subst(
      &quot;^\d+\s+&lt;&quot;,      # Matches digits followed by one or more spaces and '&lt;' bracket  at start of MESSAGE
      &quot;&lt;&quot;,             # Replace with only '&lt;' bracket so effectively RFC6587 framing removed
      value(&quot;MESSAGE&quot;)
    );
  };
  
  parser p_syslog_rfc3164 {
    syslog-parser(
      drop-invalid(no)
      flags(validate-utf8)
      template(&quot;${MESSAGE}&quot;)
    );
  };
  
  parser p_syslog_rfc5424 {
    syslog-parser(
      flags(syslog-protocol)
      drop-invalid(no)
      flags(validate-utf8)
      template(&quot;${MESSAGE}&quot;)
    );
  };
  
  destination d_test_5424 { file(&quot;/var/log/test5424/${S_YEAR}_${S_MONTH}_${S_DAY}_${HOST}.log&quot; template(&quot;${MESSAGE}\n&quot;) perm(0644) create_dirs(yes)); };
  destination d_test_3164 { file(&quot;/var/log/test3164/${S_YEAR}_${S_MONTH}_${S_DAY}_${HOST}.log&quot; template(&quot;${MESSAGE}\n&quot;) perm(0644) create_dirs(yes)); };
  
  log {
  
    # Complete message should be stored in MESSAGE and RAWMSG macro
    source(s_net_tcp602);
  
    # Strip MSG_LEN part from MESSAGE if it matches RFC6587 framing and continue
    rewrite(rw_remove_rfc6587_framing);
  
    # MESSAGE macro now contains either RFC3164 or RFC5424 formatted message
  
    # Use If mechanism to determine if MESSAGE conforms to RFC5424 or RFC3164
    # Using two different destinations to allow determining result of regular expression matching
    if (message('^&lt;\d+&gt;\d+\s')) {
      parser(p_syslog_rfc5424);
      destination(d_test_5424);
    } else {
      parser(p_syslog_rfc3164);
      destination(d_test_3164);
    };
  
  };
  </code></pre>
  <p>Example syslog message from PulseSecure:</p>
  <pre><code>196 &lt;134&gt;1 2022-03-29T10:38:05+02:00 hostname PulseSecure: - - - 2022-03-29 10:38:05 - ive - &lt;REMOVED MSG&gt;
  </code></pre>
  <p>It might be that this is an obvious mistake I am making, hopefully someone can point me in the right direction...</p>
  <p>Thanks in advance,
  Marco</p>
  <p>I tried to remove the RFC6587 framing using rewrite rule which seems to work, but afterwards the message is not successfully parsed by the syslog-parser.
  I also tried to send syslog from another device (CitrixADC) to the same destination, and it seems that parsing is tried but fails as probably CitrixADC syslog does not fully comply to RFC3164.
  Tried to remove the if-else construction completely and forcing the RFC5424 parsing without success.</p>