Good day, gentlemen.I have recently faced a problem when a device connected to the FreeBSD server via StrongSwan can't route outside its subnet. Let me explain in details:FreeBSD 10.0-RELEASE, kernel compiled with (full config: http://pastebin.com/5PQFXqhx):options IPSECoptions IPSEC_NAT_Tdevice cryptodevice cryptodevForwarding turned on in sysctl.conf:net.inet.ip.forwarding=1net.inet6.ip6.forwarding=1We have two interfaces. The first one is 192.168.1.100, the router forwards ports 500 and 4500 from the Internet to this IP (so we can connect from anywhere to the sweet home network). The second one is 10.0.0.1 which is used for Samba and other services./etc/rc.conf:hostname=bsdserverifconfig_igb0=inet 192.168.1.100 netmask 255.255.255.0ifconfig_igb1=inet 10.0.0.1 netmask 255.255.255.0defaultrouter=192.168.1.1gateway_enable=YESipv6_gateway_enable=YESipsec.conf:config setup    strictcrlpolicy = noca bsdserver    cacert = /usr/local/etc/ipsec.d/cacerts/bsdserver_CA.derconn androidauto = adddpdaction = clearkeyexchange = ikev2esp=aes-aes256-sha-modp1024,aes256-sha512-modp4096ike=aes-aes256-sha-modp1024,aes256-sha512-modp4096mobike = yesleft = 192.168.1.100leftsubnet = 0.0.0.0/0,::0/0leftauth = pubkeyleftid = 111.111.111.111 # real white ip hereleftcert = /usr/local/etc/ipsec.d/certs/111.111.111.111_cert.derleftfirewall = yesleftsourceip = %config4,%config6right = %anyrightsourceip = 10.0.0.3/32rightauth = pubkeyrightid = C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.comrightcert = /usr/local/etc/ipsec.d/certs/android_cert.derThis is the output from ipsec start --nofork:Starting strongSwan 5.1.1 IPsec [starter]...no netkey IPsec stack detectedno KLIPS IPsec stack detectedno known IPsec stack detected, ignoring!00[DMN] Starting IKE charon daemon (strongSwan 5.1.1, FreeBSD 10.0-RELEASE, amd64)00[KNL] unable to set UDP_ENCAP: Invalid argument00[NET] enabling UDP decapsulation for IPv6 on port 4500 failed00[CFG] loading ca certificates from '/usr/local/etc/ipsec.d/cacerts'00[CFG]   loaded ca certificate C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com from '/usr/local/etc/ipsec.d/cacerts/UNIT_CA.der'00[CFG] loading aa certificates from '/usr/local/etc/ipsec.d/aacerts'00[CFG] loading ocsp signer certificates from '/usr/local/etc/ipsec.d/ocspcerts'00[CFG] loading attribute certificates from '/usr/local/etc/ipsec.d/acerts'00[CFG] loading crls from '/usr/local/etc/ipsec.d/crls'00[CFG] loading secrets from '/usr/local/etc/ipsec.secrets'00[CFG]   loaded RSA private key from '/usr/local/etc/ipsec.d/private/111.111.111.111_key.der'00[LIB] loaded plugins: charon test-vectors curl aes des blowfish rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp xcbc cmac hmac attr kernel-pfkey kernel-pfroute resolve socket-default stroke updown eap-identity eap-aka eap-aka-3gpp2 eap-md5 eap-mschapv2 eap-dynamic eap-tls eap-ttls eap-peap xauth-generic xauth-eap whitelist addrblock00[LIB] unable to load 4 plugin features (4 due to unmet dependencies)00[JOB] spawning 16 worker threadscharon (49667) started after 220 ms16[CFG] received stroke: add connection 'android'16[CFG] adding virtXXl IP address pool 10.0.0.3/3216[CFG] 'android' has both left- and rightsourceip, but IKE can negotiate one virtXXl IP only, ignoring local virtXXl IP16[CFG]   loaded certificate C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com from '/usr/local/etc/ipsec.d/certs/111.111.111.111_cert.der'16[CFG]   loaded certificate C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com from '/usr/local/etc/ipsec.d/certs/android_cert.der'16[CFG] added configuration 'android'16[NET] received packet: from 22.222.222.22[16483] to 192.168.1.100[500] (660 bytes)16[ENC] parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) ]16[IKE] 22.222.222.22 is initiating an IKE_SA16[IKE] IKE_SA (unnamed)[1] state change: CREATED => CONNECTING16[IKE] local host is behind NAT, sending keep alives16[IKE] remote host is behind NAT16[IKE] sending cert request for C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com16[ENC] generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(MULT_AUTH) ]16[NET] sending packet: from 192.168.1.100[500] to 22.222.222.22[16483] (337 bytes)16[NET] received packet: from 22.222.222.22[16549] to 192.168.1.100[4500] (1964 bytes)16[ENC] parsed IKE_AUTH request 1 [ IDi CERT N(INIT_CONTACT) CERTREQ AUTH CPRQ(ADDR ADDR6 DNS DNS6) N(ESP_TFC_PAD_N) SA TSi TSr N(MOBIKE_SUP) N(NO_ADD_ADDR) N(MULT_AUTH) N(EAP_ONLY) ]16[IKE] received cert request for C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com16[IKE] received end entity cert C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com16[CFG] looking for peer configs matching 192.168.1.100[%any]...22.222.222.22[C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com]16[CFG] selected peer config 'android'16[CFG]   using trusted ca certificate C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com16[CFG] checking certificate status of C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com16[CFG] certificate status is not available16[CFG]   reached self-signed root ca with a path length of 016[CFG]   using trusted certificate C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com16[IKE] authentication of 'C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com' with RSA signature successful16[IKE] processing INTERNAL_IP4_ADDRESS attribute16[IKE] processing INTERNAL_IP6_ADDRESS attribute16[IKE] processing INTERNAL_IP4_DNS attribute16[IKE] processing INTERNAL_IP6_DNS attribute16[IKE] received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding16[IKE] peer supports MOBIKE16[IKE] authentication of '111.111.111.111' (myself) with RSA signature successful16[IKE] IKE_SA android[1] established between 192.168.1.100[111.111.111.111]...22.222.222.22[C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com]16[IKE] IKE_SA android[1] state change: CONNECTING => ESTABLISHED16[IKE] scheduling reauthentication in 9802s16[IKE] maximum IKE_SA lifetime 10342s16[IKE] sending end entity cert C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com16[IKE] peer requested virtXXl IP %any16[CFG] assigning new lease to 'C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com'16[IKE] assigning virtXXl IP 10.0.0.3 to peer 'C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com'16[IKE] peer requested virtXXl IP %any616[IKE] no virtXXl IP found for %any6 requested by 'C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com'16[IKE] CHILD_SA android{1} established with SPIs c11f2b7e_i f7ac6573_o and TS 0.0.0.0/0 ::/0 === 10.0.0.3/3216[CHD] updown: /usr/local/libexec/ipsec/_updown: iptables: not found16[CHD] updown: /usr/local/libexec/ipsec/_updown: iptables: not found16[ENC] generating IKE_AUTH response 1 [ IDr CERT AUTH CPRP(ADDR) N(ESP_TFC_PAD_N) SA TSi TSr N(AUTH_LFT) N(MOBIKE_SUP) N(ADD_4_ADDR) ]16[NET] sending packet: from 192.168.1.100[4500] to 22.222.222.22[16549] (1596 bytes)Looking at the tcpdump output: 10.0.0.3 is an IP that Android client got assigned by StrongSwan.DNS:22:21:43.971797 IP 10.0.0.3.55123 > ns1.kyivstar.net.domain: 41221+ A? www.cdn.viber.com. (35)22:26:00.484696 IP 10.0.0.3.25462 > gi-dns1-kie3.kyivstar.net.domain: 436+ A? mtalk.google.com. (34)22:25:59.225291 IP 10.0.0.3.53999 > ns1.kyivstar.net.domain: 56585+ A? graph.facebook.com. (36)OK, pinging bsdserver works: 22:26:06.460464 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0x96), length 13222:26:06.460749 IP bsdserver.local.sae-urn > 46-211-183-74-gprs.kyivstar.net.16445: UDP-encap: ESP(spi=0xe77a2567,seq=0x1f), length 132But pinging 8.8.8.8 doesn't:22:30:51.179070 IP 10.0.0.3 > google-public-dns-a.google.com: ICMP echo request, id 3, seq 6, length 6422:30:52.098315 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0xce), length 13222:30:52.098825 IP 10.0.0.3 > google-public-dns-a.google.com: ICMP echo request, id 3, seq 7, length 6422:30:52.754209 STP 802.1d, Config, Flags [none], bridge-id 8000.b0:48:7a:c2:f2:92.8001, length 4322:30:53.078436 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0xcf), length 13222:30:53.079070 IP 10.0.0.3 > google-public-dns-a.google.com: ICMP echo request, id 3, seq 8, length 6422:30:53.358693 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: isakmp-nat-keep-alive22:30:53.738572 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0xd0), length 11622:30:53.739164 IP 10.0.0.3.25984 > gi-dns1-kie3.kyivstar.net.domain: 1638+ A? www.cdn.viber.com. (35)22:30:54.098740 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0xd1), length 13222:30:54.099248 IP 10.0.0.3 > google-public-dns-a.google.com: ICMP echo request, id 3, seq 9, length 6422:30:54.754083 STP 802.1d, Config, Flags [none], bridge-id 8000.b0:48:7a:c2:f2:92.8001, length 43And routing table from Android device:I feel like problem isn't exactly IPsec related. Any bright ideas about possible solution?
FreeBSD and StrongSwan routing issue
freebsd;routing;ipsec