Fix bootstrap NAT

Docker will switch the FORWARD filter to DROP if it sets the ip_forward
to 1. Previously we were doing this in a post configuration element
rather than in the puppet run itself. This change moves the ip_forward=1
to puppet so it runs prior to docker being installed. Additionally we
are ensuring that the full set of network rules are being added to the
FORWARD filter because previously we were only setting half of them.
This would allow us to actually not have to use ACCEPT as the default
for the FORWARD filter but this would require additional testing.

Conflicts:
	elements/puppet-stack-config/puppet-stack-config.yaml.template
	elements/undercloud-install/os-refresh-config/post-configure.d/98-undercloud-setup

Change-Id: Ieae6a74f7269bd64606fd80a2a08b2058c24d2c5
Closes-Bug: #1750194
Closes-Bug: #1750874
(cherry picked from commit bfb758b5e7)
(cherry picked from commit 50217d7a93)

elements/puppet-stack-config/puppet-stack-config.pp

@@ -19,6 +19,17 @@ class { '::tripleo::network::os_net_config':
   stage => 'setup',
 }
 
+# enable ip forwarding for the overcloud nodes to access the outside internet
+# in cases where they are on an isolated network
+ensure_resource('sysctl::value', 'net.ipv4.ip_forward', { 'value' => 1 })
+# NOTE(aschultz): clear up old file as this used to be managed via DIB
+file { '/etc/sysctl.d/ip-forward.conf':
+  ensure => absent
+}
+# NOTE(aschultz): LP#1750194 - docker will switch FORWARD to DROP if ip_forward
+# is not enabled first.
+Sysctl::Value['net.ipv4.ip_forward'] -> Package<| title == 'docker' |>
+
 # Run  OpenStack db-sync at every puppet run, in any case.
 Exec<| title == 'neutron-db-sync' |> { refreshonly => false }
 Exec<| title == 'keystone-manage db_sync' |> { refreshonly => false }

elements/puppet-stack-config/puppet-stack-config.yaml.template

@@ -731,9 +731,16 @@ tripleo::firewall::firewall_rules:
     dport: 8787
   '139 apache vhost':
     dport: "%{hiera('ironic_ipxe_port')}"
-  '140 network cidr nat':
+  '140 destination network cidr nat':
     chain: FORWARD
     destination: {{NETWORK_CIDR}}
+    proto: all
+    action: accept
+  '140 source network cidr nat':
+    chain: FORWARD
+    source: {{NETWORK_CIDR}}
+    proto: all
+    action: accept
   # TODO: Do we still want this?
   '141 libvirt network nat':
     chain: FORWARD

elements/undercloud-install/os-refresh-config/post-configure.d/98-undercloud-setup

@@ -111,9 +111,3 @@ if [ "$(hiera enable_mistral)" = "true" ]; then
         mistral workbook-create $workbook
     done
 fi
-
-# IP forwarding is needed to allow the overcloud nodes access to the outside
-# internet in cases where they are on an isolated network.
-sysctl -w net.ipv4.ip_forward=1
-# Make it persistent
-echo "net.ipv4.ip_forward=1" > /etc/sysctl.d/ip-forward.conf