Merge pull request #4 from albertogeniola/development

Development

example/3. Simple NAT/README.md → example/3. Simple TCP NAT/README.md

@@ -14,3 +14,6 @@ The VyOS instance is configured to apply PAT to TCP and UDP packets coming from
 and to masquerade it via ETH1 address (10.0.0.3/16). This configuration handles the traffic against RFC1918 targets belonging
 to the external VPC, but also performs NATTING against public endpoints, using the VyOS ETH1 public address. 
 
+In case you need to only apply transparent nat/pat to traffic coming from specific instances within the internal_vpc, you 
+could simply change the default route to apply only to instances with a specific tag, making sure to assign such a tag to the
+VMs to NAT.

example/4. NAT with balancing and HA/.gitignore

@@ -0,0 +1 @@
+terraform.tfvars

example/4. NAT with balancing and HA/README.md

@@ -0,0 +1,35 @@
+# VyOS as RFC1918 High Available NAT
+This module is an evolution of the NAT example. Instead of using a single VyOS instance, it relies on
+two active/active instances, each one running on a distinct zone. 
+An internal load balancer takes care of dispatching the connections among the two instances.
+
+<img src="./ha-nat.png" width=600 alt="High Available PAT implementation"/>
+
+A custom static route redirects all the traffic from the internal vpc network towards internal load balancer on top of the two VyOS instances.
+Both the VyOS instances are configured to apply PAT to all IP traffic (including ICMP, TCP, UDP, etc) coming from 10.10.0.0/16 (address space of internal_subnet),
+masquerading it via ETH1 address (10.0.0.3/16). This configuration handles the traffic against RFC1918 targets belonging
+to the external VPC, but also performs NATTING against public endpoints, using the VyOS ETH1 public address. 
+
+In case you need to only apply transparent nat/pat to traffic coming from specific instances within the internal_vpc, you 
+could simply change the default route to apply only to instances with a specific tag, making sure to assign such a tag to the
+VMs to NAT.
+
+## Limitation
+We use two ILBs for TCP and UDP traffic translation. The usage of a single ILB with shared IP is not supported on GCP when using custom routes
+with an ILB as next hop: [Forwarding rules that use a common internal IP address (--purpose=SHARED_LOADBALANCER_VIP) are not supported.](https://cloud.google.com/load-balancing/docs/internal/ilb-next-hop-overview#additional_considerations).
+
+The usage of two VyOS instances as NAT devices in an ACTIVE/ACTIVE configuration without any client stickiness logic might be an issues in special cases. For instance, consider the FTP protocol case. 
+A classic FTP session usually establishes two or more TCP connections: one for the control channel 
+(to issue FTP commands), and one or more for the data transfers. The problem in this situation is that there is asolutely no warranty that all the TCP connections go through the very same VyOS instance. 
+If that happens, most FTP servers will refuse the connection of all the data-transfer channels that
+are NATTED via the other VyOS instance, as they are masquerated via an IP address that differs from the
+one used by the control channel.
+
+To solve this issue you might:
+- change the ILB logic, so that all the traffic coming from a client IP is routed consistently using the same VyOS instance.
+- rely on example 5, which applies masquerading using an IP address of another ILB.
+
+
+## Special notes
+Please be aware that the usage of a custom route targeting a forwarding rule (TCP/UDP load balancing) is required to handle the all TCP, UDP and lower-level IP traffic.
+This "route-all" feature is an exception which bypasses the backend-service and forwarding rule limitations, as [explained here](https://cloud.google.com/load-balancing/docs/internal/ilb-next-hop-overview).

example/4. NAT with balancing and HA/internal-instance.tf

@@ -0,0 +1,28 @@
+data "google_compute_image" "debian" {
+  family    = "debian-11"
+  project   = "debian-cloud"
+}
+
+resource "google_compute_instance" "internal_instance" {
+  project       = var.project_id
+  name          = "internal-vm"
+  machine_type  = "n2-standard-2"
+  zone          = "europe-west8-b"
+
+  boot_disk {
+    initialize_params {
+      image = data.google_compute_image.debian.self_link
+    }
+  }
+  tags = [local.allow_iap_ssh_inbound_tag]
+  network_interface {
+    network     = google_compute_network.vyos_internal_vpc.self_link
+    subnetwork  = google_compute_subnetwork.vyos_internal_subnet.self_link
+    network_ip  = local.internal_test_vm_ip
+  }
+
+  depends_on = [
+    module.vyos_instance_1,
+    module.vyos_instance_2
+  ]
+}

example/4. NAT with balancing and HA/networking.tf

@@ -0,0 +1,176 @@
+# Define an external VPC
+resource "google_compute_network" "vyos_external_vpc" {
+  name                      = "vyos-external-vpc"
+  project                   = var.project_id 
+  auto_create_subnetworks   = false
+}
+
+resource "google_compute_subnetwork" "vyos_external_subnet" {
+  region                    = var.region
+  name                      = "vyos-external-subnet"
+  project                   = var.project_id 
+  network                   = google_compute_network.vyos_external_vpc.self_link
+  ip_cidr_range             = local.external_subnet_cidr
+  private_ip_google_access  = true  
+}
+
+# Define an internal VPC
+resource "google_compute_network" "vyos_internal_vpc" {
+  name                      = "vyos-internal-vpc"
+  project                   = var.project_id 
+  auto_create_subnetworks   = false
+}
+
+resource "google_compute_subnetwork" "vyos_internal_subnet" {
+  region                    = var.region
+  name                      = "vyos-internal-subnet"
+  project                   = var.project_id 
+  network                   = google_compute_network.vyos_internal_vpc.self_link
+  ip_cidr_range             = local.internal_subnet_cidr
+  private_ip_google_access  = true  
+}
+
+# Enable firewall rules for nat access
+resource "google_compute_firewall" "nat_internal_vms" {
+  project       = var.project_id 
+  name          = "fw-inbound-nat-internal"
+  network       = google_compute_network.vyos_internal_vpc.self_link
+  source_ranges = [local.internal_subnet_cidr]
+  target_service_accounts = [module.vyos_instance_1.sa_email, module.vyos_instance_2.sa_email]
+  allow {
+    protocol  = "tcp"
+  }
+  allow {
+    protocol  = "udp"
+  }
+}
+
+# Enable firewall rules for SSH access for the internal VM
+resource "google_compute_firewall" "ssh_iap_internal_vms" {
+  project       = var.project_id 
+  name          = "fw-inbound-iap-ssh"
+  network       = google_compute_network.vyos_internal_vpc.self_link
+  target_tags   = [local.allow_iap_ssh_inbound_tag]
+  source_ranges = local.iap_cidrs
+  allow {
+    protocol  = "tcp"
+    ports     = [ 22 ]
+  }
+}
+
+# Enable ilb health-checks
+resource "google_compute_firewall" "ilb_tcp_health_checks" {
+  project       = var.project_id 
+  name          = "fw-inbound-ilb-hc"
+  network       = google_compute_network.vyos_internal_vpc.self_link
+  target_service_accounts = [module.vyos_instance_1.sa_email, module.vyos_instance_2.sa_email]
+  source_ranges = local.ilb_hc_cidrs
+  allow {
+    protocol  = "tcp"
+    ports     = [ 22 ] # TODO: change if using a different method to attestate health of the workload
+  }
+}
+
+# Instance group for the VyOS VMs
+resource "google_compute_instance_group" "vyos_nat_unmanaged_primary" {
+  project   = var.project_id
+  zone      = var.zone_primary
+  name      = "vyos-nat-primary"
+
+  instances = [
+    module.vyos_instance_1.vm_id
+  ]
+  network = google_compute_network.vyos_external_vpc.self_link
+}
+resource "google_compute_instance_group" "vyos_nat_unmanaged_secondary" {
+  project   = var.project_id
+  zone      = var.zone_secondary
+  name      = "vyos-nat-secondary"
+
+  instances = [
+    module.vyos_instance_2.vm_id
+  ]
+  network = google_compute_network.vyos_external_vpc.self_link
+}
+
+# Backend services
+resource "google_compute_region_backend_service" "vyos_nat_backend" {
+  project               = var.project_id
+  name                  = "vyos-nat"
+  region                = var.region
+  protocol              = "TCP"         # This really does not have any effect, as next hop ILB will forward all the traffic to the VMs.
+  load_balancing_scheme = "INTERNAL"
+  timeout_sec           = 10
+  health_checks         = [google_compute_region_health_check.vyos_nat_hc.self_link]
+
+  # We need to specify the network to be used as backend service, as our VMs do have multiple NICs
+  #  and we want to load-balance on the secondary NIC (internal), which is connected to the internal
+  #  vpc.
+  network               = google_compute_network.vyos_internal_vpc.self_link 
+
+  backend {
+    group = google_compute_instance_group.vyos_nat_unmanaged_primary.self_link
+    balancing_mode  = "CONNECTION"
+  }
+  backend {
+    group = google_compute_instance_group.vyos_nat_unmanaged_secondary.self_link
+    balancing_mode  = "CONNECTION"
+  }
+}
+
+# Health Checks for the backend service
+resource "google_compute_region_health_check" "vyos_nat_hc" {
+  project             = var.project_id
+  region              = var.region
+  name                = "vyos-nat-hc"
+  log_config {
+    enable = true
+  }
+  tcp_health_check {
+    port = 22 # TODO: use an HTTP HC instead (vyos apis?)
+  }
+}
+
+# Forwarding rule for VyOS ILB
+resource "google_compute_forwarding_rule" "nat_forwarding" {
+  project               = var.project_id
+  name                  = "nat-ilb-forwarding-rule"
+  region                = var.region
+  depends_on            = [google_compute_subnetwork.vyos_internal_subnet]
+  ip_address            = local.nat_ilb_address
+  ip_protocol           = "TCP" #  This really does not have any effect, as next hop ILB will forward all the traffic to the VMs.
+  load_balancing_scheme = "INTERNAL"
+  all_ports             = true
+  backend_service       = google_compute_region_backend_service.vyos_nat_backend.self_link
+
+  network               = google_compute_network.vyos_internal_vpc.self_link
+  subnetwork            = google_compute_subnetwork.vyos_internal_subnet.self_link
+}
+
+
+# Routing traffic to the NAT instances via ILBs
+# Default route for internal VPC
+resource "google_compute_route" "default_ilb_route" {
+  project     = var.project_id
+  name        = "default-route-to-vyos-nat"
+  dest_range  = "0.0.0.0/0"
+  network     = google_compute_network.vyos_internal_vpc.self_link
+  next_hop_ilb = google_compute_forwarding_rule.nat_forwarding.self_link
+  priority    = 100
+}
+
+locals {
+  external_subnet_cidr        = "10.0.0.0/16"
+  internal_subnet_cidr        = "10.10.0.0/16"
+  allow_iap_ssh_inbound_tag   = "ssh-iap"
+  iap_cidrs                   = ["35.235.240.0/20"]
+  nat_ilb_address             = cidrhost(local.internal_subnet_cidr, 3)
+  external_vyos_1_ip          = cidrhost(local.external_subnet_cidr, 5)
+  external_vyos_2_ip          = cidrhost(local.external_subnet_cidr, 6)
+  internal_vyos_1_ip          = cidrhost(local.internal_subnet_cidr, 5)
+  internal_vyos_2_ip          = cidrhost(local.internal_subnet_cidr, 6)
+
+  internal_test_vm_ip         = cidrhost(local.internal_subnet_cidr, 7)
+
+  ilb_hc_cidrs       = ["35.191.0.0/16", "130.211.0.0/22"]
+}

example/4. NAT with balancing and HA/variables.tf

@@ -0,0 +1,4 @@
+variable project_id {}
+variable "region" {}
+variable "zone_primary" {}
+variable "zone_secondary" {}

example/4. NAT with balancing and HA/vyos.config

@@ -0,0 +1,96 @@
+interfaces {
+    ethernet eth0 {
+        address dhcp
+    }
+    ethernet eth1 {
+        address dhcp
+    }
+    loopback lo {
+    }
+}
+nat {
+    source {
+        rule 1 {
+            outbound-interface eth0
+            protocol tcp_udp
+            source {
+                address 10.10.0.0/16
+            }
+            translation {
+                address masquerade
+            }
+        }
+    }
+}
+service {
+    ssh {
+        listen-address 0.0.0.0
+        port 22
+        disable-password-authentication
+    }
+}
+system {
+    config-management {
+        commit-revisions 100
+    }
+    conntrack {
+        modules {
+            ftp
+            h323
+            nfs
+            pptp
+            sip
+            sqlnet
+            tftp
+        }
+    }
+    console {
+        device ttyS0 {
+            speed 38400
+        }
+    }
+    host-name vyos-gce
+    login {
+        banner {
+            post-login "Welcome to VyOs\n=========================================================================\nPlease note the following:\n    * This image is integrated with Google Ops Agent and supports metadata\nssh-keys login;\n    * You can still manage vyos configuration using the Serial Console,\nlogging in as vyos credentials: vyos/vyos;\n    * Note: vyos ssh plaintext/password is disabled.\n\nBuilt using https://github.com/albertogeniola/terraform-gce-vyos\n========================================================================="
+        }
+        user vyos {
+            authentication {
+                encrypted-password $6$gf2ShN8QhLqyH$WedSwHWXMYgC/qoM7ibe2XwdZro.A.qsYqMH0P9jf5opselu31ACTUD1bkRTL8S3WeKjoJ1Uu2xOgZXSV9SOr1
+                plaintext-password ""
+            }
+        }
+        user admin {
+        }
+    }
+    name-server 169.254.169.254
+    name-server 8.8.8.8
+    name-server 8.8.4.4
+
+    ntp {
+        server time1.vyos.net {
+        }
+        server time2.vyos.net {
+        }
+        server time3.vyos.net {
+        }
+    }
+    static-host-mapping {
+        host-name metadata.google.internal {
+            inet 169.254.169.254
+        }
+    }
+    syslog {
+        global {
+            facility all {
+                level info
+            }
+            facility protocols {
+                level debug
+            }
+        }
+    }
+}
+// Warning: Do not remove the following line.
+// vyos-config-version: "broadcast-relay@1:cluster@1:config-management@1:conntrack@3:conntrack-sync@2:dhcp-relay@2:dhcp-server@6:dhcpv6-server@1:dns-forwarding@3:firewall@5:https@2:interfaces@22:ipoe-server@1:ipsec@5:isis@1:l2tp@3:lldp@1:mdns@1:nat@5:ntp@1:pppoe-server@5:pptp@2:qos@1:quagga@8:rpki@1:salt@1:snmp@2:ssh@2:sstp@3:system@21:vrrp@2:vyos-accel-ppp@2:wanloadbalance@3:webproxy@2:zone-policy@1"
+// Release version: equuleus