Extend Layer2 Network Across Data Center with FortiGate VXLAN

I had prepared a lab to study the concept of how to Extend Layer2 Network Across Data Center with FortiGate VXLAN

What is VXLAN

Virtual eXtensible LAN (VXLAN – RFC7348) act as Layer 2 virtual networks over Layer 3 physical networks to stretch Layer 2 networks.

It uses MAC Address-in-User Datagram Protocol (MAC-in-UDP) encapsulation. The original Layer 2 frame has a VXLAN header added and is then placed in
a UDP-IP packet.

VXLAN tunnel endpoint (VTEP) devices perform VXLAN encapsulation and decapsulation. Each VTEP has two interface types: one or more virtual interfaces called VXLAN Network Identifier (VNI) interfaces to which you apply your security policy, and a regular interface called the VTEP source interface that tunnels the VNI interfaces between VTEPs.

The destination port is UDP port 4789 by default (user configurable).

My Lab Diagram with 2 x FortiVM in a Single ESXi 6.7 host

Create a vSwitch without attached to Physical NIC with all the security policy set to YES

Configure VLAN ID for ports group to prevent VM to communicate with Layer 2 network

A. Configuration of 1st FortiVM

1. Backup the configuration and reset it to Factory Default

#Backup to TFTP Server
exec backup config tftp FG20.conf 192.168.1.248

#Reset to Factory Default
execute factoryreset

2. Assign hostname and change the default password for admin

config system global
    set alias "FG20"
    set hostname "FG20"
    set timezone 57 #Malaysia
     set admin-ssh-v1 enable #Enable SSHv1 if you are using FortiVM Trial version
end

config system admin
    edit admin  
        set password P@ssw0rd
    end

3. Configure the WAN Interface with static route point to VYOS Router
   Configure Port1 as WAN with static route

config system interface
    edit "port1"
        set mode static 
        set ip 192.168.20.254 255.255.255.0
        set allowaccess ping https ssh http fgfm
        set alias "WAN"
        set role wan
      end

config router static
    edit 1
        set gateway 192.168.20.1
        set device port1
    end

4. Create a new VXLAN called vxlan20

configure system vxlan
    edit vxlan20
        set interface port1
        set vni 1 #Must match on both FortiVM
        set remote-ip 192.168.10.254 #WAN IP of the remote FortiVM 
    end

5. Create a Switch Interface with Port3 and vxlan20 as switch members

config system switch-interface
    edit FG20-SW
        set vdom root
        set member port3 vxlan20
        set intra-switch-policy implicit
    end 

No Firewall Policy is required to allow traffic flow between VXLAN if configured as implicit mode

intra-switch-policy ?
implicit    Traffic between switch members is implicitly allowed.
explicit    Traffic between switch members must match firewall policies.

6. Assing an IP Address to FG20-SW

config system interface 
    edit FG20-SW
        set ip 172.10.10.2 255.255.255.0 
        set allowaccess ping https ssh http fgfm
    end

B. Configuration of 2nd FortiVM

1. Configure the 2nd unit of FortiVM with the following

config system global
    set alias "FG10"
    set hostname "FG10"
    set timezone 57 #Malaysia
     set admin-ssh-v1 enable #Enable SSHv1 if you are using FortiVM Trial version
end

config system admin
    edit admin  
        set password P@ssw0rd
    end

config system interface
    edit "port1"
        set mode static 
        set ip 192.168.10.254 255.255.255.0
        set allowaccess ping https ssh http fgfm
        set alias "WAN"
        set role wan
      end

config router static
    edit 1
        set gateway 192.168.10.1
        set device port1
    end

configure system vxlan
    edit vxlan10
        set interface port1
        set vni 1 #Must match on both FortiVM
        set remote-ip 192.168.20.254 #WAN IP of the remote FortiVM 
    end

config system switch-interface
    edit FG20-SW
        set vdom root
        set member port3 vxlan20
        set intra-switch-policy implicit
    end 

config system interface 
    edit FG20-SW
        set ip 172.10.10.2 255.255.255.0 
        set allowaccess ping https ssh http fgfm
    end

PC (172.10.10.100) behind FG10 should be able to ping to PC (172.20.20.200) behind FG20 via VXLAN configured now

Some commands to diagnose the VXLAN

FG20 # diagnose sys vxlan fdb list vxlan20
mac=00:00:00:00:00:00 state=0x0082 flags=0x00 remote_ip=192.168.10.254 port=4789 vni=1 ifindex=3
mac=00:0c:29:2c:a5:f7 state=0x0002 flags=0x00 remote_ip=192.168.10.254 port=4789 vni=1 ifindex=3
mac=00:0c:29:d6:8a:39 state=0x0002 flags=0x00 remote_ip=192.168.10.254 port=4789 vni=1 ifindex=3

total fdb num: 3

FG20 # diagnose sys vxlan fdb stat vxlan20
fdb_table_size=256 fdb_table_used=3 fdb_entry=3 fdb_max_depth=1 cleanup_idx=0 cleanup_timer=376

Appendix
1. Enable the DHCP in the FG20-SW

config system dhcp server
    edit 1
        set default-gateway 172.10.10.1
        set netmask 255.255.255.0
        set interface "FG20-SW"
        config ip-range
            edit 2
                set start-ip 172.10.10.3
                set end-ip 172.10.10.10
            next
        end
        set dns-server1 8.8.8.8
    next
end

2. To verify that the PC behind FG10 is able to obtain the IP Address from FG20 via VXLAN

FG20 # execute dhcp lease-list FG20-SW
FG20-SW
  IP                    MAC-Address             Hostname                VCI                     Expiry
  172.10.10.10          00:0c:29:2c:a5:f7       box             udhcp 1.22.1            Fri Nov 16 19:56:30 2018