OpnSense Multiple Virtual IP WAN Set-up

Mindwatering Incorporated

Author: Tripp W Black

Created: 12/05/2022 at 11:21 AM

 

Category:
Linux
Other

Task:
On vSphere, configure an OpnSense virtual firewall with multiple public static IPs.


Example Environment:
100.50.100.0 100.50.100.1 - 100.50.100.6 100.50.100.7
100.50.100.8 100.50.100.9 - 100.50.100.14 100.50.100.15
100.50.100.16 100.50.100.17 - 100.50.100.22 100.50.100.23
WAN Gateway IP: 100.50.100.8
WAN IPs: 100.50.100.9/29
Broadcast Address: 100.50.100.15
Subset Mask: 255.255.255.248
Total Number of Hosts: 8
Number of Usable Hosts: 6
Static/Virtual IPs: 100.50.100.9, 100.50.100.10, 100.50.100.11, 100.50.100.12, 100.50.100.13, 100.50.100.14
LAN10: 10.0.10.0/24
LAN11: 10.0.11.0/24
IP4 only, no IP6
DHCP 10.0.10.51 through 10.0.10.199

LAN Servers will map to the public static IPs:
10.0.10.10 = 100.50.100.10
10.0.10.11 = 100.50.100.11
10.0.10.12 = 100.50.100.12
10.0.10.13 = 100.50.100.13
10.0.10.14 = 100.50.100.14
10.0.10.15 = 100.50.100.15

Name of the firewall will be:
mwfwlan10.mydomain.net

SMTP Gateway:
100.50.100.18 mx.mydomain.net


Steps:
1. Download the current version of OpnSense.
Choose the VGA, DVD type/option to get an ISO image file.
e.g.
OPNsense 22.1.10_4-amd64 = OPNsense-21.1-OpenSSL-dvd-amd64.iso
OPNsense 22.7.9_4-amd64 = OPNsense-22.7-OpenSSL-dvd-amd64.iso

Upload to the appropriate compute ISO repository.


2a. In vSphere, create a new empty VM.
CPU: 2 cores, 1 socket
RAM: 2 GB minimum
DISK: 18 GB
NIC 1 (vmx0): Map to the public IP space/DMZ
NIC 2 (vmx1): Map to the appropriate VM Network or VLAN
NIC 3 (vmx2): Map to the appropriate VM Network or VLAN

After several DoS attacks, we have updated our minimum config for one of virtual customers:
CPU: 4 cores, one socket (always one socket)
RAM: 6 GB
DISK: 24 GB

It was able to handle a 48 hour surge of SYN floods, and DoS from 100 IPs.
The bandwidth used was <25%, the CPU load was between 10% and 40%, the memory used was still under 4 GB.

2b. Connect the VM to the ISO, and perform Installation.
VM --> Edit Settings --> Connect CDRom
--> Datastore ISO location
--> Connect at Power On

2c. Boot the VM
Open a web console to the firewall VM.

The boot will pause to allow an import of an existing configuration. Click to continue the boot.
The system will boot a read-only "live" mode. The login can be "root" or "installer".


3. Login with the "installer" login
The default password is "opnsense".

Keymap: default/English
Install: defaults: ZFS (disk), and UFS (swap)

Enter the new root password and confirm at its prompt.

Once disk is formatted and installation performed, at the completion notice, click Complete Install and reboot.


4. Set the Interfaces to the NICs, and set-up the Interface base settings
After the system boots, login as the root user with its new password.

We will be using the these menu options next:
1) Assign interfaces
2) Set interface IP address
...
Enter an option: _

a. Choose 1, to assign the Interfaces.
For vmx0, choose WAN
For vmx1, choose LAN1
For vmx2, choose OPT1
Note: we can rename later, as needed.


b. Choose 2, to set-up the LAN. This is generally faster from the GUI, so we'll do only the LAN on this menu.
(You can do all 3 here, though, if desired.)

Choose LAN.
Set IP to 10.0.10.1
Set /24
Enable DHCP on IP4
Enter the starting and ending DHCP range (e.g. 51 to 199)
Skip IP6 set-up

At this point, we can boot a worker VM that we've placed on the VM.
The rest of the set-up will be done from the browser on the LAN network w/a DHCP address from this LAN.


5. Set up the System properties/settings
Via web browser, login to the firewall at 10.0.10.1, with root and its password.

a. Update primary LAN settings:
Interfaces --> LAN
(vmx1)
- Lock - Prevent interface removal: check
- Description: LAN10
- Block bogon networks: check
- MTU: 9000
Click Save

b. Update the OPT1/LAN2 settings:
Interfaces --> OPT1
(vmx2)
- Enable - Enable Interface: check
- Lock - Prevent interface removal: check
- Description: LAN11
- Block bogon networks: check
- IPv4 Configuration Type: DHCP
- MTU: 9000
Click Save.

c. Add the gateway for the secondary internal LAN if not auto-detected/set-up:
System --> Gateways --> Single
Click "+" to add new one
- Name: LAN11_GW
- Description: The remote LAN11 gateway
- Interface: LAN2 (LAN11 renamed below)
- Address Family: IPv4
- IP Address: 10.0.11.1
- Upstream Gateway: check
- Far Gateway: check
- Disable Gateway Monitoring: check
Click Save.

d. Update the WAN settings:
Interfaces --> WAN
- Enable - Enable Interface: check
- Lock - Prevent interface removal: check
- Description: WAN
- Block private networks: check
- Block bogon networks: check
- IPv4 Configuration Type: Static IPv4
- MTU: 9000
- IPv4 address: 100.50.100.9
- IPv4 Upstream Gateway: WAN_GWv4
Click Save.

Note:
If the GW was not auto-added, follow step c above to add the WAN upstream gateway:
- Name: WAN_GWv4
- IP address: 100.50.100.8
- Upstream Gateway: check
(Note: Not far gateway)
- Monitor IP: <ISP monitoring IP>
Click Save.

e. Create new internal CA authority, and server certificate:
System --> Trust --> Authorities
Click "+" to add new one
- Descriptive Name: LANInternalCA
- Method: Create an internal Certificate Authority
- Key Type: RSA
- Key Length: 8192
- Digest Algorithm: SHA512
- Lifetime (days): 825
- Country Code: US
- State or Province: North Carolina
- Organization: MindwateringNet
- Email Address: myemailaddr@mydomain.net
- Common Name: laninternal-ca
Click Save to create.

Notes:
This ca certificate can be exported and used across multiple Opnsense firewalls.
If you have your own internal CA, you should import it by changing Method field to the Import and Existing Certificate Authority option.

f. After importing or creating the internal-ca, create the new server certificate:
System --> Trust --> Certificates
Click "+" to add new one
- Descriptive Name: LAN10FWSite
- Certificate Authority: LANInternalCA
- Type: Server Certificate
- Key Type: RSA
- Key Length: 8192
- Digest Algorithm: SHA512
- Lifetime (days): 397
- Private key location: Save on this firewall
- Country Code: US
- State or Province: North Carolina
- Organization: MindwateringNet
- Email Address: myemailaddr@mydomain.net
- Common Name: mwfwlan10
- Alternate Names:
-- Type: DNS, Value: mwfwlan10.mydomain.net
-- Type: DNS, Value: mwfwlan10.mydomain.local
-- Type: IP, Value: 100.50.100.9
Click Save to create.

e. Update administrative settings:
System --> Settings --> Administration
SSL Certificate: LAN10FWSite
Listen Interfaces: LAN10 / LAN 11
(Uncheck the WAN interface, so we can only administer this firewall internally.)
Click Save.

f. Update administrative miscellaneous settings for VM optimizations:
System --> Settings --> Miscellaneous
Hardware acceleration: None
Hardware: None/ACPI
/var RAM Disk - Use memory file system for /var: check
/tmp RAM Disk - Use memory file system for /tmp: check
Click Save.

Notes:
If you want less read/writes, increase the VM memory and move /tmp and /var to memory disks.

g. Update the name and location info for the firewall:
System --> Settings --> General
Hostname: mwfwlan10
Domain: mydomain.net
Time zone: America/New York
DNS Servers: <enter appropriate for your install>
e.g.
DNS Server - Use gateway
123.12.23.12 - WAN_GWv4
Click Save.

5. Perform Updates and Install the VMTools plug-in.
a. Initiate updates, first.
Notes:
Besides updating the system, it will also load the list of plugins, which starts empty.
Also, the VM Tools plugin will fail to install, if the Updates are not COMPLETELY up-to-date. So you may have to click Check for updates more than once.

System --> Firmware --> Status
Click Check for updates
Click OK on the info dialog, and accept the updates
<wait>

Once complete repeat, until there are no updates.

b. Install the VM-Tools plugin:
System --> Firmware --> Plugins
Click the + on the os-vmware plugin, and install.

c. Reboot the firewall appliance:
Power --> Reboot
<wait the 20 seconds or so for the reboot>


6. Set-up the Virtual IPs and networking:
We have 5 public IPs to add as Virtual IPs. (1 IP is use by the WAN.)

a. Create the Virtual IPs:
Interfaces --> Settings
Click "+" to add new one
- Mode: IP Alias
- Interface: WAN
- Type: Single Address
- Address: 100.50.100.10 /32
- Allow service binding: check
- Gateway: <leave empty>
- Description: Static10
Click Save.

Repeat above for .11 through .15

b. Set the general firewall settings:
Firewall --> Settings --> Advanced
Allow IPv6: uncheck
NAT - Reflection for port forwards: check
NAT - Reflection for 1:1 <leave unchecked>
NAT - Automatic outbound NAT for Reflection: check
Anti DDOS - Enable syncookies: always
Click Save.

c. Set the firewall's Outbound NAT Virtual IPs:
Update the mode to hybrid.
Firewall --> NAT --> Outbound
Mode: Hybrid outbound NAT rule generation (radio selected)
Click Save.

Return to the Outbound view and add the required entries:
Firewall --> NAT --> Outbound
Click "+" to add new one
- Interface: WAN
- TCP/IP Version: IPv4
- Protocol: any
- Source address:
-- Single host or Network
-- 10.0.10.10 /32
- Source port: any
- Destination address: any
- Destination port: any
- Translation / target: 100.50.100.10 (Static10)
- Description: OutStatic10
Click Save.

Repeat above for .11 through .15

d. Set the firewall's Incoming NAT Port Forwards:
Notes:
Update the following ports being opened to your specific app's ports needed. In this example, we will do several common ones.
Upon saving each entry, the firewall will update the "Filter rule association" with a rule it creates. (e.g. Rule Static10HTTPS).
For the SMTP example below, we restrict SMTP mail to our mail filter/gateway: 100.50.100.18. We could have allowed any (anywhere) here, and then under rules, restricted incoming SMTP mail to just the gateway instead.

Firewall --> NAT --> Port-Forward
Click "+" to add the first new one for HTTPS (443)
- Interface: WAN
- TCP/IP Version: IPv4
- Protocol: any
- Destination: 100.50.100.10 (Static10)
- Destination port range:
-- from: HTTPS
-- to: HTTPS
- Redirect target IP:
-- Single host or Network
-- IP: 10.0.10.10
- Redirect target port: HTTPS
- NAT reflection: Use system default
- Description: Static10HTTPS
Click Save.

Click "+" to add second new one for HTTP (80)
- Interface: WAN
- TCP/IP Version: IPv4
- Protocol: any
- Destination: 100.50.100.10 (Static10)
- Destination port range:
-- from: HTTP
-- to: HTTP
- Redirect target IP:
-- Single host or Network
-- IP: 10.0.10.10
- Redirect target port: HTTP
- NAT reflection: Use system default
- Description: Static10HTTP
Click Save.

Click "+" to add third new one for NRPC (1352)
- Interface: WAN
- TCP/IP Version: IPv4
- Protocol: any
- Destination: 100.50.100.10 (Static10)
- Destination port range:
-- from: Other 1352
-- to: Other 1352
- Redirect target IP:
-- Single host or Network
-- IP: 10.0.10.10
- Redirect target port: Other 1352
- NAT reflection: Use system default
- Description: Static10NRPC
Click Save.

Click "+" to add fourth new one for IMAP/S (995)
- Interface: WAN
- TCP/IP Version: IPv4
- Protocol: any
- Destination: 100.50.100.10 (Static10)
- Destination port range:
-- from: IMAP/S
-- to: IMAP/S
- Redirect target IP:
-- Single host or Network
-- IP: 10.0.10.10
- Redirect target port: IMAP/S
- NAT reflection: Use system default
- Description: Static10IMAPS
Click Save.

Click "+" to add fourth new one for SMTP (25)
- Interface: WAN
- TCP/IP Version: IPv4
- Protocol: any
- Source:
-- Single host or Network
-- 100.50.100.18 / 32
- Destination: 100.50.100.10 (Static10)
- Destination port range:
-- from: SMTP
-- to: SMTP
- Redirect target IP:
-- Single host or Network
-- IP: 10.0.10.10
- Redirect target port: SMTP
- NAT reflection: Use system default
- Description: Static10SMTP
Click Save.

Repeat with specific ports and app services for IPs .11 through .15


7. Update the firewall Rules for our NAT additions:
We need to add the firewall rules to allow the traffic for the Firewall NAT set-up we have. With the auto rule set-up, we expected to not have to do this.
With version 22.1, sometimes the automatic rules get created automatically, and sometimes they do not.

If you have auto-created rules already in this folder, and your sites are working, you don't need need to create the entries below.

a. Update the LAN10 to LAN11 traffic:
We need to update both the incoming and outgoing for both building LANs to allow traffic between the LANs on this firewall appliance.

Firewall --> Rules --> LAN10
Click "+" to add a new firewall rule
- Quick - Apply the action immediately on match: check
- Interface: LAN10
- Direction: in
- TCP/IP Version: IPv4
- Protocol: any
- Source: LAN10 net
- Destination: LAN11 net
- Destination port range:
-- from: any
-- to: any
- Description: LAN10 Incoming
- Gateway: LAN11_GW
Click Save.

Click "+" to add a new firewall rule
- Quick - Apply the action immediately on match: check
- Interface: LAN10
- Direction: out
- TCP/IP Version: IPv4
- Protocol: any
- Source: LAN10 net
- Destination: LAN11 net
- Destination port range:
-- from: any
-- to: any
- Description: LAN10 Outgoing
- Gateway: LAN11_GW
Click Save.

b. Update the LAN11 to LAN10 traffic:
Continue updating incoming and outgoing for both building LANs to allow traffic between the LANs on this firewall appliance.

Firewall --> Rules --> LAN11
Click "+" to add a new firewall rule
- Quick - Apply the action immediately on match: check
- Interface: LAN11
- Direction: in
- TCP/IP Version: IPv4
- Protocol: any
- Source: LAN11 net
- Destination: LAN10 net
- Destination port range:
-- from: any
-- to: any
- Description: LAN11 Incoming
- Gateway: <empty for default>
Click Save.

Click "+" to add a new firewall rule
- Quick - Apply the action immediately on match: check
- Interface: LAN11
- Direction: out
- TCP/IP Version: IPv4
- Protocol: any
- Source: LAN11 net
- Destination: LAN10 net
- Destination port range:
-- from: any
-- to: any
- Description: LAN11 Outgoing
- Gateway: <empty for default>
Click Save.


c. Update the WAN rules to allow incoming ports desired.
These work with the NAT --> Port Forward settings. We originally thought that the Port-Forward entries would create these WAN rules, but it was not the case. We we create "them" twice, once for the NAT at the Interface, and again under Firewall Rules.

Firewall --> Rules --> WAN
Click "+" to add a new firewall rule
- Action: Pass
- Disabled: <unchecked>
- Quick - Apply the action immediately on match: check
- Interface: WAN
- Direction: in
- TCP/IP Version: IPv4
- Protocol: TCP
- Source: any
- Destination:
-- Single host or Network
-- 10.0.10.10 /32
- Destination port range:
-- from: HTTPS
-- to: HTTPS
- Description: Static10HTTPS
Click Save.

Firewall --> Rules --> WAN
Click "+" to add a new firewall rule
- Action: Pass
- Disabled: <unchecked>
- Quick - Apply the action immediately on match: check
- Interface: WAN
- Direction: in
- TCP/IP Version: IPv4
- Protocol: TCP
- Source: any
- Destination:
-- Single host or Network
-- 10.0.10.10 /32
- Destination port range:
-- from: HTTP
-- to: HTTP
- Description: Static10HTTP
Click Save.

Firewall --> Rules --> WAN
Click "+" to add a new firewall rule
- Action: Pass
- Disabled: <unchecked>
- Quick - Apply the action immediately on match: check
- Interface: WAN
- Direction: in
- TCP/IP Version: IPv4
- Protocol: TCP
- Source: any
- Destination:
-- Single host or Network
-- 10.0.10.10 /32
- Destination port range:
-- from: IMAP/S
-- to: IMAP/S
- Description: Static10IMAPS
Click Save.

Firewall --> Rules --> WAN
Click "+" to add a new firewall rule
- Action: Pass
- Disabled: <unchecked>
- Quick - Apply the action immediately on match: check
- Interface: WAN
- Direction: in
- TCP/IP Version: IPv4
- Protocol: TCP
- Source: any
- Destination:
-- Single host or Network
-- 10.0.10.10 /32
- Destination port range:
-- from: Other 1352
-- to: Other 1352
- Description: Static10NRPC
Click Save.

Firewall --> Rules --> WAN
Click "+" to add a new firewall rule
- Action: Pass
- Disabled: <unchecked>
- Quick - Apply the action immediately on match: check
- Interface: WAN
- Direction: in
- TCP/IP Version: IPv4
- Protocol: TCP
- Source:
-- Single host or Network
-- 100.50.100.18 / 32
- Destination:
-- Single host or Network
-- 10.0.10.10 /32
- Destination port range:
-- from: SMTP
-- to: SMTP
- Description: Static10SMTP
Click Save.

Repeat with specific ports and app services for IPs .11 through .15


8. Verify access, and not too much access.
At this point the firewall should be allowing "anything" out, and only certain ports/services into the six Virtual IPs.




previous page