Network | Breek Een Been

Nested Nsx Vmware Nsx on Intel Nuc Lab Setup Part 3

Mon, 29 Dec 2014 09:37:17 +0200

In the previous two posts I described how to set-up a nested ESXi environment on the Intel NUC and how to install and configure it for NSX. So in this post I assume everything is installed and configured and we can actually start deploying a network.

Logical Switch

If not already, go to “Network & Security” and click on “Logical Switches”, click the green plus sign, to add a Logical Switch. I used the following settings:

Name: App-Tier
Description: Application Tier
Zone: LAB-Zone

This is it, we have created a working logical switch, however to see if it actually can switch, we need to put some workload on it. To show that the communication within the Logical Switch is completely independent of the underlying network, I put the two VMs on this switch, within a completely different subnet than already used.

IP1: 172.20.20.2/24
IP2: 172.20.20.3/24

We are going to deploy two VMs I used Debian for this, but you are free to use whatever you want, as long as it can do IP. This is what I configured:

Name: App-VM1 / App-VM2
OS: Debian 7 - 64bit
vCPU: 1
MEM: 512MB Ram
IP: 172.20.20.2 / 172.20.20.3
Subnet: 255.255.255.0
GW: 172.20.20.1

Make sure you deploy a VM on ESX01 and one on ESX02, also connect them for now to the pgCompute1 and pgManagement port-group.

When both VMs are deployed, go to “Logical Switches” and click on “Add VM”, select the two VMs, select both the NICs and finish the task.

Configure the VMs with the appropriate network settings and do a ping test to each other. If everything went well, you should get a reply.

Before we continue it is nice to take a look at the controller, to see what information is stored here and get an idea how the actual traffic flow is working. You can log-in to the controller with SSH and username “admin” and the password specified earlier. With the following two commands you can list the VTEP interfaces and the MAC-addresses. The number 5000 is the segment ID (VNI = Virtual Network Identifier) assigned to the logical switch. Which can be found in the webinterface under Logical Switches.

nvp-controller # show control-cluster logical-switches vtep-table 5000
VNI      IP              Segment         MAC               Connection-ID
5000     192.168.5.11    192.168.5.0     00:50:56:62:a3:86 2
5000     192.168.5.10    192.168.5.0     00:50:56:66:dd:df 7

nvp-controller # show control-cluster logical-switches mac-table 5000
VNI      MAC               VTEP-IP         Connection-ID
5000     00:50:56:99:d0:77 192.168.5.11    2
5000     00:50:56:99:6b:d9 192.168.5.10    7

In my case the MAC addresses of the VM’s are.

App-VM1: 00:50:56:99:6B:D9
App-VM2: 00:50:56:99:D0:77

You could also use the following command, to see the ARP table, remember this like normal switches has a time-out, so when it is empty initiate a ping or some other traffic.

nvp-controller # show control-cluster logical-switches arp-table 5000
VNI      IP              MAC               Connection-ID
5000     172.20.20.3     00:50:56:99:d0:77 2
5000     172.20.20.2     00:50:56:99:6b:d9 7

As you can see in the mac-table, there is a relationship between the MAC address and the VTEP-IP. What it actually says is that if App-VM1 sends something to App-VM2, the traffic will be encapsulated (VXLAN-header), with the destination IP (192.168.5.11) of the VTEP connected to the host running the VM. After the host received the traffic, it is being decapsulated and sent to the VM, so the VM’s don’t know anything about how the traffic is handled and won’t see any difference between a virtualized or a traditional network.

Distributed Router

Another neat feature of NSX is the distributed router, this not only means you can route traffic, but the neat thing about it is, that this can be done on “host”-level. This means that when two VMs are running on the same host, but in different subnets, the distributed router (remember installed VIB) will route the traffic in the kernel, so the traffic won’t have to leave the host! To create a distributed router, go to “Network & Security” and click on “NSX Edges”. (If you ask me, they should make two categories, “NSX Edges” & “Logical Routing”, this would make a in my opinion lot more sense) Click the green plus sign and select logical distributed router.

Next specify a password and Enable SSH acces.

Specify the “Management_and_Edge” cluster.

Connect the management interface to the “pgManagement” port-group and specify an IP, in my case 10.10.11.126.

Add an interface and connect it to the “Logical Switch” - “App-Tier” and specify an interface, in this case 172.20.20.1 [!

When done with step 4 it should look as follow.

We can skip the HA configuration and continue, when satisfied with the settings, click Finish.

The first thing we want to do is enable routing between two logical switches. Therefore go to Logical Switches and create a second one, named: “Web-Tier”.

Deploy (or clone) a new VM and connect it to this “Logical Switch”.

Name: Web-VM1
OS: Debian 7 - 64bit
vCPU: 1
MEM: 512MB Ram
IP: 172.20.10.2
Subnet: 255.255.255.0
GW: 172.20.10.1
Cluster: Compute

Since we didn’t connect the distributed router to the Web-Tier Logical Switch, there is no gateway and no communication possible between the App- & the Web-Tier. The App-Tier is connected and you can try ping the gateway 172.20.20.1 from one of the App-VMs, this should work. Go back to NSX Edges en open the distributed router we created, select the Manage tab, click on Settings and select interfaces. Now click the green plus sign and add the Web-Tier and create an interface with IP 172.20.10.1.

Before you click OK, open the console of the Web-VM1 and start a ping to one of the App-VMs, now click OK. Immediately after the configuration, you should get a response on the pings.

It is important to know that the logical router instance (VM) is in this example purely for configuration of the distributed router and that traffic won’t go through it. When sending traffic to App-VM1 which is on the same host, the traffic won’t leave the host, but is routed at the kernel. If traffic is send to App-VM2, the traffic is routed on the host of the Web-VM1 and send (on the App-Tier Logical Switch) to the other host, like it was L2 switching. The last part of this post is bringing data outside the virtual world, there are two ways of accomplish this. The first one is bridging and the second one is routing. I won’t go further into bridging, but what it simply said does is, connecting a VXLAN to a VLAN.

Edge Services Gateway

Best practice is that we use an Edge Services Gateway (ESG) for connection with the outside world. This means that the Logical Router will forward the traffic to the ESG and the ESG will route the traffic to the physical world. This is just one of the many functions the ESG can do, it can also NAT, Loadbalancing, basic firewalling, etc. VMware likes to call it a “swiss army knife” due the roles it can fulfill. Between the Logical Router and the ESG we will have a so called transit network, for this create a new Logical Switch and name it Transit.

My uplink is in a different VLAN, so I created a new portgroup pgUplink, but if your network is flat or you want to (mis)use the management port-group, this is no problem. Make sure, when you create an additional port-group, do this at the Management and Edge (cluster) Distributed Switch.

Go back to the NSX Edges and click the green plus sign, similar to the deployment of a Logical Router, however now we choose Edge Services Gateway.

Fill in the credentials and select the Management and Edge Cluster for deployment. A Compact deployment for this LAB is sufficient.

Now the most important part, create the interfaces. Chooste “internally” for the Transit. I will use 172.16.10.0/24 as transit subnet and the Edge will get .1 and the Logical Router (later on) .2. In the Uplink subnet I have 10.0.0.0/24 and will use 10.0.0.1 for the Edge. My physical router has 10.0.0.254, so this will become my default gateway.

Leave all the further settings default and finish the wizard.

Now open the just deployed ESG and go Manage, Routing, (for now) we will create static routes to the Logical Router we created earlier. Add the following two routes for the App and Web tier, by clicking on the green plus sign.

172.20.10.0/24 - 172.16.10.2
172.20.20.0/24 - 172.16.10.2

If you added both routes, click on Publish Changes.

The next step is to add the 172.16.10.2 interface to the Logical Router, and set the default gateway for the Logical Router to the ESG. Open up the settings page for the Logical Router and add the interface.

Go to routing and set the default gateway under Global Configuration to the ESG, make sure to Publish Changes when you are done.

Now almost everything is set, however when you will test to communicate with the outside world, you will see that it will fail, there are two more (small) steps to take. First, make sure the outside world, your router or cliënt has a route to the new Web and App subnets. I first tested it of my desktop client, which has an IP in 10.0.0.0/24. I added a specific route to the WebVM.

172.20.10.2/32 - 10.0.0.1

Now the very last step is to check the firewall of the ESG, by default it only has a rule which accept traffic, which has a source VSE (I assume this stands for vShield Edge). For this lab we will add a rule, which allows any traffic. Go to the ESG and click on firewalling and click the green plus sign. It will by default create a rule which accepts any, any, any :). Don’t forget to publish the new rule.

Now you can test and everything should work! Now we have set-up a very basic and simple lab environment which runs NSX pretty well. Even despite the limited resources, the Intel NUC has proven itself to be capable of handling the load and give a decent lab experience. I think this lab is a very good starting point for further testing and learning with NSX and probably some more posts will follow concerning NSX configuration, for me it will hopefully help to eventually get my VCIX-NV. I hope you had fun and please let me know if you have any further questions, ideas or suggestions.

Nested Nsx Vmware Nsx on Intel Nuc Lab Setup Part 2

Sat, 27 Dec 2014 09:20:49 +0200

In my previous post I described how to create a nested ESXi environment, connected to a vCenter, this as a preparation to run eventually NSX. In this post we will build further on the basic set-up we created in part 1 of this series. At the end of this post we will have the NSX installed and the network prepared for all the cool things NSX can do.

Distributed Switch

Since the NSX vSwitch is based on the “distributed switch”, this is what we are going to create first on the two clusters. We will make two distributed switches, named;

dsMgmtEdge
dsCompute1

Go to your (LAB) vCenter and go to the networking tab, right click on the Datacenter and click on “New Ditributed Switch”, for now the following options are sufficient.

Number of uplinks: 1
Network I/O control: Enabled
Default port group: Create a default port group
Port group name: pgManagement / pgCompute1

Before we can actually add hosts to the distributed switches we created, we need to add the VLAN tag to the newly created port-groups. Therefore select the distributed switch, choose the manage tab and make sure settings is selected. Select the port-group and click edit settings. Under the VLAN settings, choose VLAN and enter the appropriate VLAN ID. (if you don’t work with VLANs you can skip this step)

Now we have created the distributed switch and configured the port-group, we need to add the hosts to it. Right click on the distributed switch and click on “Add and Manage Hosts”, we want to add the host, select the appropriate host.

Make sure both the “Manage physical adapters” and “Manage VMkernel adapters” are both checked. We will move the vmnic0 to the uplink of the distributed switch.

Also move the VMkernel adater (vmk0) to the new port group.

If you screw this up, it is pretty easy to start over by accessing the console of the ESXi and in the settings, choose “Network Restore Options”.

NSX Manager

Finally we can start with some real NSX stuff. The first step is the installation of the NSX Manager. Go to hosts and clusters, right click the “Management & Edge” cluster and choose deploy OVF. Point to the NSX Manager OVA. Select Accept extra configuration options.

Read & accept the EULA, pick a name (i.e. LAB NSX Manager). At the Network setup make sure you choose the new “pgManagement” port-group. At the “Customize template”, set some passwords and make the network configuration.

VSMgmt: pgManagement
Hostname: LABNSXManager
Network 1 IPv4: 10.10.11.111
Network 1 Netmask: 255.255.255.0
Default IPv4 Gateway: 10.10.11.1
DNS server list: 10.10.11.3, 8.8.8.8
Enable SSH: checked

The settings I didn’t mentioned I left blank.

Click finish and have a bit of patience. Before we can power up the NSX manager we need to do something about the resources which are by default quite heavy and we simply have not that many resources available on our LAB set-up. Edit the VM hardware settings with the following values.

CPU: 2 (instead of 4)
Memory: 4GB (instead of 12)
Memory reservation: 0 (instead of 3GB)

Power up the NSX manager and if everything went well, you will be able to visit the NSX page with a few minutes, https://IPAddress and you can log in with the username “admin” and the password specified during the deployment.

Before we continue, I noticed that the resources are a bit short, cause after 17 minutes, the NSX Mangement Services wasn’t started and it would eventually turn out to take 19 minutes. As you can see the Memory is in full use and also on the ESXi host we can see this, this is an indication that we need to increase the memory of the ESX02 (the host of the Management & Edge) cluster. Cause it is also going to run the NSX Controller(s). Luckily this is a virtual ESX host, so extending the memory is quite easy, I doubled it at this moment to 8GB. I didn’t increase the memory of the NSX Manager, since this is a LAB, I don’t care if the start-up takes a while, as long it will run fluent enough to do some testing.

vCenter integration

NSX has a tight integration with vCenter, at this moment it is an 1 on 1 relationship, which means for every vCenter, you need one NSX Manager. Log in to your just deployed NSX Manager and click on “Manage vCenter registration”, click on edit and enter your vCenter information.

After clicking ok, trust the certificate and within a few seconds it should say that is is successfully connected.

Now we are going back to our vCenter, if you are already logged in, log out and log in again. The first time it can take a few minutes, cause it will configure itself for NSX. If everything went fine you should see a new option in the menu on the left, named “Networking & Security”.

Prepare the hosts

The next step is to prepare the host, NSX will install a couple of VIB’s into the hosts, knowing;

Logical Routing
Distributed Firewall
VXLAN

Click on the new “Networking & Security” option and go to “installation”, click the “host preparation” tab. It will show the clusters available within the vCenter.

Click the install link in the column “Installation Status”, do this for both clusters. After the installation it should say “ready” on the particular host and the version for the cluster.

Controller

Now that the hosts are ready we can start deploying the controller(s), normally you would deploy at least 3 controllers, but in this LAB we will start with just one, mainly due resource restrictions. The controller is the control plane of the environment and will keep three primary tables.

ARP table
MAC table
VTEP table

Go back to the “Management” tab of the “Installation” settings and click on the green plus sign, below NSX Controller Nodes. Fill in the following.

NSX Manager: 10.10.11.111
Datacenter: Datacenter
Cluster: Management_and_Edge
Datastore: NAS
Host: 10.10.11.102
Connected to: pgManagement (select distributed switch)
IP Pool: ClusterIPPool - Create one
- Name: ClusterIPPool
- Gateway: 10.10.11.1
- Prefix Length: 24
- Primary DNS: 10.10.11.3 -> Your own DNS server
- Secondary DNS: 8.8.8.8
- Static IP Pool: 10.10.11.120 - 10.10.11.125

Password: Pick Something

Click OK, it will start deploying a controller and this can take a while.

VXLAN

The last step in the preparation of the NSX environment is the VXLAN preparation, VXLAN is the tunnel protocol which makes sure every host participating in the NSX environment is able to communicate with each other. First we going to create an IP pool, with the IP adresses for the VMKernel adapters serving the VTEP (VXLan Tunnel End Point). Therefore click in the left bar on NSX Managers and select “10.10.11.111”, make sure the “Manage” tab is selected and click on “Grouping Objects”, select IP-Pools and click on the green plus sign. To show that communication between the hosts is really independent of the underlying network, I created a separate subnet without a default gateway (although specified).

Name: VXLAN-VTEP
Gateway: 192.168.5.1
Prefix Length: 24
Static IP Pool: 192.168.5.10-192.168.5.20

Now go back to the installation, “Host Preparation” tab and in the column VXLAN click “configure”.

Leave everything default except:

VLAN: 5
IP Addressing: Use IP Pool
IP Pool: VXLAN-VTEP

Click ok and wait a few seconds, although it could give an error (didn’t find out why yet), the error will dissappear after a refresh.

If you go to the distributed switch of one of the cluster you should see something as follows:

Now we need to specify the VXLAN segment ID’s, therefore go to the “Logical Network Preparation” tab in the installation settings, select “Segment ID” and click edit. Fill in the following segment ID pool, which should be more than enough for this lab environment.

Segment ID Pool: 5000 - 5999

We don’t have to enable multicast, since we are running on ESXi 5.5 hosts.

The last part is to create a “Transport Zone”, this will tell NSX which clusters are able to communicate with each other. Click on “Transport Zones” and click the green plus sign.

Name: LAB-Zone
Control Plane Mode: Unicast
Clusters: Select both

Now we have everything prepared to do some actual networking with NSX. This is what we will be doing in my next blog post.

Nested Nsx Vmware Nsx on Intel Nuc Lab Setup Part 1

Wed, 17 Dec 2014 08:52:50 +0200

While the VMware Hands-on Labs are extremely cool and useful, I found it always very learning-full and nonetheless fun to build my own labs. To keep it fun and keep down the noise and powering costs I decided to buy some Intel NUCs, which have a good price tag and are very low on energy. Last couple of months I’m very interested in VMware NSX and it is time to try-out running this in my own Lab and on one Intel NUC, which means not much resources but I’m wondering if it can be done and hopefully work well enough to play around. In the next coming posts I will show you the steps I take to get things working, so you can follow me on this journey. This first part will concentrate on how to get the nested environment up & running. One of the biggest downsides of the Intel NUC is that it only has one NIC, which makes it difficult to run all needed parts and another caveat is that chances are that you will lock-out yourself. Luckily ESXi can run itself, we call this a nested environment. Below you see a very abstract picture about the set-up we are going to create. As you can see I choose to let the nested ESXi servers interact directly with my NAS (Synology), mainly to save resources and time. If you want a complete nested and virtualized environment including storage you are of course free to do so.

Nested NSX abstract

The blue components are the more physical parts of this environment and the actual lab is more located at the green components. I assume you have already running ESXi on your Intel NUC (or other system), if not below two good links to get you started.

When you have ESXi running on your NUC, we can create a nested environment on it. I will start small, cause I have no idea, how the NUC will handle the load and it will be easy to add another node in the feature (also a good test case). How to create nested ESXi servers isn’t new, but I will cover shortly how I’ve done it. A small note lots of older blogpost write about the vhv.enable feature, this isn’t necessary in ESXi 5.5 cause it will be set to true by default.

Network

This has nothing to do with NSX yet, but it will enable our nested hosts to communicate with the outside world. We are going to create a TRUNK port-group, so that we can use in the future different VLANs to segment the traffic, this however requieres that the connected physical switch also supports VLANs. Don’t worry if not, than this all will probably still work, however I didn’t tested it. Go to the host, networking configuration and add a port-group, with the following settings.

VLAN ID: ALL (4095)
Security - Promiscuous mode: Override Accept
Security - MAC address changes: Override Accept (optional)
Security - Forged transmits: Override Accept

Nested ESXi servers

Deploy a new VM, I have my ESXi connected to a vCenter environment, this is not the vCenter we will discuss later and is running on another NUC. So I will use the web interface. The steps taken can also be done on the traditional GUI, the only difference will be the HW version, 10 for the WUI and 8 for the GUI. Use the following settings:

Pick a VM name, for the sake of simplicity I use “ESX01” and “ESX02”
OS Family: Other
OS Version: Other (64 bit)
CPU: 4 (I started of with 2, but quickly changed to 4, needed for some componentens (i.e. Controllers))
CPU - Harware virtualization: Enable
MEM: 4 GB (enough to get started) (In part 2 I increased the memory of the ESX02 to 8GB)
HD: 1GB
Network: Trunk port-group (1 nic is enough to get started)
CD: Mount the ESXi installer

Now we created the VM, we can boot it up and start the installation, which is really a next, next finish job. When the installation is done, this can take a several minutes we start configuring the host. Open the console and press F2 and after entering your password go to “Configuring Management Network”. If you have a trunk port, don’t forget to enter the VLAN ID. Give the host an IP address, subnet, gateway and optionally for now some DNS settings. For me the IP’s are.

ESX01 IP: 10.10.11.101/24
ESX02 IP: 10.10.11.102/24
Gateway: 10.10.11.1

Since this is a lab environment I will enable the SSH console, you can do this under “Troubleshooting Options”.

Before you exit the console test if you can reach the host, a simple ping test would be sufficient. The last thing we will need to do for now on the new hosts is adding the storage. For me this is a simple NFS share on mine Synology. Go to the host, storage configuration and add the NFS storage.

Before continuing, make sure both hosts are deployed, have storage and are reachable.

vCenter

The last part of this post is about deploying the vCenter appliance, which is sufficient, for running NSX. Deploy the vCenter 5.5 OVF, I used the following settings.

Name: LAB-vCenter
IP: 10.10.11.110/24
GW: 10.10.11.1

(Since the last Chrome update, the vCenter plugin is broken, didn’t fixed it yet)

After the OVF is deployed I decreased the memory to 4GB, which is in my experience enough for a fluent experience in a LAB environment. Start the vCenter VM, as you can see on the console, you should be able to access the initial configuration on https://IPAddress:5480 and login with the default username “root” and password “vmware”.

After logging in, accept the license (of course first read it). I choose for “configure with default settings”, after clicking next and start, it will configure itself, but this will take a while. When the configuration is done, the last change I make on the configuration is that the password of the admin will not expire.

Now we are done we can actuale access the vCenter interface by going to https://IPaddress:9443/vsphere-client

We don’t worry about license keys for now, so you can simply ignore this message or close it by clicking the cross sign on the right. Go to vCenter and click on hosts and clusters. Now we start by creating a new Datacenter. Click on “Create datacenter” and if you want give it a name, you can also leave it default “Datacenter " and click ok. Now we are going to create two clusters, called;

Management_and_Edge
Compute

Click on create cluster and give it the desired name, we can leave HA and DRS off. You can create the second cluster by right clicking on the Datacenter. When both clusters are created we need to add the ESX hosts to the appropriate cluster. We add ESX02 to the Managment & Edge cluster and the ESX01 to the Compute cluster.

Right click on the cluster and select “Add Host”. Fill in the IP of the host, on the next page the username and password of that specific Host, leave the other settings default.

Do this for both hosts, when you have done this, your environment should look as follow.

For now we are done, we have setup a basic nested environment on which we will go install and configure NSX. When you are bored and can’t wait for the next post or just want to optimize your environment, consider the following actions, these are all optional and not necessary to install NSX, also these steps can be taken on a later time.

Optional but nice!

Mikrotik VLAN Switching Without Bridging

Thu, 11 Dec 2014 19:41:18 +0200

One of the greatest networking vendors for homelabs is in my opinion Mikrotik, they offer great (often enterprise) features for a very compelling price. On the other hand it can be a bit daunting to configure and the firmware releases aren’t always equally stable. Last week I upgraded in my homelab one of two RB2011UiAS’s, which are I think the most popular Routerboards, for a CRS125, cause I was in need of more 1 gigabit ports. Previously I did all my VLAN configuring using bridging, which works, but there is a faster way, by using the internal switch chip, instead of the CPU. So this upgrade was the perfect case for me, to change this. Because this can be a bit hard in understanding and most tutorials on the web are about bridging. I decided to do a little write-up, about how this can be done without bridging. I will take it even a bit further and making sure we have a management address configured on the switch and we can use the wireless network. Before we actually start, a bit of understanding how the internals work, it is based on this site, but I will try to clarify it even a bit more.

A small disclaimer; the workings are described as far as my understanding goes, if you find any mistakes, please let me know!

In the below picture you find a simple logic representation of the inner workings.

Mikrotik inner workings

The physical switch ports are connected to the switch chip and the switch chip has also a connection to the CPU. The CPU is where all the clever things happen, think about, routing, bridging, nat(-ting?), etc. So when we created simple interfaces for bridging the flow will go like the picture below.

Mikrotik bridging

There is nothing wrong with this and gives us great flexibility, but when you only need layer 2 switching this generates a lot of unnecessary pressure on the CPU. So this can be done more efficient by connecting the right interfaces on the switch chip, this way the traffic won’t need to pass the CPU, which is far more efficient. We can talk about “wire speed” here.

Mikrotik hardware switching

In the picture above interface ether01, ether02 and ether03 are connected with each other and ether01 is a trunk (=multi-vlan port), probably an uplink and ether02 and 03 could be access ports which go to a vlan unaware device a PC or printer for example. This is the basic idea what we are going to set-up, although we make it a bit more complex. We want a management interface which must also be routable (i.e. for NTP and updates) and we want to connect the wireless. So let’s say for example we have 4 vlans.

Servers – vlan 10
Clients – vlan 20
Wireless – vlan 30
Management – vlan 40

The interfaced are used as follow:

ether01 – Uplink / trunk
ether02 – trunk (vlan 40, vlan 10) (i.e. ESX)
ether03 – access port vlan 10 (i.e. NAS)
ether04 – access port vlan 20 (i.e. desktop)

What we would like to accomplish would look like this, assuming that we only need layer 2 switching for the vlans and that only the management vlan is routable.

Mikrotik mixed setup

Be aware that in this set-up devices in different vlans aren’t able to communicate with each other! Therefore you need a router or make a routable interface, like the management one.

The first step is to connect the interfaces to each other, this can be done by choosing a master-port and make the other interfaces slaves of it, in this example we choose to make “ether01” the master port.

[admin@MikroTik] > interface ethernet
[admin@MikroTik] /interface ethernet>set numbers=1,2,3 master-port=ether01

[admin@MikroTik] > interface ethernet print

Flags: X - disabled, R - running, S - slave
 #    NAME    MTU  MAC-ADDRESS       ARP        MASTER-PORT       SWITCH
 0 R  ether01 1500 D4:CA:6D:CE:3C:20 enabled    none              switch1
 1  S ether02 1500 D4:CA:6D:CE:3C:21 enabled    ether01           switch1
 2 RS ether03 1500 D4:CA:6D:CE:3C:22 enabled    ether01           switch1
 3  S ether04 1500 D4:CA:6D:CE:3C:23 enabled    ether01           switch1

After this we can start with the vlan configuration, which also consists of a few steps and is done differently than you see at more regular switch vendors (i.e. Cisco, HP). We start deciding which vlans are needed on which port, important to see here is that switch1-cpu is actually a port and will forward the traffic to the CPU. So the management and the wireless vlan are needed on the switch1-cpu port.

[admin@MikroTik] > interface ethernet switch vlan
[admin@MikroTik] /interface ethernet switch vlan> add ports=ether01,ether02,ether03 vlan-id=10
[admin@MikroTik] /interface ethernet switch vlan> add ports=ether01,ether04 vlan-id=20
[admin@MikroTik] /interface ethernet switch vlan> add ports=ether01,switch1-cpu vlan-id=30
[admin@MikroTik] /interface ethernet switch vlan> add ports=ether01,ether03,switch1-cpu vlan-id=40

Now we need to decide which ports are carrying “tagged” vlan traffic and which ports are receiving “untagged” traffic. In this example “ether03” and “ether04” will receive untagged traffic. To make this traffic go to the right ports we need to “tag” this traffic when we receive (ingress) it.

[admin@MikroTik] > interface ethernet switch ingress-vlan-translation
[admin@MikroTik] /interface ethernet switch ingress-vlan-translation> add ports=ether03 new-customer-vid=10
[admin@MikroTik] /interface ethernet switch ingress-vlan-translation> add ports=ether04 new-customer-vid=20
[admin@MikroTik] /interface ethernet switch ingress-vlan-translation> print
 0   ports=ether03 service-vlan-format=any customer-vlan-format=any new-customer-vid=10 pcp-propagation=no sa-learning=no
 1   ports=ether04 service-vlan-format=any customer-vlan-format=any new-customer-vid=20 pcp-propagation=no sa-learning=no

Now all the traffic that is coming in the switch is “tagged” correctly, we can decide where the “tagged” traffic may go (egress). We also need to specify the “switch1-cpu” port here, cause for the “switch chip” it is just a port, where traffic can go.

[admin@MikroTik] > interface ethernet switch egress-vlan-tag
[admin@MikroTik] /interface ethernet switch egress-vlan-tag> add vlan-id=10 tagged-ports=ether01,ether02
[admin@MikroTik] /interface ethernet switch egress-vlan-tag> add vlan-id=20 tagged-ports=ether01
[admin@MikroTik] /interface ethernet switch egress-vlan-tag> add vlan-id=30 tagged-ports=ether01,switch1-cpu
[admin@MikroTik] /interface ethernet switch egress-vlan-tag> add vlan-id=40 tagged-ports=ether01,ether02,switch1-cpu

[admin@MikroTik] /interface ethernet switch egress-vlan-tag> print
Flags: X - disabled, I - invalid, D - dynamic
 #   VLAN-ID TAGGED-PORTS
 0        10 ether01
             ether02
 1        20 ether01
 2        30 ether01
             switch1-cpu
 3        40 ether01
             ether02
             switch1-cpu

vlan 10 and 20 should be working by now. So now we only need to do something clever with the management and the wireless. Let’s start with the management. We create a vlan interface, which can be handled by CPU.

[admin@MikroTik] > interface vlan
[admin@MikroTik] /interface vlan> add name=vlan40 interface=ether01 vlan-id=40

Connect the management IP to this interface.

[admin@MikroTik] > ip address
[admin@MikroTik] /ip address> add address=10.10.10.10 netmask=255.255.255.0 interface=vlan40

To make sure this interface can be used for updating ntp or download updates we need to make it routable. (assuming the router address is 10.10.10.1)

[admin@MikroTik\] > ip route
[admin@MikroTik\] /ip route> add dst-address=0.0.0.0/0 gateway=10.10.10.1

From now on this IP could be used for management. The last step is the wireless, for this we also need to make a vlan interface, which we will bridge to the wireless interface. (for as far as I know this is the only way).

[admin@MikroTik\] > interface vlan
[admin@MikroTik\] /interface vlan> add name=vlan30 interface=ether01 vlan-id=30

Create a bridge

[admin@MikroTik] > interface bridge
[admin@MikroTik] /interface bridge> add name=br-wireleass

The last step is to actually bridge the wireless with the vlan interface

[admin@MikroTik] > interface bridge port
[admin@MikroTik] /interface bridge port> add bridge=br-wireless interface=vlan30
[admin@MikroTik] /interface bridge port> add bridge=br-wireless interface=wlan1

That’s all! Hopefully it was any helpfull and will give you a clear understanding how things work.

Update 2015-07-15: Check this awesome Youtube explanation by David Gonzalez

Bind DNS Server in Homelab With Split View

Tue, 23 Sep 2014 19:54:11 +0200

One of the most undervalued infrastructure components in my opinion is DNS. A lot of services / components rely upon DNS and if DNS is mis-configured, not available, slow function or somehting else doing that shouldn’t be happening, it can lead to performance and other strange problems. In my homelab I’m running BIND DNS. One of the main reasons I choose BIND is the option to use Views. Views are making sure if I’m internally connected (e.g. WiFi @ Home) I get an internel IP as a response and if I’m externally connected, I get my public IP as response. My installation is based upon Debian 7, I won’t go into much details about this installation, since it is pretty straight forward. Make sure you use an static IP, don’t set your DNS to the server IP yet :) and I prefer to only install SSH, so I can start with a nice clean machine. First let’s install the BIND packages.

aptitude install bind9

Al the configuration files are stored in /etc/bind, so let’s go to this directory and start with creating a forwarder, so all DNS request can go through this DNS server. We will protect this forwarder by an ACL, so only internal cliënts can use this DNS server for relaying.

cd /etc/bind
nano named.conf.options

At the top of this file, before options, create the ACL, I use RFC1918, cause my subnets will vary a lot in my lab.

acl "trusted" {
        192.168.0.0/16;
        172.16.0.0/12;
        10.0.0.0/8;
        localhost;
        localnets;
};

In this example localnets, isn’t necessary assuming the server is already part of one the above listed subnets. Next we will add the lines, which enables the forwarding, these should be placed within the options placeholder.

forwarders {
       8.8.8.8;
       8.8.4.4;
};

allow-query { any; };
allow-recursion { trusted; };
allow-query-cache { trusted; };

That’s it for the forwarding. Now let’s create our own domain. First let’s disable the default configuration.

nano named.conf

and comment the line include “/etc/bind/named.conf.default-zones”;

//include "/etc/bind/named.conf.default-zones";

Open up named.conf.local

nano named.conf.local

and add the following lines.

view "local" {
        match-clients {10/8; 172.16/12; 192.168/16;};
        zone "lab.breekeenbeen.nl" {
                type master;
                file "/etc/bind/zones/lab.breekeenbeen.nl.local";
        };

        zone "11.10.10.in-addr.arpa" {
                type master;
                file "/etc/bind/zones/rev.11.10.10.in-addr.arpa";
        };

};

Here we declare a view, which will only listen to RFC1918 (local) addresses, we than create two different zones, one for forward look-ups and second for reverse look-ups. In his case the domain/zone is “lab.breekeenbeen.nl” and the IP range for the reverse look-up is 10.10.11.*. As you can see, we are pointing to two different files, these files will actually contain the hostnames. So let’s create and edit them. I added the .local for recognitizion that it will contain local addresses.

mkdir zones
touch zones/lab.breekeenbeen.nl.local
touch zones/rev.11.10.10.in-addr.arpa

Let’s start with the forward look-up table.

nano zones/lab.breekeenbeen.nl.local

$TTL 3D
@       IN      SOA     ns1.lab.breekeenbeen.nl. root.localhost(
        2014092201      ; serial number
        28800           ; refresh (i.e. 3h)
        3600            ; retry (i.e. 15M)
        604800          ; expire (i.e. 3W12h)
        38400           ; minimum (i.e. 2h20M)
);

@                       IN      NS      ns1.lab.breekeenbeen.nl.

dnsserver               IN      A       10.10.11.3;
ns1                     IN      A       10.10.11.3;

The above sample is pretty basic, remember everytime you make changes and commit them, you need to change the serial number, I use year-month-date-follow-up as a serialnumber. Every host can be added on a new line, here is one host “dnsserver” which will result in IP 10.10.11.3, the FQDN of this host is dnsserver.lab.breekeenbeen.nl. Restart the DNS server and check if it works with nslookup.

root@dnsserver:/etc/bind# nslookup
> server 10.10.11.3
Default server: 10.10.11.3
Address: 10.10.11.3#53
> dnsserver
Server:         10.10.11.3
Address:        10.10.11.3#53

Name:   dnsserver.lab.breekeenbeen.nl
Address: 10.10.11.3

If it doesn’t work, you can check on errors in the syslog.

tail /var/log/syslog

That’s it, now the last thing to configure is the reverse look-up.

nano zones/rev.11.10.10.in-addr.arpa

$TTL 3D
@       IN      SOA     ns1.lab.breekeenbeen.nl. root.localhost(
        2014092201      ; serial number
        28800           ; refresh (i.e. 3h)
        3600            ; retry (i.e. 15M)
        604800          ; expire (i.e. 3W12h)
        38400           ; minimum (i.e. 2h20M)
);

                        IN      NS      ns1.lab.breekeenbeen.nl.

3                       IN      PTR     dnsserver.lab.breekeenbeen.nl.

Restart the DNS server and check if the reverse look-up works.

root@dnsserver:/etc/bind# nslookup
> server 10.10.11.3
Default server: 10.10.11.3
Address: 10.10.11.3#53
> set type=PTR
> 10.10.11.3
Server:         10.10.11.3
Address:        10.10.11.3#53

3.11.10.10.in-addr.arpa name = dnsserver.lab.breekeenbeen.nl.

Now we can do exactly the same for the external look-ups. We just add another view for the external requests.

nano named.conf.local

view "external" {
        match-clients { any; };

        zone "lab.breekeenbeen.nl" {
                type master;
                file "/etc/bind/zones/lab.breekeenbeen.nl.external";
        };
};

It depends on your configuration if you need/want a reverse lookup for external requests. If you run an e-mail server for example it would be wise to set-up a reverse lookup zone. For now I leave the reverse look-up out of it, but it is exactly the same as described above. Like we did for internal, we create the external look-up file and fill it with the appropriate configuration. To speed things up, we just copy the file.

cp zones/lab.breekeenbeen.nl.local zones/lab.breekeenbeen.nl.external
nano zones/lab.breekeenbeen.nl.external

$TTL 3D
@       IN      SOA     ns1.lab.breekeenbeen.nl. root.localhost(
        2014092201      ; serial number
        28800           ; refresh (i.e. 3h)
        3600            ; retry (i.e. 15M)
        604800          ; expire (i.e. 3W12h)
        38400           ; minimum (i.e. 2h20M)
);

@                       IN      NS      ns1.lab.breekeenbeen.nl.

*                       IN      A       8.8.8.8

dnsserver               IN      A       8.8.8.8

As you can see, you can also add an asterisk (*) to answer the public IP on all public requests ending on .lab.breekeenbeen.nl. Before we can test the external request, you have to make a firewall rule which NAT all the UDP/53 request, to this server. Make sure if you’re going to test the configuration, that you do this from an external client and your DNS server set to your public IP. If you also want other requests for your domain coming to you through other DNS servers, you have to make a so called glue record at your domain hosting provider.

Nac Getting in Control of Your Mab Enabled Clients

Fri, 25 Mar 2011 19:11:08 +0200

It has been a while since I wrote a paper about the implementation of NAC. Now almost a year later it is finally in progress of implementation. One of the most time consuming processes and error sensitive ones is the adding of MAC addresses to the Active Directory of devices which doesn’t support Dot1X. Unfortunately there are a lot of devices which don’t speak Dot1X or having troubles with it. So if you want to do it right (IMO) you put these devices in different categories (and subnets) so you can put ACL’s on it (MAC spoofing can be easily done). At this moment we have three different categories within MAB authentication, which may grow in the near future; Thin clients, Printers and temporarily devices. To keep an clean view of all these MAC addresses in the AD I categories these MAC address in different OU’s, so I have three different OU’s which represent the different devices. We use Microsoft NPS server as Radius server and unfortunately you can’t (at least I didn’t find it) use the OU as a hit for a rule. So you also need to make three groups in which you place the MAC addresses (these are user objects in the AD). You also want to delete the “Domain User” group from the MAC address. Otherwise people would be able to login with MAC address on you domain members. So there you have already three different steps to just add one MAC address.

Add the MAC address to the right OU.
Add the MAC address to the right group.
Delete the group Domain Users (to accomplish this, set the other group as primary.)

This isn’t a problem, but if you have to add MAC addresses regularly, this is quiet annoying and you easily forget one of those steps. Another thing you might consider is that most of the time MAC addresses are added by other persons, it would be nice if you give them a tool which makes it easier for them and less faulty. To accomplish this I wrote a Powershell script (actual my first one, so be nice :-) ). This scripts draws a simple menu where a user simply can add or delete a MAC address. Since we in the same project move the printers to DHCP (reservations) I also added this option.

Below you find the powershell script, make sure the users have rights to the right OU in the AD, in the example it is “Network Access” and also give them rights on the User folder in the AD, otherwise the “Domain Users” group can’t be deleted (Took me almost an hour). If you also want to use the DHCP functionality, make sure they have rights there too. The script makes use of Quest “ ActiveRoles Management Shelf”, so this one needs to be installed. I tried to translate all the Dutch comment, to English and I also filled the variables with fictional values. If you have any comments or improvements, please let me know. Since it is my first powershell script, I’m sure there are.

# =============================================================
#	 NAAM: nac_menu.ps1
#  AUTEUR: Rob Maas
#   DATUM: 21-03-2011
# COMMENT: Adding/deleting MAC addresses to the AD for MAB.
# =============================================================

#[ GENERAL ]
# DHCP Settings
$dhcpServer 		= "DHCPSERVER1"
$dhcpScopePrinters 	= "192.168.1.0"

# Apparaat specifieke eigenschappen
$printer 	= @{"Name" = "Printer"; `
		       "OU" = "ou=Printers,ou=MAC,ou=Network Access,ou=corporation,dc=domain,dc=com"; `
		       "GRP" = "NA_Printers"}
$thinclient	= @{"Name" = "Thin Client"; `
		       "OU" = "ou=Thin Client,ou=MAC,ou=Network Access,ou=corporation,dc=domain,dc=com"; `
		       "GRP" = "NA_Thinclient"}
$temporary = @{"Name" = "Temporarily device"; `
		       "OU" = "ou=Temporarily,ou=MAC,ou=Network Access,ou=corporation,dc=domain,dc=com"; `
		       "GRP" = "NA_Temporarily"}

#[ MENU Declaration ]
# MAINMENU
$mnuMainTitle = "NAC Menu"	#Title
# Name
$mnuMainItems = @((0..4),(0..4))	#Dummy values
$mnuMainItems[0][0] = "Add MAC address"
$mnuMainItems[0][1] = "Delete MAC address"
$mnuMainItems[0][2] = "Add printer to DHCP "
$mnuMainItems[0][3] = "Delete printer from DHCP"
$mnuMainItems[0][4] = "Exit"
# Corresponding functions
$mnuMainItems[1][0] = {ShowAddMacMenu}
$mnuMainItems[1][1] = {DelMAC}
$mnuMainItems[1][2] = {AddPrinterToDHCP}
$mnuMainItems[1][3] = {DelPrinterFromDHCP}
$mnuMainItems[1][4] = {Exit}

#ADDMAC MENU
$mnuAddMacTitle = "Add MAC address"
$mnuAddMAC = @((0..3),(0..3))
#Name
$mnuAddMac[0][0] = "Add Thinclient"
$mnuAddMac[0][1] = "Add printer"
$mnuAddMac[0][2] = "Add temporarily device"
$mnuAddMac[0][3] = "Main menu"
#Corresponding
$mnuAddMac[1][0] = {AddMac $thinclient}
$mnuAddMac[1][1] = {AddMac $printer}
$mnuAddMac[1][2] = {AddMac $temporarily}
$mnuAddMac[1][3] = {ShowMainMenu}

#[ -- SCRIPT -- ]
#Methods in alphabeticall order

function AddMac{
	param($device)
	$address = GetMac;
	#New device
	Write-Host "Give in the" $device["Name"] "name: " -NoNewline
	$name = $Host.UI.ReadLine()
	New-QADUser -Name $address -UserPassword $address `
		-DisplayName $address `
		-LastName $address `
		-FirstName $address `
		-UserPrincipalName $address `
		-Description $name `
		-SamAccountName $address `
		-ParentContainer $device["OU"]
	#Unable to change password
	Set-QADUser -Identity $address -PasswordNeverExpires $true | Out-Null
	#Add to right group and delete "domain users" group
	Add-QADGroupMember -Identity $device["GRP"] -Member $address | Out-Null
	Set-QADUser -Identity $address -ObjectAttributes @{PrimaryGroupID=(Get-QADGroup -Identity $device["GRP"]).PrimaryGroupToken } #| Out-Null
	Remove-QADGroupMember -Identity "Domain Users" -Member $address #| Out-Null
	Write-Host "Account is created !!!" -ForegroundColor Yellow
	#If it is a printer, add to DHCP?
	if ($device -eq $printer){
		$dhcp = Read-Host "Add the printer to DHCP? (Y|N) ?"
		if ($dhcp -eq "y") {AddPrinterToDHCP $name $address}
	}
	if (Again) {AddMac $device} else {ShowMainMenu}
}

function Again{
	$again = Read-Host "Again (Y|N) ? "
	switch ($again){
		"Y" {return $true}
		default {return $false}
	}
}

#Add printer to DHCP
function AddPrinterToDHCP{
	param($name, $address)
	if (($address -eq $null) -or ($name -eq $null)){
		$name = Read-Host "Printer name: "
		$address = GetMac $false
	}
	$ip = Read-Host "The IP address of the printer (192.168.1.0)?"
	Write-Host "Printer: `t $name"
	Write-Host "IP: `t`t $ip"
	Write-Host "MAC: `t`t $address"
	$ok = Read-Host "Are the above details correct (Y|N|X = Menu) ?"
	switch ($ok){
		"y" {
			 Invoke-Expression -Command "netsh dhcp server $dhcpServer scope $dhcpScopePrinters add reservedip $ip $address $name" | Out-Null
			 Invoke-Expression -Command "netsh dhcp server $dhcpserver scope $dhcpScopePrinters set reservedoptionvalue $ip 012 STRING $name"
			 if (Again) {AddPrinterToDHCP} else {ShowMainMenu}
			 break}
		"n" {$name = $null
			 $address = $null
			 AddPrinterToDHCP
			 break}
		"x" {ShowMainMenu
			 break}
	}
	$name = $null
	$address = $null
	if (Again) {AddPrinterToDHCP} else {ShowMainMenu}
}

function DelMaC{
	$address = GetMac $false $true
	$delete = Read-Host "Are you sure, you want to delete $address (Y|N) ? "
	switch ($delete){
		"y" {Remove-QADObject -Force -Identity $address | Out-Null
			 Write-Host "$address verwijderd!" -ForegroundColor Yellow
			 if (Again) {DelMac} else {ShowMainMenu}
			 break}
		default {ShowMainMenu
				 break}
	}
}

function DelPrinterFromDHCP{
	$address = GetMac $false $true
	$ip = Read-Host "The IP address of the printer (192.168.1.0)?"
	Write-Host "MAC Adres: `t $address"
	Write-Host "IP Adres: `t $ip"
	$ok = Read-Host "Are the above details correc (Y|N|X = Menu) ?"
	switch ($ok){
		"y" {
			 Invoke-Expression -Command "netsh dhcp server $dhcpServer scope $dhcpScopePrinters delete reservedip $ip $address"
			 if (Again) {DelPrinterFromDHCP} else {ShowMainMenu}
			 ShowMainMenu
			 break}
		"n" {$address = $Null
			 DelPrinterFromDHCP
			 break}
		"x" {ShowMainMenu
			 break}
	}
}

function DrawMenu{
	param($menuItems, $menuTitle, $menuPosition)
	$fgColor = $Host.UI.RawUI.ForegroundColor
	$bgColor = $Host.UI.RawUI.BackgroundColor
	cls
	$l = $menuItems[0].Length - 1;
	$menuWidth = $menuTitle.Length + 8
	Write-Host "`t" -NoNewline
	Write-Host ("*" * $menuWidth) -ForegroundColor $fgColor -BackgroundColor $bgColor
	Write-Host "`t" -NoNewline
	Write-Host "*   $menuTitle   *" -ForegroundColor $fgColor -BackgroundColor $bgColor
	Write-Host "`t" -NoNewline
	Write-Host ("*" * $menuWidth) -ForegroundColor $fgColor -BackgroundColor $bgColor
	Write-Host ""
	Write-Debug "L: $l MenuItems: $menuItems MenuPosition: $menuPosition"
	for ($i = 0; $i -le $l; $i++){
		Write-Host "`t" -NoNewline
		if ($i -eq $menuPosition){
			Write-Host " $($menuItems[0][$i])" -ForegroundColor $bgcolor -BackgroundColor $fgColor
		} else {
			Write-Host " $($menuItems[0][$i])" -ForegroundColor $fgcolor -BackgroundColor $bgColor
		}
	}
	Write-Host
}

function Exit{
	Invoke-Expression -Command "exit"

}

#Check if the given MAC address is valid.
function GetMAC{
	param([bool]$new = $true,		#Has to be new in the AD
		  [bool]$exist = $false)	#Must exist in the AD
	$mac = Read-Host "Give in the MAC address in lowercase and withouth punctuations: "
	#Check if the address has a length of 12
	If ($mac.length -ne 12){
		Write-Host "Invalid address!" -ForegroundColor Red
		if (Again) {GetMac $new $exist; return} else {ShowMainMenu; return}
	}
	If (($new) -and (Get-QADUser -Name $mac)){
		# Already exist
		Write-Host "Address already exist!" -ForegroundColor Red
		if (Again) {GetMAC $new $exist; return} else {ShowMainMenu; return}
	}
	if ((-not $new) -and ($exist) -and (-not (Get-QADUser -Name $mac))){
		#Address not found
		Write-Host "Address not found! " -ForegroundColor Red
		if (Again) {GetMAC $new $exist; return} else {ShowMainMenu; return}
	}
	return $mac
}

function Menu{
	param($menuItems, $menuTitle)
	$vkeyCode = 0
	$menuPosition = 0
	DrawMenu $menuItems $menuTitle $menuPosition
	While ($vkeycode -ne 13){
		$press = $Host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")
		$vkeyCode = $press.virtualkeycode
		Write-Host "$($press.character)" -NoNewLine
		If ($vkeyCode -eq 38) {$menuPosition--}	#Down
		If ($vkeyCode -eq 40) {$menuPosition++}	#Up
		If ($menuPosition -lt 0) {$menuPosition = $menuItems[0].Length - 1}
		If ($menuPosition -ge $menuItems[0].Length) {$menuPosition = 0}
		DrawMenu $menuItems $menuTitle $menuPosition
	}
	$($menuItems[1][$menuPosition]).Invoke()
}

function ShowAddMACMenu{
	Menu $mnuAddMac $mnuAddMacTitle
}

function ShowMainMenu{
	Menu $mnuMainItems $mnuMainTitle
}

ShowMainMenu

Configuring Vlans on a Draytek Vigor 2130 N

Tue, 15 Mar 2011 15:35:10 +0200

Before I begin, I really need to thank the guy from the Vigor 2130 Google Code page, I did not get his name, but without his help and patient it would never succeeded. So thank you!

When I bought the Draytek Vigor 2130n one of the requirements I had was VLAN support. I could imagine that in the near future I was willing to split up my home network in multiple networks (VLANs). So I was amazed when the time was finally there, that the VLAN support I was looking for wasn’t supported through the web interface. I even read the manual and there was just no way to configure VLANs the way I wanted. So as my last resort I went to the Google Code page and asked for help. Below you find a summary of how I configured my vlans. Before we actually begin, let me explain what I wanted to built. My idea was to get some experience with the Vyatta firewall which has a free community edition and since I have a couple of services which are accessible through the Internet I planned to create a DMZ, a LAN and a Transit zone. If we place it in a logical view, it looks something like this.

As you can see, there are three different zones, so I needed to create at least 3 VLANs. Later on you’ll notice, that the Vigor handles the WAN zone (connection between de router and the modem/Internet) also as VLAN. To make everything just a little more complex the VLANs should be available on a smart switch ( Netgear GS108t v2) too. This smart switch is connected on the Vigor 2130 on LAN port 4, so I needed to make this port a trunk(multi-vlan port), the other three LAN ports, could be placed in the LAN network. Before we dive any deeper, here are the details for the VLANs, the VLAN ID and subnets. (I know that VLANs are a Layer 2 functionality, but to make the above situation work, we also need to make some Layer 3 changes, which I will explain later).

VLAN	ID	Subnet
Transit	10	192.168.1.0/24
LAN	20	10.0.0.0/24
DMZ	30	172.16.0.0/24
WAN	2

Unfortunately for those of you who like the mouse, all the settings are made through the CLI. So login to the router with telnet or SSH, depending on your configuration. The VLAN settings are placed in the “vlan” and “vlan_port” files under “/etc/config/switch/”. The “vlan file”, contains the VLAN ID and which ports are member of this VLAN. The membership is a binary filter.

128	64	32	16	8	4	2	1
x	x	LAN4	LAN3	LAN2	LAN1	WAN	x

As I mentioned before, LAN Port 4 is becoming a trunk (multi-vlan) port for VLAN 20 and 30, the other LAN ports are becoming a member of VLAN 20 (LAN). After configuring the file, with “vi”, it look like this.

2/2 10/32 20/60 30/32

For example in case you didn’t see where the numbers are coming from, VLAN 20 has LAN ports 1,2,3 and 4 in it. So if you fill in the table.

128	64	32	16	8	4	2	1
x	x	LAN4	LAN3	LAN2	LAN1	WAN	x
x	x	yes	yes	yes	yes	no	x

Now count the fields that contain yes, with the above values and you will find out, where the number 60 is coming from. So far the “vlan file” configuration, now we need to set the specific port details for the vlans, which can be done in the “vlan_port file”. The vlan_port file, contains the following information.

Port (1 – WAN, 2-5 LAN)
VLAN Aware (Bring VLAN tag out)
Ingress filter (Only accept the VLAN belong to this port)
Frame Type (0 – Accept all frames, 1 – Accept tagged frames only, 2 – Accept untagged frames only)
VLAN ID

There are only two options which need our attention for the above set-up. The VLAN Aware option, which we need for LAN port 4, cause this port is becoming a trunk (multu-vlan) port. The last port, sets the native VLAN of the Port, which is in our case 20. So after changing the file, it will look like this.

1/0/0/0/2
2/0/0/0/20
3/0/0/0/20
4/0/0/0/20
5/1/0/0/20

The most important thing is the last “1” just after the “5”, this makes port 5 (LAN Port 4) a trunk (multi-vlan) port. Now that that the VLANs are created, we can put devices in it. The only problem left is that the devices, cannot communicate with devices in other VLANs, let alone the Internet. So we need to create some gateways. The Vyatta firewall will take care of the routing between the different VLANs, to make sure the routing is working probably, I created rules which allow everything. (So I don’t go blind on a firewall deny rule). This is the picture, with the subnets and specific gateways in it. On the Vyatta firewall, I created a default routing to the Vigor. So every package which isn’t directly connected (DMZ/LAN/Transit), goes to 192.168.1.254. Before we actually can reach this address on the Vigor, we need to add it. This can be done in the “network file”, which is placed in /etc/config. We need to change the IP-address and the interface name, the number after the period tells to which VLAN this interface belongs to. So ifname ‘eht0.10’ belongs to VLAN 10. The corresponding part after changing the file.

config interface lan option ifname ’eth0.10’ option proto static option ipaddr ‘192.168.1.254’ option netmask ‘255.255.255.0’ option detect 0 option type ‘bridge’ option pppoe_pass 0

For the Vigor we need to do the same thing, cause every packet which is for the DMZ or LAN cannot be delivered directly. So we create two routing rules, to make sure the packets are delivered correctly. This can be done in the web interface or in the CLI. In the CLI, it can be done by editing the “s_route file” which is also placed in the /etc/config directory.

config static-route sr0 option enable ‘1’ option net ‘172.16.0.0’ option mask ‘255.255.255.0’ option gateway ‘172.16.1.253’ config static-route sr1 option enable ‘1’ option net ‘10.0.0.0’ option mask ‘255.255.255.0’ option gateway ‘172.16.1.253’

Finally it is time to reboot the router and make the settings active! If everything worked out well, you can now send packages between the VLANs. There is only one thing left. You can’t send packages to the Internet. There are two reasons why sending packages to the Internet doesn’t work.

The IP-Tables firewall is blocking “unknown” subnets, in this case the DMZ and LAN.
There is no natting done, so packages(responds) can never find there way back home.

First we fix problem one. This can be done, by the following command.

iptables -I FORWARD -s 172.16.0.0/24 -d 0.0.0.0 -p ALL -j ACCEPT iptables -I FORWARD -s 10.0.0.0/24 -d 0.0.0.0 -p ALL -j ACCEPT

With the following command, you can check if the entries appear in the FORWARD list.

iptables -L FORWARD

For the second problem, we add both subnets to the NAT list. With the following command you can see the current NAT list.

iptables -t nat -L zone_wan_nat

You will see that only the subnet which is known by the router is added to the list. So we need to add the DMZ and LAN zone manually.

iptables -t nat -I zone_wan_nat -s 172.16.0.0/24 -d 0.0.0.0 -j MASQUERADE iptables -t nat -I zone_wan_nat -s 10.0.0.0/24 -d 0.0.0.0 -j MASQUERADE

Now the list, will look like.

Chain zone_wan_nat (1 references) target prot opt source destination MASQUERADE all – 192.168.1.0/24 anywhere MASQUERADE all – 172.16.0.0/24 anywhere MASQUERADE all – 10.0.0.0/24 anywhere

Unfortunately the iptable settings are gone after a reboot. These settings can probably be added in the following two files, but I haven’t tested it yet.

/etc/firewall.user
/lib/firewall/uci_firewall.sh

If you have modified these files, please let me know. Have fun with your VLAN enabled Draytek Vigor 2130

Automatically Send Network Device Configurations to Your Support Partner

Thu, 09 Sep 2010 20:00:58 +0200

In the last part of the Rancid set-up I created a simple Bash scripts which e-mails the configurations of the network devices to myself and our support partner once a week. This because when your network is down, you always want access to your configurations. Further for our “sub” locations, we have a support partner which fixes the issues on these locations and it would be nice if they always have the up-to-date configuration files. Since there is not much changing in these configurations sending them once a week is more then enough, but this is of course up to you. The script, works fairly simple, it is important that you have set-up the right SMTP server!

#!/bin/bash 

# Script for mailing the network device configurations 

# Make a directory where we can temporarily place the configurations 
mkdir -p /tmp/backup/switch 

# Looking for files starting with “sw” and that are greater then 0 bytes. Copy them to # the just created temporarily folder.
find /usr/local/rancid/var/ -name "sw*" -not -name "\*svn\*" -prune  -size +0 -exec cp '{}' /tmp/backup/switch/;

# Put the files together in one tar file.
cd /tmp/backup/switch tar -cf /tmp/backup/switch/bck\_switch\_cfg\_week$(date +%U).tar *

#Mailing the just created tar file 
echo "Backup netwerkdevices week $(date +%U)" | mailx -s "Switch backup week $(date +%U)" -a bck\_switch\_cfg\_week$(date +%U).tar -r backup@\_nospam\_robmaas.eu  rob@\_nospam\_robmaas.eu support@partner_xx.com

# Delete the just created folder and files
rm -Rf /tmp/backup/switch

That’s all, keep in mind that the find and mailing (echo) commands are on one line, otherwise they will fail. Also you may want to change the –name parameter of the find command. No we have a working script we only have to schedule it to run regularly. This can easy be accomplished with cron. I created a cronjob in the /etc/cron.d/mailbackups with the following cronjob.

@weekly /usr/local/bin/backup_switch_cfg.sh 2>&1

We are done, now every week the configurations are mailed to the given mail addresses.

Backing Up Network Devices With Rancid Opensuse 11 2

Wed, 25 Aug 2010 19:58:06 +0200

One of the most forgotten backups is that of the network devices, while it can save you a lot of time (thus money) when things get broken. Luckily there is a tool, called Rancid. I’m using it for all our HP and Cisco devices for more then a year now. Officially it was running on a (Ubuntu) test-server which was placed under my desk :-). Not the best place for such a critical server. So now it is time to set-up a “real” management server, which takes care of the network devices backup. Since we already have some Suse servers running, I picked OpenSuse 11.2 as OS for this server. Let’s get started. Before we can install Rancid, we need to install “expect”, this is the language Rancid is written in. The most easy way to accomplish this is by using yast and search for “expect”. Now we have started yast, we also need to install “cvs”, which Rancid uses for file comparison.

Now download Rancid, it can be retrieved from this website. I downloaded it in my home directory under downloads/rancid and unpack it.

wget ftp://ftp.shrubbery.net/pub/rancid/rancid-2.3.4.tar.gz tar -xvf rancid-2.3.4.tar.gz

Go to the unpacked directory and install Rancid.

cd rancid-2.3.4 ./configure make install

If everything went ok, there is a directory created /usr/local/rancid. Since we don’t want to use a root account for Rancid to work, we create a user called rancid and set his home directory to the installation directory /usr/local/rancid. After we created the user, we make him the owner and give right permissions.

useradd -d /usr/local/rancid rancid chown -R rancid /usr/local/rancid/ chmod -R 770 /usr/local/rancid/

Now we can start configuring Rancid. If you’re a real 1337 person, you use vi, but I like nano more, so the next command is.

nano /usr/local/rancid/etc/rancid.conf

What I usually do is create a group for every location, for this go to the line

#LIST_OF_GROUPS=”sl joebobbisp”

and change it to.

LIST_OF_GROUPS=”Headquarters” LIST_OF_GROUPS=”$LIST_OF_GROUPS Location1” LIST_OF_GROUPS=”$LIST_OF_GROUPS Location2” …

Now let’s run Rancid CVS to create the directories and configuration files for the locations. We do this with the Rancid user.

su – rancid ./bin/rancid-cvs

If you now look in the /usr/local/rancid/var/ directory, you should see the directories which Rancid created by following the rancid.conf file. Additional there are two directories created, “CVS” and “logs”. Within the location directory there is a file, called router.db, this file contains all the routers off that location, which we like to back-up. Since Rancid has no idea by itself, what we like to back-up we need to add these devices ourselves.

cd location nano router.db

Type the following lines, replace the “core” and “distr” with your own names or IP address. My advice is to take DNS names, cause it is more clear then IP addresses, also the name or IP is used to create filenames which contains the actual configuration.

#Devicename or IP : Devicetype : Up CORE:HP:UP DISTR:CISCO:UP

As you probably already guessed, the # is used for comments. Now there is only one thing left. Rancid needs credentials to get access to the device. We create a file in the home directory of the rancid user and since we’re running the command as the user rancid, the command is very simple.

nano ~/.cloginrc

Let’s add the following lines for general access and one specially for the core.

#Specific location/device add password core verydifficultandlongpassword #General add password * asimplepassword add method * telnet add autoenable * 1

Unfortunately Linux thinks that this file may be read by everyone, we definitely don’t want this and rancid will even give an error on this, so we need to make sure this file is only readable by the rancid user and the group root.

chmod 770 ~/.cloginrc

Now we can test Rancid by running it.

~/bin/rancid-run

When it is done, check the config directory in the ./var/location directory and see if there is a file placed, use cat to check if the configuration is actually in i.

cd ~/var/location1/config ls cat core

Congratulations! If everything went ok, you now should see the configuration of the switch you backed upped with Rancid. If you don’t see the configuration, go to the var/logs directory and see if you can find out what went wrong in the log.

cd ~/var/logs ls less location1.20100825.160011

But we ain’t there yet, of course we like to automate this process. This is fairly simple, cause we have created a user, this user has his own cron.

crontab –e

In my case it is enough to make a daily back-up, but this is of course your own decision.

@daily /usr/local/rancid/bin/rancid-run @daily /usr/bin/find /usr/local/rancid/var/logs -type f -mtime +1 -exec rm {} ;

The second line deletes log files, older the 2 days. Now we’re done with Rancid. The next step is create a nice web interface for configuration comparison and something we do is mail all the configurations to our support partner weekly. So when there is trouble they also have the latest configurations. I’ll cover these steps later on.

Mac Authentication Bypass Mab on Hp Procurve 2600

Wed, 23 Jun 2010 19:36:52 +0200

For my thesis I did a little research on Network Access Control and the possibilities. This research was focused on the environment of the company I work for, this means I included both Cisco and HP switches in my research. After this research I build a test environment to test the authentication mechanisms 802.1x, MAB and web authentication. With Cisco everything was working flawless, but I also wanted a sort of MAB authentication on the HP switches, unfortunately HP doesn’t speak MAB. So after some puzzling I found a work around which is close enough to MAB. Normally you can only use 802.1x or MAC Authentication on a HP Procurve switch (2600). To work around this problem HP included a feature, so called Client Based Network Authentication. This feature is initially created to make it possible to authenticate devices which are connected to a HUB on the switchport. It is possible that devices connected to the HUB require different authentication mechanisms, so this gives the opportunity to allow MAC and 802.1x authentication on the same port. When we switch to Client Based Network Authentication and we set the allowed clients to 1, it is possible to let the client choose which authentication is going to be used.

The figure above shows how it schematically works, the HUB is just virtual and used as example. To let this work you only need to configure the client limit.

aaa port-access authenticator 1-48 aaa port-access authenticator 1 client-limit 1 aaa port-access authenticator 2 client-limit 1 aaa port-access authenticator 3 client-limit 1 … aaa port-access authenticator active aaa port-access mac-based 1-48 aaa port-access 1-48 vlan 31 name “MACAuth_Vlan” tagged 49,50 exit

Dynamic Acl on Cisco With Ms Nps

Wed, 28 Apr 2010 13:45:57 +0200

As some of you might know, I’m busy setting up a test environment for Port Based Network Access Control. With Cisco switches this is plain simple, cause when 802.1x fails you can switch to MAB. A bit less secure, but better then nothing. When MAB fails you have the option to switch to Web authentication, a so called Captive Portal. This web authentication is also easy to setup, but the last step was a bit tricky. Very short summary, when the switch, switches to web-authentication, it creates an access control list (ACL) to block all the traffic, but still give the user IP connectivity to open a browser and be able to fill in the credentials and post this form. When the credentials are validated by the RADIUS server, it has to say what the new ACL is, cause otherwise the user has very limited access. Most tutorials on the web (the ones I found), explain this very well, but uses a CiscoSecure ACS. In my situation I don’t have an ACS, but I uses the Microsoft Network Policy Server (NPS) as RADIUS instead. This works fine, but it was a bit hard to find how to define the ACL. (as always afterwards it is quite easy :-)) This tutorial, explains very well how to configure the switch for web authentication with an Cisco ACS, I’m not going to try to do this better, so for the switch settings, please follow that tutorial. When the switch is configured, you have to configure NPS. I assume you already added your device, to the device list including a shared key. So we can go on with the policy, for authentication I still use the default policy (all users), but for authorization we create a new one. In my test environment I putted all the users which may authenticate on the web to a group called GG_WEBAUTH, so it easy to define the matching criteria. By default there is no encryption, so we use PAP for authentication (don’t know yet, if you can change this). After creating the matching criteria, we have to set the vendor specific attributes, for Cisco this is the “Cisco-AV-Pair” attribute. I added 2 parameters to this attribute

priv-lvl=15 ip:inacl#100=permit ip any any

Of course you can change the number, you can even add more numbers to create longer ACL’s. With this two parameters your web-auth profile should work. There should also be a way with the ACL defined on the switch and make use of the filter-id attribute, but I was unable to get this to work.

Ldap Authentication for Ipplan Apache Linux

Mon, 21 Dec 2009 19:46:45 +0200

As you may have read, I’m testing with an IP management tool, called IPPlan. Since I’m not the only one at our company, that need access to IPPlan and I wasn’t planning in doing a lot of user management, I did a try to connect IPPlan to our Active Directory (LDAP) environment. It took me a while to figure it out, maybe cause the lack of “good” documentation or my experience. It doesn’t matter, because it is working now. This is what I did to get it working. Go to the subdirectory “user” in the “ipplan” directory, the full path (here) is: “/var/www/ipplan/user” and create a file with the name “.htacces”. Edit the file with the following information.

AuthType basic AuthName “IP Plan LDAP Authentication” AuthBasicProvider ldap AuthLDAPURL ldap://ldapserver:389/ou=accounts,dc=robmaas,dc=eu?cn AuthLDAPRemoteUserIsDN off require valid-user

The “ldapserver” is just the IP or DNS name of your LDAP(AD) server. After the URL you can use a standard LDAP query (don’t forget to replace the domain name). If your LDAP server needs authentication like mine does, you need to add the following two variables.

AuthLDAPBindDN “ ldap@robmaas.eu” AuthLDAPBindPassword “secret”

Make sure, your Apache configuration does support the including of “.htaccess” files. This can be done by setting the “AllowOverride” to All, like this.

AllowOverride All

Don’t forget this; it took me about 2 hours, before I got it. Also don’t forget to include the LDAP module in to Apache.

a2enmod authnz_ldap

After this we need to edit “config.php”, find the following line:

define(“AUTH_INTERNAL”, TRUE);

and change it to:

define(“AUTH_INTERNAL”, FALSE);

The last change I had to make, was changing this line:

define(“AUTH_VAR”), ‘PHP_AUTH_USER’);

into

define(“AUTH_VAR”), ‘REMOTE_USER’);

That’s it, after restarting the Apache (httpd) service, it should all work. Don’t forget, the authentication is done through LDAP, but you still have to create the users in IPPlan.

Sidenote: If the webpage is running on HTTP the username(s) and password(s) are sent in plain text to the webserver. In this case you should think about migrate it to HTTPS.

Ipplan Export to Import for Poller

Wed, 16 Dec 2009 19:55:04 +0200

Like many others, I also hate administrative tasks. One of these tasks that comes back again and again is IP documentation. To make my life easier and probably that of others :-), I decided to go testing with IP management software. After some googling I found IPPlan, this tool which looks very promising, can do everything I need. It manages my IP addresses and gives me a quick overview of the status, it also gives me the possibility to “poll” addresses and automatically find used ones. This feature is great, except for one little thing, you have to create a new file with all the subnets you like to poll. If you are a greedy bastard like me, you like to have all subnets polled. We have over 500 subnets and after adding them to IPPlan I wasn’t very pleased with the idea of adding them manually to the list. Luckily for me, it is very easy to make a export from IPPlan with all the subnets, unfortunately this export is not well formatted for the polling list. The exported list will look like this:

10.10.4.0 Client-Vlan4 255.255.255.0 10.10.5.0 Client-Vlan5 255.255.255.0 10.10.6.0 Client-Vlan6 255.255.255.0

The import list should look like this:

10.10.4.0/24 10.10.5.0/24 10.10.6.0/24

What I did is created a very small and simple vbscript, which will make this transition for me. You can start the vbscript as follow.

cscript ipplantopoller.vbs exportfromipplan.txt importtopoller.txt

As you maybe have already guessed the exportfromipplan.txt is the exported list and the importtopoller.txt is created by the script. You can download the script below and if you have any comments or questions please let me know, IPPlanToPoller.vbs .

Connection Pool vs Firewall

Tue, 13 Oct 2009 15:32:47 +0200

As some of you might have noticed, I’m busy at work to create a new reverse proxy environment to publish our websites. The whole set-up is very simple and commonly used in the world, so I wasn’t expecting a lot of problems. A short summary:

We have a ISA array set-up as reverse proxy and behind the ISA there are the webservers. The webservers get there data from a database server. In our test environment all these three machines are in a different subnet. The database was placed in an “old” subnet which is deprecated after the completion of the project.So we would not harm the production environment and the webservers connect to that subnet. The ISA runs a Form Based Authentication (FBA) against an Active Directory in the same subnet as the webservers.

About a week ago, we got into a beta setup, to let some well choosen customers test the environment. And since the first day we got some tips that a customer has to push twice the logon button to get actually logged in. The biggest problem for us, that this “twice” login problem just occured a few times a day, which makes it hard to find the actual problem. First we thought this problem was caused by the ISA’s, so we checked every option of the publish rule, but without any luck. The next thing we started to do was to capture all the packets on all the interfaces, which took a lot of time, cause we needed a capture of the moment that the problem occured. Just to get a good capture took us at least a day. After searching a lot through the capture files, we finally find out, that there went something wrong between the webserver and the database communication. This is a part of the communication between the webserver and databaserver, this capture is made on the webserver. ]

no_syn_synack

As you can see the webserver started immediately with PSH, ACK instead of building up a connection with SYN, etc. The big question for us was why and this was a bit of puzzling, but it tuns out that the firewall between these subnets was terminating the connections. So when the webserver was in need to contact the DB server, it takes a look in its connection pool and picked an connection which was (by his thoughts) open, so no SYN was sent. You can guess what happened, the firewall has closed that particular connection and denied the connection, cause no SYN packet was sent. So whenever you want to set-up a connection pool, make sure there is no Firewall between them or that the time-out for IDLE connections are properly configured on both sides. This can save you a lot of time :-)

Citrix Web Interface 5 X 5 1 2 Client Deployment

Mon, 31 Aug 2009 15:45:00 +0200

A few days ago I updated our Citrix Web Interface to version 5.1.2. After this update it seems that the Client Deployment process was broken. Users still got the download button, but after pressing this button it opens the Citrix website. The problem can be easily solved, but it took some time to figured it out. By default the web-interface is installed under “C:Program FilesCitrixWeb Interface5.1.2” and it contains a directory which is empty by default. In this directory you have to create a subfolder, called “ica32”. Now download the latest client(s) from the Citrix website and copy these to the new created folder. It is very important that the names are identical to the names below: XenAppHosted.msi XenAppWeb.msi XenAppWeb.exe Now the download button should work again. Note: If you still want to use the old plugins ica32**, you can do this by simplying renaming them.

Isa SP1 Intra Array Communications Outside Domain

Fri, 21 Aug 2009 19:52:57 +0200

I’m busy to create a nice reverse proxying environment, for this we choose the ISA 2006 server. Mainly because we have some experience with this and it should be really easy to create a High Availabilty solution. The biggest problem in our case is that the ISA servers are not member of a domain and that the ISA Configuration Storage Server is placed on a separate server in a whole different subnet. After some puzzling everything was working great, until we updated the environment to ISA 2006 SP1. The ISA server went complaining that the Intra-Array communication was broken (event id: 221225). The exact message:

For intra-array authentication when array members are in a workgroup, the intra-array account must be defined and enabled. Some features such as VPN, CARP, and reporting will not work unless the intra-array account is properly configured.

It took not really long to figure out what went wrong, but I find it worth mention it, in case someone else rans in the same situation. The problem was that the local accounts had different passwords and since SP1 wasn’t asking for the Array members password anymore it could not establish a connection. So setting the same password on the ISA CSS machine was all we had to do to fix the problem.