Thin Clients Part II - Security Added

Late last week I took my little thin client computing concept a step further and added some security. My goal was to add at least one layer of security, if not two layers to this process. Like a smart card concept I wanted to have a physical token (the thumb drive itself) as well as a "secret PIN" which the user would need to supply. This might add a bit of time to the login process, but the security would be well worth it.

The New Process
We will still use an autorun.inf file which will allow the user to plug in the thumb drive and simply press enter for the default choice. The default choice, is to run a batch script which I have called "connect.bat". Here are those files.

autorun.inf
[autorun]
open=connect.bat
ACTION = LAB CONNECT

connect.bat
SET STARTRUN=%0
7za.exe e -oC:\TEMP -y RDPPACK.zip
REM COPY %0\..\*.* C:\TEMP
C:
cd C:\TEMP
start /normal ssh.bat
PING -n 8 127.0.0.1>null
start /normal RDP.bat
start /normal WAIT.bat

When connect.bat runs, it launches 7zip, which extracts a zip file to the c:\Temp directory. I used 7zip for a couple reasons: a) it's freely downloadable at http://www.7-zip.org, b) you can create password protected (encrypted) zip files with it, c) I had it installed and I all ready use it in other batch processes.

My zip file, RDPPACK.zip, is an archive which contains several other batch files. One of these batch files contains passwords so we protect the zip file by password protecting it when we create it.

My RDPPACK archive contains the following files:
labconnect.rdp - This is the file which contains RDP details such as IP and port number.
plink.exe - This is a free command line secure shell client for windows.
RDP.bat - Simple batch file to connect to the host using the settings in labconnect.rdp.
ssh.bat - This is a new batch file which uses plink.exe to open a secure shell session with our VirtualBox server.
WAIT.bat - This waits for the thumb drive to be removed, and then cleans up and kills things when that happens.

Once the zip is un-extracted (takes only a split second) we start up three other batch files simultaneously.

Here is the play by play. The user plugs in the thumb drive which runs connect.bat. This starts the file extraction which pauses momentarily and waits for the user to enter their password. In my examples I used a simple four digit "PIN Number". The files are extracted, and "ssh.bat" is launched.

ssh.bat
SET STARTRUN=%0
plink -ssh -L 13390:localhost:3390 -pw password username@10.0.46.55
EXIT

There are two items of bad news to mention here. One is that there is an EIGHT SECOND wait placed after this script runs. This is the unfortunate amount of time it takes for the secure session to be established. The other bad news is that the username and password are exposed here "in the clear". This could be hidden by putting an @ symbol in the batch script in front of that first line (keeping the command from being echoed to the user). But it will sill remain in the Temp directory while this script is running and a bad guy could find it. I would like to think that the user will not share the PIN number which revealed this information, and if their thumb drive was lost or stolen you could simply change this password on the server. So it's "pretty good" security in my book.

The plink syntax works like this: "-ssh" means to use the secure shell protocol, the "-L 13390:localhost:3390" will redirect connections that the host makes to itself on port 13390 to the server on port 3390, the "-pw password" would be this users password on the server, "username@10.0.46.55" would be the users username and the servers IP address. This means that we have created the user on the server, assigned them a password, enabled the secure shell daemon, and we are firewalled to disallow connections on port 3390. Yes, we do NOT want to allow connections to port 3390 from anybody. The only reason our remote users can do it is because they are sending these connections through the secure tunnel we established here. Clever, huh?

Once the secure session is established (we simply waited 8 seconds and assumed it's ready) we then run RDP.bat and WAIT.bat simultaneously.

RDP.bat
mstsc /f labconnect.rdp
EXIT

WAIT.bat
PING -n 10 127.0.0.1>null
GOTO CHECKEXIST

:CHECKEXIST
IF EXIST %STARTRUN% GOTO WAIT
GOTO KILLTASK

:WAIT
PING -n 3 127.0.0.1>null
GOTO CHECKEXIST

:KILLTASK
taskkill /f /im "mstsc.exe"
taskkill /f /im "plink.exe"
DEL /F C:\Temp\autorun.inf C:\Temp\connect.bat C:\Temp\labconnect.rdp C:\Temp\plink.exe C:\Temp\RDP.bat C:\Temp\ssh.bat C:\Temp\7za.exe
EXIT

RDP.bat will connect us up to the server. There were some changes made in the RDP file. That is, the client now connects to localhost:13390 instead of the server IP and port 3390.

WAIT.bat will start pinging itself in a loop, and wait for the drive to come disconnected. When that occurs, it will immediately end task on the terminal services connection, and then the secure shell tunnel. Afterward (and this is new) it does some cleaning up and deletes all that stuff that it left laying around in C:\Temp. The only thing which will remain is the WAIT.bat file itself. Which as you can see, presents no risk as it contains no passwords, etc.

Problems
I probably spent a couple of hours on this process over the span of a week. What was most frustrating was getting VirtualBox to cooperate with me. There are a few known issues with various VRDP elements. One is "authentication". Theoretically you can authenticate your VRDP sessions against a local user database on your VirtualBox server. This didn't work at all for me and after reading through a couple of forums I found that it doesn't work for anyone else either.

I am also having problems with my Windows clients when they connect to a VirtualBox at full screen. It seems that the windows get doubled up and don't display correctly. If you specify an exact window size in your RDP file such as 800x600, you will not have this problem. I went through the trouble of setting up a Windows VirtualBox server and found that the problem exists there as well. I have since opened up a Bug report with VirtualBox which I hope gets some attention.

Lastly, expect VRDP on VirtualBox to provide you with rather slow window refreshes. I would like to think that this is also something that the VirtualBox developers are improving as they have always been aware of this bug.

Hack on, and I hope that someone out there finds this information useful, at some point. ;-)

Thin Client Computing on the Cheap

Many, many years ago I was attending a trade show and I saw something really cool. There was a booth set up with a couple of screens which had card readers attached to them. You could insert one of their sample cards and a screen popped up running Windows and a couple of applications. When you removed the card, it was gone. You could then walk to one of the other terminals and insert the card - and there was what you were last working on (instantly). It was pretty neat, and the concept was simple. Running on the back end was a heavy duty server which was emulating a dozen or so Windows machines. The front end was a dumbed down Linux terminal which just connected the user to the virtual Windows machines by means of a remote connection protocol (RDP). There was a little more to it, such as strong certificate based security, but we won't tackle that just yet.

My plan today was to create a collection of virtual Windows machines, and a USB "key" which could connect me to one simply by inserting it to a workstation.

The Server: In my case, this was easy since I all ready have a Linux box running VirtualBox. If you want to create this environment, go on out to VirtualBox.org and get yourself a copy. Note that it's *FREE* to those who qualify (read the fine print). Also, there is an Open Source Edition which is free to everybody, but it lacks some key features like USB support (so avoid it for this discussion). Once you have VirtualBox you will want to create at least one workstation. This can be anything really. In my case, it was Windows XP. In the settings for that workstation you will want to go into Settings and then Remote Display. Enable remote display and set your port number (default will be 3389).

The Workstation: I am referring here to the "dumb terminal" that you will be using. This should be on the same network as the server (or there should be routing established between them). Nothing needs to be done special on this workstation. It should be running Windows for our discussion. In my case I am using Windows XP boxes.

Prepare an RDP File: This can be done on any Windows machine. Basically we just want to make a settings file that we can put on our Thumb Drive. To create this, get onto a Windows PC and click Start > All Programs > Accessories > Communication > Remote Desktop Connection. Enter your IP and port number like this SERVER:3395. If you used the default port of 3389, just enter the server name. You can specify all sorts of other info here if you want. Many of these settings have no bearing since you are connecting to VirtualBox, and not "Windows itself". When you are done, choose to save your settings. Save this right onto your thumb drive and call the file "connect.rdp".

The Thumb Drive (or Jump Drive): This is where all my work came in. You will need to create a couple of batch files on the root of the thumb drive. Here is what they are named, and what should go inside of them ....

autorun.inf - This will initiate your remote client upon plugging in the Thumb Drive.
[autorun]
open=connect.bat
ACTION = LAB CONNECT

connect.bat - This is required to launch the RDP session, and the "watcher".
SET STARTRUN=%0
COPY %0\..\*.* C:\TEMP
C:
cd C:\TEMP
start /normal RDP.bat
start /normal WAIT.bat

RDP.bat - This will launch the remote window and ultimately quit.
mstsc /f connect.rdp
EXIT

wait.bat - This will watch for the removal of the thumb drive. If it's removed, the remote session is closed within 3 seconds.
GOTO CHECKEXIST

:CHECKEXIST
IF EXIST %STARTRUN% GOTO WAIT
GOTO KILLTASK

:WAIT
PING -n 3 127.0.0.1>null
GOTO CHECKEXIST

:KILLTASK
taskkill /f /im "mstsc.exe"
EXIT

With all this in place, here is how it will work.

When you insert your thumb drive, Windows XP will find the autorun.inf file and use it to launch an "Autorun list" in Windows XP. All you should have to do here is press enter (for security reasons this choice cannot be made automatically). At that point, you should see a remote connection window pop up. This whole process takes a few seconds.

While you are remotely connected, there will be two Command Prompt windows lingering in the background. One is just running the RDP application. The other is running a watch on the thumb drive. If you watch it, you will see that the PC pings itself three times, sending the result to "nowhere". The reasoning behind this is to give the PC something to do to waste time. Windows XP does not have a sleep or wait method that you might use to waste time cycles. Every time it completes it's three pings, it will check for the existence of the drive letter being used by the thumb drive. Through some clever tricks involving the "%0" variable, we are able to determine this drive letter regardless of what was chosen when it was inserted. If the drive letter is gone, the batch process hunts down the RDP task and kills it, then ends that script by exiting. The other script which had been running the RDP task moves to the next line, which tells it to exit also. The result is, the remote connection window and all it's friend vanish almost the instant the the thumb drive is removed.

You will see that my scripts first copy themselves to C:\Temp before running. The reasoning behind this was that if the drive is removed while a batch script is running from it, the script will fail and leave a "Terminate Batch" prompt on the screen. A colleague noted that in a production environment you would probably want yet another batch file in this process which removes all these items from Temp once it's done running. But it's a work in progress.

Next, I will focus on adding some form of security to this process as there presently isn't any.

Poor Mans Low Level Format

On occasion I am asked to "blank out a device" or remove any data it contains. Usually this is because we are disposing of media or we are selling off equipment at the hospital which once may have contained a patients medical records. This seems to be an easy task, with a complicated solution. The goal is to "write zeroes" to the hard drive repeatedly. This is affectionately referred to as a low-level format.

Sure, there are utilities to perform this task. Some are free, while others and get quite expensive. I also seem to run into problems where certain utilities only work with certain drives (a Western Digital utility only works with Western Digital drives).

Enter the simple and free solution: Linux.

I have several different versions of Linux laying around. Old versions of Ubuntu, new versions of Xubuntu, you name it. So here is what I did.

1) Insert your live distribution of Linux, and boot to it (this may require changing BIOS options, or changing boot options).
2) Wait for the desktop to appear, or fail to appear. I was working with some bizarro medical machines today which failed to boot completely and instead dumped me into "BusyBox". BusyBox is like a small shell which can only execute very minimal commands. But this will do.
3) If you boot all the way to a graphical desktop you can either open a Terminal window, or press Ctrl+Alt+F1 to get a virtual terminal.
4) Enter this command: cat /dev/zero > /dev/sda (or /dev/hda for older IDE drives).
5) Wait for the error message, "No space left on device".

The error message is inevitable. We are simply running a contents list of an imaginary device called "zero" which is filled with an infinite amount of zeroes. Then we are redirecting that stream of zeros right into the hard drive device ignoring all boundaries, partitions, master boot records, etc. Eventually we strike the end of the drive and it tries to keep going, hence the error message.

If you want to follow the old "D.O.D. Standards" you will want to repeat this low level format at least 6 more times (if not 9 more). You can run this command repeatedly by separating your commands with semicolons. For example ...

cat /dev/zero > /dev/sda; cat /dev/zero > /dev/sda;cat /dev/zero > /dev/sda

... would perform three consecutive low level formats. Sure, you could write this into a shell script. But then we are talking about something quick and dirty here which you can do simply by booting whatever distribution of Linux you have laying around.

Disclaimer: A purist might say "That's hogwash Steve! That data is still retrievable by using a chemical separation process on the platters". To which I would say, "then take them home and prove me wrong". Yes, data could still theoretically be retrieved from these disks ... if you have a laboratory environment, or the money to pay someone retrieve it. If you are really paranoid, consider alternating between writing zeroes to the device, and writing random data to the device. This can just as easily be performed with ...

cat /dev/urandom > /dev/sda

They say that by alternating and randomizing the data that you write, recovery becomes all the more impossible.

Thinkpad x60 Booting Disaster

A while back one of our doctors at the hospital bought himself a ThinkPad x60. At the time, it was about the most portable model that you could buy without sacrificing speed and extra memory. Yet - it was a ThinkPad. I will spare you the soapbox essay on why I believe that ThinkPads are crap. If you are using a ThinkPad and you you "just love it", good for you. But heed my warning: keep your stuff backed up, because your hard drive will fail in a year (or maybe sooner). And don't even get me started on the "lenovo" brand - which produced this gem. I think lenovo must translate into Chinese as "cheaply manufactured crap".

The good doctor brought me his laptop with a failed hard drive. That was no surprise. It was very well covered under warranty, but I had a hell of a time finding a number to call on the lenovo support site. IBM seemed to have disowned anything that lenovo produced, so they were not offering anything but a redirecting URL. Eventually I called a "paid support line" where I would be expected to put $70 on a credit card for a one-time support request. Knowing full well I wasn't paying a dime for something covered under warranty - I bounced around in their phone system a couple times and eventually found a live person. Lo and behold, this was the right department and they were able to send me a new drive. To their credit - it arrived less than 24 hours later.

All I had to do now was to restore the ghost image I took of the failing drive. I went through the usual process only to find that the PC was not going to boot. This was not all that shocking seeing how I imaged a failing hard drive and probably picked up a few errors along the way. All I really needed to do was to boot the Windows XP SP2 CD, and slip into the Recovery Console. From there you can run a "fixboot" and "fixmbr" to put things in order. There was just one problem ... this model has no CD-ROM drive.

Following what seems like poorly written instructions - I was able to slap together a Windows XP SP2 bootable ThumbDrive image, using this guide. While I was able to boot from the USB stick, I was not able to get past the "Setup is starting Windows" before it would blue-screen on me with a stop message. The problem seemed to be that Windows was losing itself, after having booted from the stick.

The solution to that issue ended up being to go into the BIOS of the x60 and setting the SATA option from "AHCI" to "Compatibility Mode". Not sure what that had to do with the USB boot problem, but it worked. No more blue screens. And I was able to start the Windows XP Recovery Console. But here was the other catch - by performing a "fixboot" and "fixmbr" I actually fixed the boot files of the laptop hard drive, but then BROKE the boot sector of the USB stick!

Lessons learned. Who knew that these old Recovery Console commands had arguments and switches. After rebuilding my thumb drive (there's an hour lost) I was able to get back to the Recovery Console and run both commands with a drive letter. That is, "fixboot c:" and "fixmbr c:".

Now - I am back in business with a booting, working, and updated copy of Windows XP. I went ahead and set the BIOS options back to default for the SATA controls, as I don't know if that really has any effect on how the drive is accessed. You had better believe that I am taking a ghost image of this while it's working. This hard drive will surely fail in another year or so.

Automatic Photo Resizing

Last year we took a family vacation to Hatteras Island. Being the geeky guy I am, I wanted to take all of my digital photos and upload them to an online gallery while I was still vacationing. Then friends and family could see all the beautiful things that we were seeing as the trip went on. This gallery can be found here.

While I enjoyed sharing the pictures, I didn't enjoy all the time it took away from my trip. I had dragged along my old iBook for the trip and it was not very efficient at getting images off of my camera (it took an hour or more). Resizing the pictures was extremely slow due to the low processing power. Uploading the pictures was painful because the beach house had a highly unstable DSL connection which was constantly disconnecting mid-image transfer.

This year will be different! In a few weeks we will be driving out to Beavers Island Michigan on a family getaway. I will be taking along my Asus Eee PC for image transfer. This laptop has an SD Slot in the side which matches the memory card style that our camera uses. So after a long day of hanging out at the beach or taking adventurous hikes - I can take the card out of the camera and stuff it into my laptop. I will use my cell phone, and a USB connection to my laptop to provide myself "modem speed" Internet access so that I can upload my photos.

But wait - I can't upload these pictures in their native resolution (they're too big). So I wrote a script to use. This script requires that you have imagemagick installed, which can be added with "sudo apt-get install imagemagick" if you are running Debian/Ubuntu/etc.

# Image Resizing Made Fun
# http://steveballantyne.blogspot.com
#
# Set this as the path to where your picture card is/gets mounted
cameracard="/media/disk"
# If you want your resized images to have a prefix, set that here
prefix="beavermi_"
# Create a subdirectory in your home directory, which will contain
# folders with the MMDDYY directories beneath it.
subdir="to-post"
# Choose your resize percentage
resizevalue="25%"
# Create a folder under subdir with a date code in this format: MMDDYY
newdir=`date +%m%d%y`
mkdir -p $HOME/$subdir/$newdir
# Perform all the resizing work
for i in `ls $cameracard`; do convert -resize $resizevalue -quality 80 $cameracard/$i $HOME/to-post/$newdir/$prefix_$i; done

In a nutshell, it does a listing of all the pictures on my memory card. Then one by one it resizes the pictures to 25 percent of their original size, at 80 percent the quality. I am using 'convert' and not 'mogrify' here. The difference is that this will not affect the original photos. When done, I will have a folder of pictures that will be small enough to share in my online gallery.

I also wanted to script the process of uploading the pictures ... but this proved to be a bit more difficult. There are scripts out there for uploading to online galleries but none for Blogger/Picassa.

Perhaps I will work on that later.

Green Wallpapers

Do you like green wallpapers? You are not alone. There seems to be an abundance of lovely chloroform laced wallpapers out there. Some of the extreme close-up photography has got me asking "did nature make that?". Whether you are just celebrating the outdoors or you are sick of that ugly field in Windows XP ... here are some green wallpapers that are sure to please.

I have linked all of these thumbnails to their homes on Interfacelift.com which features some of the most stunning photography I've ever seen. And yes, they have wallpaper in your size (including wide screen displays).



Still need more green? Try this search!

*NEW* I will be getting an EEE PC soon to use at work. If you have an EEE PC, and are looking for some nice green wallpaper, look no further. Download this zip package which includes all the above wallpaper which I have scaled and cropped to fit the 800x480 dimensions of the screen.

-Steve Ballantyne

Fun with Screenlets in Ubuntu

If you're not big on eye candy, this software likely won't "do it for you". In the spirit of things that you really don't need cluttering up your desktop - I bring you "Screenlets". The idea is that a Screenlet is a desktop widget. These widgets range from things that you might want to keep at arms length, to silly gadgets that might entertain the duller moments of your day. These widgets can be "locked down" so that they become active elements of your wallpaper (more or less). If you have used Windows Vista or modern versions of OS X you should be familiar with this concept. Most folks keep their widgets floating around the right edge of their screen.

Admittedly this whole concept is growing old but I was impressed with how easy it is to install Screenlets in Ubuntu, add a collection of fun gadgets, and perhaps even create your own. Screenlets are written using the popular Python scripting engine. While I have had bad luck with similar widget engines in the past, it seems that using common libraries and Python elements has allowed the desktop widget concept to become stable and usable. I have played with these same widgets for several days and have yet to crash anything, or suffer through any error messages.

You can install Screenlets in Ubuntu with a single command in terminal:
sudo apt-get install screenlets

The above will also add all the supporting libraries, and the default "base" of Screenlets to begin using. It's likely that whatever you had in mind comes in the default Screenlet collection. If not, there are few hundred more available from http://www.screenlets.org. There's a large community of widget developers and users out there sharing their ideas.

Once you have installed Screenlets, you can begin adding your widgets using the Screenlets Manager. To access the manager, click System > Preferences > Screenlets. You may also use a "Screenlets Daemon" icon which should appear in the upper right hand corner of your display near the time and volume control slider.

Here you can select a Screenlet, and click "Add" to the right. It should immediately appear on your Desktop. From there you should be able to slide them around by clicking and dragging on them. With a right-click you will find options to change the size of the widget, decide if they should linger on top or underneath your windows, and even "lock" them into place to prevent the accidental sliding around of them. I installed a few common Screenlets and took this screen shot.



These widgets are pretty large on my desktop. The only one I really refer to often is the old analog clock. It's nice to have an analog clock, with the correct time displayed on it. We actually have three analog clocks in my department at the office. They are all battery operated, and each keep their own time. One has recently quit all together, and rather fuss with it I have decided that this clock should display "important times in nerd history". This weeks theme was Back to the Future. I'm waiting for the strike of lightning that will take Marty back to 1985.



Now, go get busy cluttering your desktop with stupid eye candy. It's time well wasted.

-Steve Ballantyne

Windows XP Service Pack 3 - Choose Your Browser

Having applied the newly released Windows XP Service Pack 3 to my virtual administrative machine yesterday I can now safely say "mistakes were made". This morning I spent a good couple of hours trying to determine what went wrong, and how to correct it. I hope you can learn from my mistakes. This is what I discovered.

1) Once you install Windows XP Service Pack 3, you will never be able to change your browser. If you were running IE 6, you will be stuck with it from here until re-installation (that's a joke). If you have IE 7, you can never revert back to IE 6. That "uninstall IE 7" ability will be removed upon successful installation of Service Pack 3.

2) If you were running IE 6, and then installed SP3, you can never have IE 7. Attempts to install it manually will fail.

3) If you are stuck with IE 6, you cannot install post IE 7 updates, for which there are about three. You would think Windows Update would recognize this flaw and not attempt to install these updates that you cannot even accept, but that is not the case. If you get stuck in this loop, you will have 3 updates that are pending installation ... forever. Every time you update, it will fail. Every time. Sucks to be you.

4) If you install Windows XP right out of the box (like a normal clean install) and run through the updates that are recommended to you - you will effectively screw things up. That's because Windows Update will not offer you the IE 7 update until after you have installed SP 3 and it's too late.

With all that said - here is how you *should* update Windows XP. First, install Windows XP Service Pack 2 if it's not all ready streamlined into your product disc. Then run Windows Update and grab the dozen or so updates that are offered, and reboot. Run Windows Update again. If it offers you Service Pack 3, stop there and don't install it. Now browse to the home for IE 7 and choose to "Download Now". You want to install it manually. Once that completes, reboot. At this point you can run Windows Update and accept the Service Pack 3 update.

Hey Microsoft, what the hell are you thinking?

-Steve Ballantyne

Watch Movie DVD's in Ubuntu

Due to some rather silly and lengthy legalities, Ubuntu Linux does not install in a state that allow you to watch a regular old DVD movie. I have tried without success on multiple occasions in the past to insert a disc to watch - only to have it fail. It never really occurred to me that this was "by design" and I always assumed that it must have been something I was doing wrong. Recently I found that I could play any of the movies that I had created (without encryption) but not a single commercial disc.

After some investigation I learned that you can enable the ability to watch commercial discs rather easily, but it's likely illegal (read the fine print).

Rather than act as if I came up with the solution, I am pasting a link to a pretty good article on the subject matter. While the article was written on earlier versions of Ubuntu, this trick still seems to work on the latest release (Hardy Heron).

How to enable DVD playback

-Steve Ballantyne

Scheduled and automated Secure FTP (SSL)

A few months ago I had a request from one of our department members to automate a tedious process. One of our health systems crank out a plain text file which on a daily basis needs to be sent off to an insurance carrier. The insurance company uses FTP to facilitate the "upload" of data, but there is a catch. Because this information contains patient health data, you would not want to send it in plain text. Their method of securing the data is to use certificate based secure FTP. I should state that there are other methods of securing FTP traffic, namely with SSH tunneling. Using certificates for an FTP server is not common practice, and so automating it was difficult.

Here is the script that I created which happily runs on an Ubuntu Linux box. Note that names, passwords, and the like have been removed to protect confidentiality.

#!/bin/bash
#
# HL7 FTP drop
# By Steve Ballantyne 02-28-08
#

# Calculate the date in specified format and make a directory with it
TIME=`date +%m-%d-%Y`
# Build working directory
if [ -f /data/healthinsurance/README.txt ]
then
sleep 0
else
mkdir "/data/healthinsurance/$TIME"
fi

cd "/data/healthinsurance/$TIME"

# Check for SERVER Share
if [ -f /mnt/healthinsurance/README.txt ]
then
mv /mnt/healthinsurance/Submissions/healthinsurance/working/KCHDEM* /data/healthinsurance/$TIME
else
#Map to SERVER drive
mount -t smbfs -o username=ballantynesd,password=password "//SERVER/e$" /mnt/healthinsurance
# Now move it
mv /mnt/healthinsurance/Submissions/healthinsurance/working/KCHDEM* /data/healthinsurance/$TIME
fi

# Go put the file on the server
curl --ftp-ssl ftp://ftp.generichealthcare.com -u username:password -T /data/healthinsurance/$TIME/KCHDEM*.txt --insecure
# Done - Exit

Here's the play-by-play of what this script does.

1) A check is made for a README file. If this file doesn't exist, someone may have messed with our directory structure.
2) A date variable is created in the form of MM-DD-YYYY. Then, a directory is created using that variable, and we change directory, or 'cd', into it.
3) We check to see if a README file exists on a shared drive, which has been mounted. If it does not exist - then the drive has come un-mounted so we run through a routine to re-mount the Windows share.
4) Once the mounting issue is resolved, we move the file from the SERVER into that local folder we created with the date variable.
5) With the file in our grips, we use 'curl' to send the file to the health insurance company. Note that we needed two special switches: '--ftp-ssl' and '--insecure'. The '--insecure' was required in my case because the certificate being used for this connection was self-signed.

Now that we understand what we are doing, how will we automate it?

I saved the above script into a file called /root/healthins.sh. Then I did a 'chmod 755 healthins.sh' so that it was executable. Next, I became root by using 'su -' and supplying the root password. Finally, I ran 'crontab -e' which allowed me to edit the root users cron entries.

This is the line I added to the bottom of my scheduled jobs (it was empty).

1 1 * * * /root/healthins.sh >> /root/healthins.log 2>>&1

That line says that on the first minute of the first hour of every day ... run the script called 'healthins.sh' which exists in the /root directory. Then, redirect all of the output from this process into a log file called 'healthins.log' which also exists in root. The '>>' indicates that this file should be appended. That is, *added to* and not overwritten.

I'm happy to say that this has been running well, aside from a mistake I had made in my logging process. But I figured since I was in fixing this - I should document this process for future generations. If this helped you in some way, won't you please drop me a line?

-Steve Ballantyne

Dress up Ubuntu 8.04 Hardy to look like gOS Space

The nice people over at gOS (read: good Operating System) have released their latest project called "gOS Space". Think Ubuntu, if you layered it up with some eye candy, and then added a nifty dockbar and populated it with MySpace and YouTube Launchers. I have to admit that the visual appearance is pretty good. It's also oddly familiar, because with all of these various pieces mixed together, it resembles the latest OS from Apple.

I downloaded it and tried to boot it in a virtual PC (using VirtualBox). I failed. I also tried burning it to a CD-ROM and I failed at that too. That's because the ISO image is about 68 meg's off from fitting on a standard CD. So I grabbed a DVD and burned a copy. Then I booted it up. The "beauty" of gOS is that you don't have to install it. You can boot to it, and try it out. I tried it. I played with it a while. I was ready to get back to Ubuntu which I had all ready installed, but I wanted to take this nice theme with me.



Here's how you can re-theme your Ubuntu 8.04 Hardy with the pretty space theme.

#1 - Get the wallpaper
If all you wanted was the wallpaper, this is an easy task for you. Because I have mirrored it. you can right click here and download a copy. Then make it your background. You're done!

#2 - Install the wicked green bubbly window theme
One of the niftier looking aspects of this theme are the windows. You won't really see what these look like unless you do some digging. The screen-shots on the gOS site don't show them. I borrowed them the best way I knew how. Find the directories ... tar-ball them up ... and carry them over.

* Right click here and Download this file (save it anywhere).
* Move the file to the root of the file system, and un-tar it. In a Terminal ...
cd Desktop
sudo mv newgosspace.tar.gz /
cd /
sudo tar -xzvf newgosspace.tar.gz
* Apply the theme you just installed: click System > Preferences > Appearance. Then select the "gOS_Theme". Click "apply".
* You're done. Everything is green.

#3 - Install the pretty Dock Bar
gOS makes use of the "Avant Window Navigator". This has been around for quite a while, and can be installed relatively easily to Ubuntu Hardy. But ... the version that they are using for gOS is slightly newer than what you would get from the Hardy repositories. Trust me, you will want the latest stuff if you want the full effect. Namely, what the new version offers over the old one is a big collection of "applets" for your dock-bar, and that neat looking "tilted tile" of your icons (as seen in the screenshot).

If you want to live on the edge, and install the latest stuff ...

* Open up /etc/apt/sources.list and add this line to the bottom:
deb http://ppa.launchpad.net/reacocard-awn/ubuntu hardy main
You will need roots authority to save this file, so you might want to run something like "sudo gedit /etc/apt/sources.list" to open it.
* Now, in a Terminal run "sudo apt-get update".
* Also in a terminal, run this ...
sudo apt-get install awn-core-applets-bzr avant-window-navigator-bzr python-alsaaudio python-awn-bzr python-feedparser
* You will be asked if you want to install all the above, plus some dependencies. Say Yes!

You are almost done. But here is the problem: you have no "preferences manager". Why didn't you install it in the above command? Because it won't work. Basically, you are mixing old and new parts of this application together, and Ubuntu doesn't like it.

* Open this web page, and find the version of awn-manager-bzr_0.3.1.bzr234.1~hardy that matches your version of Ubuntu and PC type. If you are running Hardy and have a standard Intel i386 version, you want to download this one.
* Find the file you just downloaded, and open it (double click it). Then, click the green Install button to install the package. You may be told that this same package exists in your repository. You should ignore that message. It's referring to an earlier release of that same package.
* Click Applications > Accessories > Avant Window Navigator. It should launch at the bottom.

If you were using a "bottom Panel" you now have some real redundancy going on here. You may move that bar to the right side of the screen (click and drag it) or you can right click on it and "delete it". I suggest you keep it around in case you don't like this dock bar (it's been known to be buggy, while it's been very stable for me).

Now you have some configuration to do. If you right click the left most portion of this Avant bar (there's an empty little slice there) you can select "Preferences". From there you can add applets, and the like. I suggest you add the "Stacker" applet (one for each stacker menu you want to create). Then you can right click on each Stacker (as it appears in the dock bar) and click "Preferences". If you would like to use the "visual styling" that I created for my dock, you can download this file. Just save it somewhere handy, and in the dock preferences, click the Themes tab, then Add, and browse to that file. I found the manager to be a bit buggy with themes. If you add the theme, and it doesn't show up in the list - close the manager and re-open it.

If you want the slick Apple look for a Stacker... click the Backend tab and choose "Folder backend". Point it to a folder which contains shortcuts. If you don't have a folder of shortcuts - create one and come back. Click on the Applet Layout tab and check "Composite applet icon". Click the Stack Layout tab and change the Layout to "Curved gui". You may want to fiddle with the other options and tweak them to your liking.

Now you can run full fledged Ubuntu, with all the style of gOS Space! Enjoy.

-Steve Ballantyne

Nightmares with Exchange 2003 Free/Busy Time

I hadn't been in my current position long before I started noticing strange problems with our Public Folder structure. When I accepted my position as the Network Administrator I became the lucky winner of a battered 2003 Exchange server which was migrated over from version 5.5 back in ... 2005 or so.

I suppose the first sign that something was wrong was that there were strange little entries in the Public Folder list that didn't do anything. When you clicked on them in Outlook a message said "Unable to display the contents of this folder". I used to wonder what was in there and why I couldn't see it. But after asking around I could see that these were "artifacts". That's a nice way of saying, "junk that was left behind and for reasons beyond explanation - they will remain to exist for the life of the server". I tried to delete them. Operation failed. Crap. Now I didn't really care what was in them. That fun curiosity had left me. I just wanted them to go away.

And then came the Free/Busy issue. I had one of our more important and highly scheduled administrative folks ask me why people could schedule her to meetings when she was clearly booked solid. It only took a little bit of clicking around to see that something was wrong here. She was booked all day long but her "free time" only reflected an hour of busy time. I began with trying all the easy fixes, starting with "outlook.exe /cleanfreebusy" from the command line. It ran, without error. It fixed nothing.

Then I stared digging through an endless search of forums, newsgroups, and "knowledge-less bases". It seems I am not alone in my quest for operational Free/Busy functionality. There are many out there that like me are having the same types of issues. I saw a lot of people asking "can't I just delete this Free/Busy time and start it over"? The answer is, no. Because Free/Busy time is a system folder which lives in Public Folders, it isn't easily accessible by anyone. From within the Exchange System Manager you can navigate to it, play with it's simple permissions, and check it's replication. But that's about it. Oddly enough, I didn't really see anything wrong with this folder, despite the fact that it was totally broken.

I also publicly displayed my frustration in a newsgroup, which bared no helpful advice whatsoever.

After reading techno-babble off and on for the better part of two days I came across something of interest. It asked me to check an attribute on the Exchange server using the adsiedit.msc tool. Lo and behold I had located a problem. A particular attribute still held a link to my dead Exchange 5.5 server. Fixing this broken link would theoretically release, recreate, and reattach my Public Folder infrastructure. And so the following weekend I stayed up late drinking diet soda and hacking up my server only to find that the problem STILL EXISTED.

So here is the solution for anyone else that might end up in this mess. First I should mention that I have held back on releasing this entry until today (even though I performed this work nearly a month a ago). The details are a bit fuzzy to me now, but I didn't want to post a solution that wouldn't work. Today, I can honestly say that everyone's Free/Busy time is in good standing - and all those odd-ball Public Folders have been done away with.

Following this procedure will blow away your Public Folders completely, leaving nothing behind. The majority of this process covers how to backup and restore the data that your users will want back. This is a risky procedure, so if you try to do this and break things really badly - don't come looking for me. You have been warned.

Another important note: Performing this procedure will break "favorites". That means if your clients have opened up Outlook and said "add this folder to Favorites", they will now have a broken link. Even if your folder comes back with the same name and the same location - the shortcut will still not work. They (with your help) will need to recreate all of those shortcuts. Expect calls. Clicking on a dead shortcut will cause Outlook to crash!!

Step #1 - Make a backup. If you use Veritas, Symantec, or something of that nature - make a full backup of your Public Folders now. Hope you will never need it. In fact, try not to ever use it. Refer to the notes at the end of this post.

Step #2 - Back up the Public Folder permissions. Get a copy of the Microsoft Exchange Server Public Folder DAV-based Administration Tool. Install this tool (it extracts to a folder) and run it. Then click File > Connect. Enter the properties for your server, and run this as someone with Administrative access. Make sure that the radio button option is selected for "Public Folders". Now you should be able to expand Public Folders, and see them all listed in the left pane. Click on the very top item "Public Folders", and then click Tools > Export Permissions. Leave things at default, and click OK (you may have to set up a log file, so create one if prompted). This will create a text file with all of the Public Folders names, and all the permissions to go with them. In my case, I then opened up this text file ... went down to where it switched from "real" public folders to invisible System Folders. Then I *deleted every line* which referred to System Folders. You should do this too. Problems with Free/Busy could be related to incorrect permissions being applied to your folder set. You do not want to re-import those faulty permissions back onto a healthy Public Folder store.

Step #3 - Back up your Public Folder data. I did this the old fashioned way. By that, I mean that I opened up Outlook, expanded Public Folders, then selected All Public Folders. Then I performed a File > Export, and exported *everything* in Public Folders to a PST file. The danger in doing this is: you cannot back up folders that you have no permission to. So if someone has excluded you access to a folder, it will not get backed up. That could get you in trouble. Compare what you see in Step #2, with what you see inside of Outlook. Make sure you are not missing anything. Also know, this could take *HOURS* depending on the amount of Public Folder data that you have. In my case the store was a little less than 700MB and it took 45 minutes.

Step #4 - Remove Public Folders. To do this, go into Services on the Exchange Server and stop the Information Store. Now, browse to where the Exchange data files are physically stored. Usually this is in x:\Programs Files\Exchsrvr\mdbdata. There are two files; pub1.edb, and pub1.stm. Rename these files - but do NOT delete them. I just added an .old extension to them. Now, go back and restart the Information Store service. This will cause chaos and confusion to your Exchange Server. It should give you a bad news message and ask if you want to create a blank Public Folder set. Say yes. Congratulations, you just destroyed all of your users data. Better act quickly on step #5.

Step #5 - Put the Public Folder data back. This is the reverse of exporting. Go into Outlook, Expand Public folders and notice that it's empty. Now, import your PST back to Public Folders. Note that there is a trick to this! The trick is, you cannot import back into a system folder without Outlook telling you to "stop doing that". What you can do, is expand Public Folders, then expand All Public Folders - and then start the File > Import wizard. At the second or third step where you tell it where you are importing to - select "to currently selected folder". You will also notice that in your PST file this subtree has some bizarre name like "IPM_NON_SUBTREE". Don't worry about that. It will restore to where it needs to. Watch the files copy. When done, make sure things look okay.

Step #6 - Still awake? Now fix the permissions. Open up your tool from Step #2, and click File > Connect. Again, fill in your server properties and make sure Public Folders are selected. Now select Public Folders and then Tools > Import Permissions. This should go pretty quickly. To see if things worked, you should go right clicking randomly on your folders and make sure that the permissions look right. You can also check the log for this tool you have been using.

Step #7 - Update everyones Free/Busy time. The best way to do this is to send out a mass meeting notice for a "fake meeting". You can put it for a Sunday at Midnight, make it last 10 minutes, and put the location down as "fairy land". What's important is that everyone in your organization gets it, and agrees to the meeting. Doing so, will reset their free/busy data on the server. You can also recreate this data by having each user run Outlook from a command line with the /cleanfreebusy switch. Good luck with that! I used the fake meeting method, and it worked wonders.

That should be all there is to it. But there are some ...

Possible Pitfalls!
Free/Busy still not accurate - Let's say that you have imported everything, fixed all the permissions, and the Free/Busy is still whacked. Take a moment to think about this. We have fixed all future appointments, but existing ones may still be a problem. I found that if you delete a reoccurring appointment and recreated it, the free/busy became accurate. Also - waiting longer seems to work. I waited about two weeks and everything seems correct. What fixed it? Who knows. It's Exchange Server.

Folders are missing - If someone had a folder which you could not access, than you probably didn't back it up. Way to go! The good news is that you *renamed the data files*, you did not delete them. The bad news is that you will have a hard time getting the data out of them. If at all possible - do NOT restore from a tape backup. The best thing you can do is use a tool to extract the data from the public folder files. One such tool is called OnTrack PowerControls. It's expensive to buy, but you should be able to use the trial version to extract from a detached Information Store database file. Basically you need to extract the data into a PST, and then import that PST back to the Public Folder tree. If you are stuck doing this ... read the manual for the PowerControls product. ;-)

I sincerely hope that this information comes in useful to someone, some day. It took me a few weeks of off and on experimentation to come up with this. If this helped (or harmed) you won't you please drop me a comment and let me know?

-Steve Ballantyne

VirtualBox with Multiple Bridged Network Interfaces

Several months ago, I made the switch from VMWare over to VirtualBox. It didn't require a lot of arm twisting. VMWare costs around $500-$600 (for a basic Workstation license) and VirtualBox is absolutely free. While VMWare is a far more robust product, I don't really use most of the advanced features that justify the inflated cost.

Now - many months later I have run into a dilemma with VirtualBox. I want to have two virtual machines running (simultaneously) which can both access the network using IP addresses which they have obtained through DHCP. Setting up a single workstation proved to be quite a challenge, and two required a lot of reading and digging. VMWare definitely makes virtual networking easier - at least on the Windows side of things. I am, of course, running Ubuntu Linux natively and virtualizing all my Windows Operating Systems with VirtualBox.

Here is the script, which made this all possible for me - with comments to follow. In this example, I have TWO physical network cards. eth0, and eth1. eth0 connects my host (the Linux box) to one network for Internet connectivity, etc. in Linux. eth1 is connected to our production network, and will be used solely for my virtual guests.

Note: You will need to install uml-utilities and bridge-utils first. Do that with: sudo apt-get install uml-utilities bridge-utils.

modprobe tun
tunctl -t tap0 -u ballantynesd
brctl addbr br0
ifconfig eth1 0.0.0.0 promisc
brctl addif br0 eth1
ifconfig eth1 up
dhclient br0
brctl addif br0 tap0
ifconfig tap0 up
chmod 0666 /dev/net/tun
# This was added Apr 22 2008
tunctl -t tap1 -u ballantynesd
brctl addif br0 tap1
ifconfig tap1 up
echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/tap1/proxy_arp

The above was saved into a file, which should be run as root *before* starting VirtualBox. You can do this with a 'sudo vboxup.sh', or similar. Exchange 'ballantynesd' with the user name that you are running with on your Linux box. Exchange eth1 for your production NIC, whatever that happens to be.

To complete setting up your virtual guests, you will need to shut them down, open up the settings for them, browse to network settings. Change from "NAT" to "Host", and in the lower area set the network card to tap0 or tap1. Leave the rest alone!

With luck, and prayer - you should be able to boot up your virtual machine and obtain an IP with DHCP (or assign one statically if you like).

Good luck! Give me a shout if this should help you out.

-Steve Ballantyne

EDIT: 05/08/2008

It seems that an upgrade to Ubuntu 8.04 LTS, and an upgrade to the new "Sun" branded VirtualBox 1.6.0 ... is not a good idea. I have tried for the past two days to make things work as documented. Namely, the nice little bridge that I had going on. Following the prescribed documentation got me nowhere, so I reverted back to configuring my interfaces "the old fashioned way" and used the above script. My new script is for a single Virtual Box, and it looks a little something like this ...

As stated earlier - this script must be run with 'sudo'.

# Don't need these, so they die
ifconfig vbox0 down
ifconfig eth1 down
# Throw up a bridge
brctl addbr br0
# Add my main card to the bridge
brctl addif br0 eth0
ifconfig eth0 0.0.0.0 promisc
# Bridge goes up
ifconfig br0 up
# Bridge obtains an IP address
dhclient br0
# Give me a virtual adapter
modprobe tun
tunctl -t tap0 -u ballantynesd
# Add the adapter to the bridge
brctl addif br0 tap0
chmod 0666 /dev/net/tun
ifconfig tap0 up


Good luck!!

Connecting RightFax to SharePoint Data

If you have set up SharePoint “correctly”, then everything should be going into two SQL databases. One to hold the configuration notes for the server, and another to hold the bulk of the user data. It’s likely that you have created a few “Lists” for the sake of holding onto (and sharing) contact data.

Why not then – tie your RightFax Phonebook right into your SharePoint contact data?

To begin, you will need to identify the “List” in which you want to query. From within SharePoint, your List has a “name” which you have given it. In my case, it was called “KCH Directory”. SharePoint has an “ID” which it uses to track this data, and we need to locate this string.

Get onto the SQL server which houses your SharePoint data, and run the SQL Query Analyzer. Point the query analyzer to the correct database. Mine was called “KCH_WSS_CONTENT”. Now, enter this SQL query …

select tp_ID from Lists where tp_Title = 'KCH Directory'

… where ‘KCH Directory’ is the name of your list. Then click the green arrow to process the query. You should see one row returned, with the ID we are after. Right click on that ID and “copy” it. Now paste it to Notepad for later reference.



Next, we need to create an ODBC connection for your RightFax server to use. Open Administrative Tools > Data Sources (ODBC).

Click the System DSN tab, and click “Add …”. Scroll to the bottom of the list and choose SQL Server, then “Finish”. Next you will be given a wizard. Name your source whatever you want. I named mine “passql”. For server, enter the name of the SQL server which houses the SharePoint database, then click next. Change the authentication type if you need to (I didn’t) and click next. Make sure you stop here and “Change the default database to” – and enter the name of the database that has all of your SQL data in it. Then click next, change nothing, and click Finish. Finally, click Okay to leave this window.

Now right-click the RightFax system tray icon, and choose “ODBC Configuration”. Then click “Add”.

Now, name this phone book whatever you like and select our recently created ODBC connection from the ODBC source list.

Fill in your fields “appropriately”. You may need to do some mapping here. That means going back to your SQL Query Analyzer and performing a dump of your UserData table with a query like this …

select * from UserData where tp_ListId = 'YOUR-ID-STRING'

In my case, I was able to match up fields in RightFax to these:

nvarchar1 – last name
nvarchar2 – first name
nvarchar17 – fax number
nvarchar11 - location
nvarchar15 - voice
nvarchar6 – company

So entering them into the RightFax ODBC window, looked like this:



Notice that we only have one field for a name, while SharePoint divides it into a first, last, and middle initial. I have combined first and last, and skipped middle initial (creating one solid RightFax field). All of this information will end up on your Fax Cover Sheet (FCS), which is nice.

The “WHERE” box is a SQL statement “where”. What we are saying here is what we want and don’t want from the UserData SQL dump. In my case I am specifying the particular List that I had in mind, minus the blank company names, and fax-less entries. I also (optionally) told it to only show me entries where the fax number started with a left parentheses “(“. That was because someone had typed textual information into the fax fields on some of my SharePoint data.

My WHERE looked like this (photo obscured) …

(tp_ListId = ‘YOUR-ID-STRING-HERE' and UserData.nvarchar17 is not null and UserData.nvarchar17 like '(%' and UserData.nvarchar6 is not null)

The “ORDER BY” allows you to sort your entries. In my case, I just let it try to sort by last name. But this could have been Company Name, or even numerical Fax Number.

Click OK, and try it out! In my case, I had to do a lot of checking and double checking of my SQL syntax. One little typo and your Phonebook will show up stone empty (there will be no error messages).

To test it – open RightFax FaxUtil, and click the yellow address book symbol. You should have a new tab with the name that you used in the ODBC configuration (mine was passql). When you click the tab, the actual query will take place so there might be a slight delay. You should get a nice list of data, and a few boxes overtop to help you filter out what you were after.

Obviously the ODBC connection, and the RightFax ODBC configuration will need to be repeated for your clients. But there is a way to cheat this as well. You can export elements from the registry, combine them into one .reg file, and import it after running through the installation. You can find the Captaris KB article: here.

I certainly hope this is useful to someone else out there. Drop me a line in the comments and let me know! ;-)