Tech That!

At Bond University the IT department has now finally taken the step from an open wireless network with a login proxy to a proper WPA2 Enterprise setup.

Apart from giving much higher security since data is not transmitted unencrypted through the air, this also makes it a lot easier to automatically connect to the internet. With the previous setup, a post_up netcfg hook was needed that used some specially crafted cURL code to post the login form on the proxy. Ugly as hell, and not very reliable or secure either.

The only issue is that WPA2-Enterprise has a lot of different configuration options, so it took a while to figure out the exact setup to use.

First of all I had a look at the configuration options for Windows XP here: http://www.bond.edu.au/student-resources/student-support/computing-support/for-students/wireless-access/index.htm. The advantage of looking at the XP setup guide is that it doesn’t have as much fancy auto-detection as 7 or Mac OSX, so they give some more low-level details that we can base our setup on.

In this setup I decided to use netcfg simply because it provides a flexible utility for managing multiple wireless configurations and switching between them in a simple fashion. Furthermore, it abstracts away some of the uglier command line options of wpa_supplicant into configuration options.

All netcfg configuration files live in /etc/network.d. Each file represents one network, and contains at the very least for a wireless connection an interface specification, the SSID of the network and an IP configuration line. Have a look in the /etc/network.d/examples directory for, well, examples..

For WPA2-Enterprise however, we need quite a few more parameters:

CONNECTION='wireless'
INTERFACE=wlan0
SECURITY='wpa-configsection'
ESSID='BondStudents'
IP='dhcp'
CONFIGSECTION='
  ssid="BondStudents"
  key_mgmt=WPA-EAP
  eap=PEAP
  group=CCMP
  pairwise=CCMP
  identity="<student ID goes here>"
  password="<password goes here>"
  priority=1
  ca_path="/etc/ssl/certs"
  phase2="auth=MSCHAPV2"
'

Quite a mouthful.. In essence, this means we are using WPA enterprise with CCMP (an AES-based encryption algorithm) and MSCHAPV2 authentication with PEAP (TLS) encapsulation. As if that helped..

The most interesting line here is: ca_path=”/etc/ssl/certs”.. Why would we need to import certfiicates? Well, turns out Bond uses a SSL certificate that is not exactly mainstream called “UTN-USERFirst-Hardware”. This certificate is usually included in the certificate directory of most distros, but is not very secure as it is not a trusted signing authority. If you want to know more about this, then Google it.

Well, that should be it. You should be able to run “sudo netcfg BondStudents” if you’ve called the configuration file “BondStudents” and you’ll be online!

Have you ever wanted to watch a video online, but due to a slow connection or frequent dropouts, streaming is impossible to watch. In these cases, there is rarely a “Download” button that allows you to download the entire thing and watch it in full when it’s done. Evidently, this is a real-world application of Murphy’s law.

Here at Bond University, some lectures are streamed and saved for later viewing. These are available to all students from an online interface. The problem is that these videos are streamed (in the proper sense of the word – that is, not that they just play as you download, but as in that the browser has to play back a continuous video stream which causes problems on slow connections since the browser cannot keep up, and has to stop the video all the time and request that the server restart from a previous point in time) even when the lecture has been completed. Even when sitting in the on-campus accomodation, the connection or the server (I don’t know which) is too slow to cope with playing these videos in real time, and such, it is a nightmare trying to watch any of these lectures.

The streaming plays in Windows Media Player and uses a protocol called mms:// (Multimedia stream). VLC and Mplayer can both play this as well, but take ages to load for some reason.

In my frustration, I decided to find out how to download the stream so I can play it without delay, and rewind and fast-forward as much as I wanted. Turns out this is not as straight forward as one would expect with streams.

First of all, the file is never present as a file from the server, only as a stream. This means that you cannot download the video faster than the actual length of the video. A two hour lecture therefore takes at least two hours to download. Furthermore, you cannot simply right-click the video and attempt to get the URL and download that because this will only give you a tiny text file with more URLs.

So, here is what you have to do:

1. Get Mplayer
2. Go to the page with the streaming video on it and right click the video
3. Select properties and copy the URL from the window that opens
4. Open a new tab in your browser and navigate to the given URL
5. Press Ctrl+S, or otherwise save the page
6. Open the downloaded file with notepad, you should see something like this:

   [Reference]
   Ref1=http://straumod.nrk.no/disk02/Lovebakken/2009-09-11/?MSWMExt=.asf
   Ref2=http://10.103.0.56:80/disk02/Lovebakken/2009-09-11/?MSWMExt=.asf

7. Copy either of the URLs
8. Start mplayer with the following parameters: “-dumpstream -dumpfile stream.wmv <URL>”
   For those of you who are not familiar with MPlayer and run Windows, here is how you do that:
   • Press Win+R or press the Start menu and click “Run”
   • Type “cmd” and press enter
   • In the new window, type “cd \”, press enter, type “mkdir stream”, press enter, type “cd stream” – The previous commands made a new folder in the root of your main drive called “stream”
   • Next, to run mplayer: type “C.\Program Files\mplayer and press tab (with the opening quote at the beginning before pressing tab), type “\mplayer” (without the quotes) and press tab again
   • Write a space, followed by the parameters written above (starting with “-dumpstream”), replacing “<URL>” with the URL you copied in step 7
   • Press enter and wait
   • When the program finishes (i.e. the last line says something like “C:\stream>”), you should find the video in the folder “C:\stream” as “stream.wmv”.
   • Rename, play and enjoy!

You may notice that every now and then there appears a post that seems forced, or not in line with the kinds of posts I usually make such as the Javascript and DOM references post. This is because one of my university courses at Bond University is running a Blog Assessment in which we are given certain topics to blog about every now and then. Since these assignments are compulsory, I cannot refrain from writing them though they clutter the rest of the blog entries.

Oh well, just thought I would explain the issue for those who are confused by these seemingly random posts.

The first thing I noticed at Bond, and which continues to confuse me to this day, is the extremely low speed ( and quality – but that is a discussion for another day ) given to each user on campus. Wherever you log onto the network, your connection speed is limited to 1 Mbit/s, which is far below the 20 I am used to back home. The connection is also rather unstable, so you are lucky if you get much more than about 70-80% of that. This makes all downloads a real pain, as even leaving the computer online over the night is not enough.. Several days are often required, depending on what you download ( only legal material of course……. )

Today, I believe I have found away around this problem, provided you have multiple computers available to you. This solution allows to multiply your speed by the number of computers you have, provided you have enough Bond logins for half of them ( i.e. if you have four computers, you will need two logins as each user is only allowed to log in twice ).

The solution is based on the fact that the internal network provides speeds exceeding that of the internet speed, and that most torrent software will attempt to connect to peers as close to you as possible. All you have to do is start up the same torrent on all you computers, and leave it running. The clients will all share what they download between them near instantaneously through the local connection, and at the same time, they will all download from the internet with the regular speed the packets that they cannot get locally, effectively splitting the download. Since each login receives its own speed limit, you are not bound by a single 1 Mbit/s limit, but rather x Mbit/s where x is the number of computers. Quite a substantial increase.. When using three computers, you will decrease the download time by a threefold!

My two cents on downloading for now

This week I have been setting up my new computer – a complete beast with Intel Quad Core i7 processor, 12 GB memory, 4 * 1TB drives in RAID 1+0, etc. On this computer, I decided to put both Windows 7, which is provided for free through the MSDN Academic Alliance and Arch Linux ( which I fell in love with the first time I tried it ).

Since getting an account with MSDN took a while, I decided to put Arch on the box first, even though the Windows bootloader is known to foul up GRUB and make it impossible to boot into linux.

First step was installing Arch.. Usually, this is quite hassle free, but because of the slow internet at the accomodation center at Bond University, I downloaded the Net Install CD so that I could install and download only what I needed. Sounds logical, right? Well, not when I tell you that, as I discovered, you have to login to access the wired network. When opening a browser, you’re presented with a login screen though HTTPS, that has to be completed every time your IP changes. Problem is, the netinstall CD has no browser installed as it is command-line only, and as such, I had no way of authenticating with the network, which again lead me to being unable to download any packages for my system. So, what do you do?

Logging into an HTTPS proxy through command line tools

My first though was to use links or lynx ( text-based unix browsers ), however neither were available on the netinstall CD, and I couldn’t compile either from source since the build tools and dependencies were not there. At this point, I was certain I would have to download the full Arch install CD, and start all over again, however, I was not prepared to give up that easily. There is a reason I use Arch – to understand how things work from the ground up, and to force myself into exploring Linux.

At first, the only solution I could think of was to telnet into the login server over HTTPS, send the proper POST headers by hand, and thereby become authenticated. Finding the correct headers on my laptop was not a problem, however hand-typing them onto the linux shell posed a problem. Not in getting it right, but because the HTTPS connection of the login server had a timeout for requests at about 10 seconds… The end request looked something like this:

POST /login.pl HTTP/1.1
Host: login.bond.edu.au
Connection-type: keep-alive
Keep-alive: 300
Content-type: application/x-www-form-urlencoded
Content-length: 112

_FORM_SUBMIT=1&which_form=reg&source=<my IP>&destination=&error=&bs_name=<Student ID>&bs_password=<URL encoded password>

As you can probably imagine, hand-typing that in 10 seconds is not an easy task. Evidently it was not going to work, which was why I started exploring the unix philosophy of separation of tasks. Why should I type all that text, why couldn’t the computer type it for me? I created a file with the request, and used the unix command “cat” to print the file. I then piped the output through my telnet connection as such:

cat request | telnet login.bond.edu.au 443

To my surprise, this just caused the connection to time out without any error message… After trying a multitude of alternative versions of the above, I concluded that parts of the request was probably printed to the server before the connection was actually established, which caused the server to disconnect the session.

I felt quite lost, and was very close to getting a full Arch install ( and thereby have to wait for about 5-6 hours for the download to complete… ), when I remembered the “wget” command. This is a command that allows you to download files from the web through HTTP/HTTPS/SCP/SFTP/FTP. Maybe it could also send a POST request?

Not only was wget included in the netinstall CD, but after looking at the manpages, I also found the argument “–post-file”, which allows you to send urlencoded data through POST when submitting the request. I was saved! I stripped everything except the data from my request file, and issued the following command:

wget --post-file request --save-cookies s.cookie https://login.bond.edu.au/login.pl

Looking at the downloaded HTML file, I soon found that I had successfully been logged in, and I could start the actual installation!

Both the installation, and the subsequent configuration ( installation of GNOME, setting up drivers, etc.. ) posed no problem as usual, though it all took quite a while having to download it all though the 1 Mbit/s throttled connection at the student residences. Next morning however, my computer was up and running just the way I wanted it. Windows 7 next…

Windows 7

First of all, I had to download Windows 7 from MSDN, which proved to be impossible from linux. It provided a downloader, which, when run through wine, simply refused to download the file properly.. In the end, I had to download the ISO on my laptop ( running Vista ), and burn the DVD from there. From here, the ride was smooth. Installation of Windows 7 was both painless and fast, and I was up and running in 30 minutes. Great!

Next was getting GRUB back on the MBR, since Windows overrides all other bootloades when installed. At first, I tried looking for a windows installer that could restore GRUB to the MBR, but this does not seem to be available, so I had to get down and dirty with the unix command line again. Hooray! =D

I rebooted, and started up from the Arch installation CD. From there, you have two options to restore the grub. Both involve getting your original Linux partition mounted, and then running grub-install from there.

1. Boot the Arch Linux Live CD, mount your linux partition using “mount /dev/sd** /media/fl” where ** is the device and partition of your Linux boot partition. Next, you have to run: “grub-install –root-directory=/media/fl /dev/sd*” where * is the device you wish to boot from..

2. Open the “More Options” selection on the boot screen of the CD, and then highlight the option “[EDIT ME] Boot Linux Directly”. Next, press ‘e’ to edit the line. Here, edit the line “root (hd0,0)” to match your device and partition. Next, edit the two other lines, and change /vmlinuz and /kernel ( can’t remember the exact filenames ) to read “/boot/vmlinuz” and “/boot/kernel” respectively. Due note that these are the default paths, but yours might differ. Also, you might have to change the line that contains “root=/dev/sda3″ to fit your setup. Finally, press ‘b’ to boot the linux partition. You will now find yourself in your normal Linux install. From there, you can run “grub-install /dev/sd**” as in 1.

Why the two options? For some reason I didn’t think of the first option until after I did #2. Maybe it will come in useful at some time…?

Now, to get Windows available from GRUB, edit “/boot/grub/menu.lst”, and uncomment the Windows lines at the bottom, and input the correct device and partition.

And then, you’re done! Congratulations! You’re dual-booting Windows 7 and Linux