Tech That!

For those of you concerned with web usability, there are few better sources than Jakob Nielsen’s AlertBox.

Although his site is quite infrequently updated, it contains very informative posts. Unfortunately and ironically, the site does not have a RSS feed.

After an especially boring afternoon, I decided to see if I couldn’t do something about that, so with the use of this PHP Feed Generator class and some HTML parsing, I came up with the following script to generate an RSS feed from the AlertBox website:

Just download the FeedGenerator FeedWriter class from the given link, put the following code and FeedWriter.php in the same directory, upload it to a webserver with PHP support, and voilla, you have an AlertBox RSS feed.

Enjoy!

<?php

date_default_timezone_set("UTC");
class FetchAlertbox {
    private $contents = array();
    private function fetch() {
        $page = file_get_contents("http://www.useit.com/alertbox/index.html");
        $startList = stripos ( $page, '<ul>' );
        $page = substr ( $page, $startList, stripos ( $page, '</ul>', $startList ) - $startList + strlen ( '</ul>' ) );
        $page = str_ireplace ( '<p>', '', $page );
        $page = preg_replace ( "/<li>(.*)/i", '<li>$1</li>', $page );
        $page = preg_replace ( "@</?strong>@i", "", $page );
        $p = @DOMDocument::loadHTML($page);

        foreach ( $p->getElementsByTagName("li") as $article ) {
            $url = '';
            foreach ( $article->getElementsByTagName("a") as $link ) {
                if ( preg_match ( "@\w+.html@i", $link->getAttribute('href') ) ) {
                    $url = 'http://www.useit.com/alertbox/' . $link->getAttribute('href');
                    break;
                }
            }

            if ( $url === '' ) {
                continue;
            }

            $contents = str_replace ( "\n", " ", $article->textContent );
            $contents = preg_replace ( "/\s{2,}/", " ", $contents );
            $date = 0;
            $possibleStartDate = strrpos ( $contents, '(' );
            if ( $possibleStartDate !== false ) {
                $timestring = substr ( $contents, $possibleStartDate + 1, strpos ( $contents, ')', $possibleStartDate ) - $possibleStartDate - 1 );
                $parsedDate = strtotime ( $timestring );
                if ( $parsedDate !== false ) {
                    $date = $parsedDate;
                    $contents = str_replace ( "($timestring)", "", $contents );
                }
            }

            $this->contents[] = array ( 'url' => $url, 'title' => trim ( $contents ), 'date' => $date );
        }

        return $this->contents;
    }

    public function fetchWithSummaries($type = "RSS2", $limit = 10, $start = 0) {
        if ( empty ( $this->contents ) ) { $this->fetch(); }
        include ( "FeedWriter.php" );

        $feedType = RSS2;
        $feedDate = DATE_RSS;
        switch ( $type ) {
            case 'RSS1':
                $feedType = RSS1;
                break;
            case 'ATOM':
                $feedType = ATOM;
                $feedDate = DATE_ATOM;
                break;
        }

        $feed = new FeedWriter($feedType);
        $feed->setTitle("Alertbox");
        $feed->setLink("http://www.useit.com/alertbox/");
        $feed->setDescription("Current Issues in Web Usability - Bi-weekly column by Dr. Jakob Nielsen, principal, Nielsen Norman Group");

        if ( $type === 'ATOM' ) {
            $feed->setChannelElement('updated', date($feedDate, $this->contents[0]["date"]));
        } else {
            $feed->setChannelElement('pubDate', date($feedDate, $this->contents[0]["date"]));
        }

        $feed->setChannelElement('language', 'en-us');
        $feed->setChannelElement('author', array('name' => 'Dr. Jakob Nielsen'));

        if ( $type === "RSS1" ) {
            $feed->setChannelAboute("http://www.useit.com/alertbox/");
        }

        for ( $i = $start; $i < $start + $limit && $i < count ( $this->contents ); $i++ ) {
            $p = @DOMDocument::loadHTML ( file_get_contents ( $this->contents[$i]["url"] ) );
            $blockquotes = $p->getElementsByTagName("blockquote");
            if ( $blockquotes->length > 0 ) {
                $this->contents[$i]["summary"] = trim ( str_replace ( "Summary:", "", $blockquotes->item(0)->textContent ) );
            } else {
                $contents = $p->saveHTML();
                $endHeadline = stripos ( $contents, "</h1>" );
                $this->contents[$i]["summary"] = trim ( strip_tags ( substr ( $contents, $endHeadline + strlen ( "</h1>" ), stripos ( $contents, "<p>" ) - ( $endHeadline + strlen ( "</h1>" ) ) ) ) );
            }

            $item = $feed->createNewItem();
            $item->setTitle($this->contents[$i]["title"]);
            $item->setLink($this->contents[$i]["url"]);
            $item->setDate($this->contents[$i]["date"]);
            $item->setDescription($this->contents[$i]["summary"]);
            $feed->addItem($item);

//            echo date ( "F jS, Y", $this->contents[$i]["date"] ) . ": " . $this->contents[$i]["title"] . " -> " . $this->contents[$i]["url"] . "\n";
//            echo "\t" . $this->contents[$i]["summary"] . "\n\n";
        }

        $feed->genarateFeed();
    }
}

$s = new FetchAlertbox();
$s->fetchWithSummaries(isset($_GET['type']) ? strtoupper ( $_GET['type'] ) : 'RSS2', isset($_GET['limit']) && ctype_digit($_GET['limit']) ? $_GET['limit'] : 10);

How often have you found yourself creating a great new website design for a friend, family or a client, just to realize after all your HTML and CSS is done that the thing will need an administration panel? Way too often, the administration comes as an afterthought, and thus it ends up being incomplete, hard to use or in some cases absent. Two months down the track you end up being contacted again to do some “site updates”, and these just keep coming.

Making a good administration panel takes a bit of work, and it takes planning to ensure that the client can update the parts of the site as he/she wishes, and more importantly, that your interface gets all the information it needs. If your design has a thumbnail for every post, your administration panel has to provide the opportunity to upload an image and attach it to a post.

For years, I have been making these admin pages myself, until I read a post on WordPress custom themes. This post was on how to write a theme for WordPress, but it hinted at the possibility of having a standalone site that used WordPress as its backend, thus providing a full-fledged, well designed and familiar administration interface to any site. At first I was skeptical to the idea because I thought WordPress was way to inflexible to allow for the variety of pages I make. I was wrong. You see, if you really thing about it, all you need to make most websites are two things: Posts in categories and dynamic text/HTML blocks.

To give you an idea on how flexible WordPress is, and how it can be used, consider the following website which I am currently working on. The site is for a sound production company, and will contain the following:

• Short posts giving updates on what the studio is working on at the moment. These have to have a short text to be shown on the front page, a feature image and a rich-text full article.
• An image feed showing images of the sound studio
• A page with the various projects the studio has completed. Each project has an image, a description and one or more music tracks. Each track has two variants, a streaming-quality MP3 and a high-quality WAV.
• An about page with two separate columns, one on the sound designer, and one on the company
• A list of previous clients with a short description and a link to the client’s website

At first, this seems outside the scope of WordPress which only deals with Posts and Pages, but let’s have a closer look.

The posts are clearly just regular WordPress posts. Let us put them in a category called “Frontpage”.

The image feed is just a stream of images. We here have two choices, use WordPress’ media library, and attach all images to be shown in the feed to a static page, or use a flickr stream. I opted for the latter, but WordPress could have handled this fine!

Each music project can be a blog post (you heard me right) in the category “Projects”. We can then attach images and songs to the post using the WordPress media library. The two versions of each music file can be given the same name, and we can use the MIME type to distinguish between them.

The two text blocks 0n the about page can be two WordPress Pages (let’s call them ‘designer’ and ‘studio’)

Finally, the list of clients can be implemented using a set of links in a link category.

So, how would we go about and convert our static HTML into a fully dynamic site with a complete administration interface? Let’s dig into some code.

Getting access to WordPress from your code

The first step to a well-integrated site is to include the following code at the top of any page that needs to access WordPress functions. Naturally this does not need to be included in included/required files.

define('WP_USE_THEMES', false);
require('./wp-blog-header.php');

This gives you access to a whole set of WordPress functions (http://codex.wordpress.org/Function_Reference) that will aid us in integrating your site with WordPress. Unfortunately, the WordPress API is not very well structured, and naming conventions are a bit all over the place, but we’ll make do.

Getting content from WordPress

From the WordPress function list, there are a couple of terms you need to become accustomed to in order to start using the API. First of all, “The Loop”.

Now, we will not actually be using the loop to display any of our pages for two reasons: “The Loop” is a magic thing that mysteriously figures out what posts/pages to display, and is associated with some magic methods such as get_the_author which magically contain data about the “current” post, whatever that is. Second, it provides very little flexibility for selecting only certain posts/pages.

I will not go into the details of “The Loop” here, I will just say that it is a while loop that most WordPress templates use to print blog posts, pages, etc. to abstract away the backend query. There are several methods in the WordPress API that depend on being used in The Loop, and these usually contain “the” in the function name. Avoid these!

Next, in WordPress, pages are posts. Special types of posts, but posts nonetheless. This means that if you fetch a page, the various fields available will be the exact same as those for post, and they will be named post_title, post_content, etc…

When printing the content of posts (that is, any field that has a rich text field as input), WordPress depends on you running the data through another magical function: wpautop. This function automatically adds p tags where it thinks is appropriate to mimic the appearance in TinyMCE in the admin panel. Always put post_content through this function, otherwise your output is going to look very weird indeed.

Finally, the WordPress API usually returns objects or lists of objects. This is very convenient for most uses, but it also means that you have to take care in those cases where it doesn’t. One such method that you will probably be using is wp_get_attachment_image_src; this function actually returns a numerically indexed array.

Most function that return objects contain all the fields outlined in the appropriate table in this database diagram: http://codex.wordpress.org/Database_Description. Note that almost all models will contain an ID field which comes in very handy. Most other columns are prefixed by the name of the table, and this prefix is also used in the object attributes.

Getting posts

When getting blog posts, the main function to think about is get_posts. This function has a plethora of configuration options, but usually, you will only need the numposts option, and maybe offset. In the case of the website I was developing, I wanted just the posts in a given category

foreach ( get_posts ( 'numberposts=7&category=4' ) as $post ) {
    echo $post -> post_title;
}

This is a very simple example which only prints the title of each post, but you get the drift.

If you want a full version of a post given its ID, you would use the quite similar get_post (http://codex.wordpress.org/Function_Reference/get_post) function. This function takes a post ID, and returns an object representing that post. This object tells you nothing about the author or any attached images, so these will have to be fetched separately as such:

$post = get_post ( $_GET['p'] ); // This is probably quite insecure. Sanitize your input!
$author = get_userdata ( $post -> post_author ); // Here we should do some error checking on $post first
$images = get_children ( array(
    'post_type' => 'attachment',
    'post_parent' => $post -> ID,
    'post_mime_type' => 'image'
) );

There is a bit of voodoo going on here, so let’s take it step by step.

The first line should be pretty straightforward, we simply get the appropriate post object by its ID (which we take from the query string).

The next line is also quite simple, we fetch the user data of the user with the ID matching that of the author of the post.

Now we get to the strange bit; get_children. You see, WordPress treats almost everything as posts. Even attachments, no matter the type, are considered posts, and are part of the page/post hierarchy. Thus, to get the attachments of a post or page, we are actually getting all the children of the given object of the type ‘attachment’. I have also added a filter on ‘post_mime_type’ to ensure we only get images. Notice how WordPress sometimes uses strings as arguments, and other times uses arrays? Turns out you can usually get away with both approaches… Someone should really write a wrapper class to sort out that mess, but until then, we’ll have to deal with it. The good part though is that the process for getting the images for a page is exactly the same, just replace get_post with get_page!

The most interesting part though is showing an image you’ve fetched. WordPress “conveniently” provides user-customizable thumbnails for all uploaded images. Unfortunately, these tend to be cropped in weird ways, and are very unpredictable and unlikely to look nice. When printing an image, you have a choice between several formats, amongst others ‘thumbnail’ (the default) and ‘full’. The only one that gives you an uncropped image is ‘full’, but this will give you the image in its original resolution. True, the users can edit and scale the images in the admin panel, but how many end-users can you expect to do that? Unfortunately there is no way around that at the moment AFAIK, but one happy day…

Anyway, until then, you have a choice of two functions for printing your images: wp_get_attachment_image and wp_get_attachment_image_src. They both take the same arguments, but the difference is that the first one prints a full ‘img’ HTML tag with the alt, title, width and height attributes already set, whereas the second one just returns the image url, the widht and the height as a numerically indexed array that you can decide what to do with. They both take the ID of the image as a first parameter, and the size you want as the second. Here, you can either give a predetermined size such as ‘thumbnail’, get the full image with ‘full’ or get a cropped thumbnail that fits inside a certain box by passing an array of two values, width and height as such: array ( 64, 64 ). If you want to get the image description and title yourself, those are stored in the object you used to get the ID for wp_get_attachment_image_*, i.e. in $images[$i].

Other attachments (mp3s for instance)

When it comes to getting other post attachments, this is actually quite trivial once you know how to fetch images. Instead of using ‘post_mime_type’ => ‘image’, you simply use another MIME type. On the site I am developing, I will use the MIME type for mp3 which is ‘audio/mpeg’ as far as WordPress can tell (you can see this in the WordPress admin panel -> Media). I would therefore substitute  ’post_mime_type’ => ‘image’ with ‘post_mime_type’ => ‘audio/mpeg’. Simple as that!

To get the direct URL for a non-image attachment, you can use the wp_get_attachment_url function. As for getting the high quality version of a file, this is just a matter of selecting an attachment with the same title (i.e. the name of the file without the extension), but a different MIME type.

Dealing with pages/editable content boxes

Now, for the boxes on the about page which the administrators of the page should be allowed to edit. This is as easy as just creating two new pages in the WordPress admin and noting down the name you use. Back in your code, you can then use the following snippet to print the content of the box/page:

$page = get_page_by_title ( 'About box left' );
echo wpautop ( $page -> post_content );

By now this should look familiar. We are simply fetching the page by its title, and then passing the pages content through wpautop, and echoing the result.

Lists of links with descriptions

Our final challenge for this site will be to fetch the list of clients. We’ve already determined that we are going to use the WordPress Links library because this provides exactly the fields we need, a title, a URL and a short description. However, if you start looking through the WordPress API for anything related to links, you will come up empty handed. The reason for this is that in their wisdom, WordPress decided to call links “bookmarks” in their API for the sake of clarity. The function we are looking for here is called get_bookmarks, and again we may specify lots of parameters. In our case, however, we are only concerned with one of them; category. Since we may want to add other links later that should not show up in the clients list, we create a link category from the WordPress admin and note down the category ID. In my setup it was 3, and so my code to get the links/bookmarks becomes:

foreach ( get_bookmarks ( array ( 'category' => 3 ) ) as $link ) {
    echo '<a href="' . $link -> link_url . '">' . $link -> link_name . '</a>';
    echo '<blockquote><p>' . $link -> link_description . '</p></blockquote>';
}

Of course, this is a simplified version of the end result, but it should give you enough of an idea to get you on your way.

Final thoughts

As you have now seen, this entire page can now be administered fully through WordPress with its quite good admin panel, and the user won’t even think twice about WordPress really being a blogging tool. In fact, neither should you, because as you can see, it is more than flexible enough to be used for quite complex websites. Your users will be happy with a comfortable admin interface, and you won’t have to touch a single piece of admin code!

Almost all modern websites require some sort of administration, and this usually involves creating a separate administration page where articles can be added and users managed. Lately, I’ve been making quite a few new websites that will be released in the upcoming year, and all of these have been quite simple sites with a single user and where the administration consists mainly of adding simple news updates and updating page text. For these sites, a full blown administration panel is not necessary, and is also quite inconvenient as the user will have to go back and forth to see the results. So, what are the alternatives?

(Live examples are not available at the moment, but might come later)

AJAX driven, on-page administration

Here, the user (the person administering the website) is allowed to edit content on the same page as the content through a rich text area in a popup, and the text is then changed afterwards to reflect the users edits.

The simplest, and in my experience most flexible way of doing this is through named fields. Each block of text on the site gets its own unique name, and is linked to a plain text file on the server. In my small site setups, I usually use a structure like this:

/
 pages/
  about.inc.php
  projects.inc.php
  bio.inc.php
 api.php
 page.php
 index.php

The .inc.php files can either be plain text or contain PHP code. The most important thing is that they have a unique name. The files the usually look something like this (simplified for clarity – remember security and error checking!)

<?php

// page.php
function printBlock($name) {
    if ( !file_exists ( 'pages/' . $name . '.inc.php' ) ) return;
        echo '<div id="' . $name . '" class="editable">';
        require 'pages/' . $name . '.inc.php';
        echo '</div>';
    }
}

// api.php
require 'page.php';
$action = $_GET['a'];
$block = $_GET['e'];
switch ( $action ) {
    case 'get':
        printBlock ( $block );
        break;
    case 'post':
        file_put_contents ( 'pages/' . $name . '.inc.php', $_POST['content'] );
        break;
}

// index.php
require 'page.php';
?>
<!-- HTML structure -->
<!-- Then, whenever you're printing a block or page that should be editable, call printBlock -->
<?php printBlock ( 'about' ); ?>

Next, you will have to make some sort of JavaScript hook to make all editable areas editable. I like to use a combination of CKEditor, a simplified version of lightbox and jQuery so the end result looks something like this when a user double clicks on a box with the editable class:

Upon saving, jQuery sends a AJAX request to the api.php file with the updated contents, and also changes the contents of the block on the page using the .html() method on the element with the same ID as the block name.

In-line administration

On some sites, popup boxes simply won’t cut it. In fact, they might even become a bit cumbersome when working with news articles and such where you might want a live preview of the article as you’re typing it. Earlier, one had to have a rich text editor with a “Preview” button, but now we have a much better tool available: contentEditable. This awesome attribute allows you to tell the browser to allow the user to change the contents of an element on your page at will. Consider these screenshots that illustrate adding a new news post on a page utilizing this attribute for administration:

As you can see, this is a very simple way of creating and editing posts – and immediately seeing how it would look on the page. The major drawback is that you cannot easily accept rich inline content such as images and video, or even simple text formatting. On the other hand, such features often clutter the articles anyway. On this site, I have overcome this by allowing file attachments that are placed beneath the article based on their type (images are shown in a gallery strip, videos are embedded, etc.) Text formatting is achieved through a markdown-like syntax handled by JavaScript. There is no rich text logic in the backend.

Using contentEditable is quite simple. All you have to do is use JavaScript’s setAttribute/removeAttribute functions on any element you want to be editable. Set the attribute to true when you want it turned on, and remove it when you want it off. Apart from this, everything is quite straight-forward and very similar to the previous method of popup administration. JavaScript sends the new content to the backend, which saves it and returns the HTML rendering of the content as it would be displayed when loading the front page regularly. JavaScript then swaps the editable post area with the HTML from the server and disables editing on it.

Rounding up

Both these techniques provide quite intuitive and easy-to-access administration equivalents to classical admin-panel interfaces. They are not especially complex to build either, though they provide the user with a more comfortable and usable way to manage their sites. If you have any questions regarding these techniques, don’t hesitate to use the comment field below or e-mail me at jon <you know what goes here> thesquareplanet <and you know this one as well> com.

Web development today is a constantly struggle between three major stakeholders: the customer, the designer and the developer. The customer tries to push through his or her (often distorted and silly) mental image of the website, the designer wants to be original, creative and fancy creating lots of intricate designs with fancy visual effects, and the developer who attempts desperately to explain to both the customer and the designer why what they’re doing is a bad idea (heavy background images, crammed pages, no whitespace, confusing visual effects…). The developers aren’t all good either though – They tend to put in as many fancy tricks and solutions in the final product as they can, often resulting in exotic bugs in various browsers and usually ungraceful downgrading™. In all of this, one stakeholder is often wholly forgotten, even though it is probably the most important one; the users.

Users often don’t know the first thing about how the web works. They don’t care whether the site is optimized for Firefox, Internet Explorer, Chrome or Safari (in fact, they probably don’t even know what a browser is…) The users want a site that is visually appealing, but not distracting – informative, but not cluttered – clear, but not over-simplified – and most importantly, one that is responsive. When a user does something, they should begin to see something happening within .1 seconds (http://www.useit.com/alertbox/timeframes.html) to feel as though they aren’t being slowed down by the site itself. Furthermore, the total loading time for whatever action the user initiates should be less than a second for the user not to fall out of his or her “flow”. Way to many websites violate these simple rules, causing the site to feel unresponsive to the user, and the users are likely to jump to the next site on their list.

In this post, I hope to show you how to make your website faster – mainly through optimizing the initial page load. In order to do this, there are three steps that need to be taken: Combine, Compress and Communicate. Repeat after me: Combine, Compress and Communicate.

Combine

Many developers seem to think (albeit erroneously) that many small files are better than few large ones. This might seem intuitive since a smaller file downloads faster than a large one, and you would think they could all be gotten out of the way quicker. The truth is quite different. Due to limitations of the HTTP protocol, the browser has to initiate a new request to the server for every single file, causing quite a bit of overhead when having to download several files. Also, modern browsers limit the amount of simultaneous downloads to 6, meaning downloading all of your small files will go even slower. Add to this the sequential nature of JavaScript, and the fact that the browser stops loading the page once it hits a JavaScript piece (external or not), and doesn’t continue loading until the JavaScript file is finished downloading and has been interpreted.

Therefore, you should work to combine as many of your files as possible. Don’t jump to put all your scripts and styles inline, however (you will understand why in Communicate). Instead, you should attempt to combine all your CSS files into one, all your JavaScript into another, and all your images into a third. Ideally, you should need no more than three external files on your site. So, how do you go about doing this?

CSS and JavaScript

Combining CSS and JS files shouldn’t itself be a problem.. Open up a text editor, copy-paste all of your CSS or JS into that file, save it and upload. You should probably still keep the separated files for readability though. Of course, modern web applications are usually a bit more complicated. For instance, you might have a stylesheet that is only included on sites with ads on them or a JavaScript file that is only needed on your frontpage. In these cases, your should look into using a combinator. One of the best sites describing the techniques of combining is this one. The mod_concat plugin for Apache2 provides several advantages over traditional scripting approaches especially with regards to communication (as will be discussed later)

Images

All your images should be done as sprites. Ideally, you should even be able to put every single image on your site into a single png image. Do this, and you will substantially reduce the loading time of your site. For an introduction to CSS sprites, have a look here.

Compress

All your CSS and JS files should be compressed to reduced overall download size. Again, it is usually a good idea to keep the original, uncompressed versions of the files, and re-compress the files whenever you change them. For CSS, I recommend the YUI compiler (http://www.refresh-sf.com/yui/). It does JavaScript as well, but Google’s recently released Closure Compiler seems to be even more effective at compressing it. You can find it at http://closure-compiler.appspot.com/home. With the Closure Compiler, you can also select the advanced compiler which will decrease the total file size even more, but will mess up your files’ external API. This means that any functions you define inside your files won’t be available from the outside by the same name. The internal workings of the file will be preserved though.

Apart from minimizing the files, you should also compress them using something like GZip which is natively supported by several browsers. To see how to do this automatically with Apache2, have a look at http://www.cyberciti.biz/tips/speed-up-apache-20-web-access-or-downloads-with-mod_deflate.html.

Communicate

OK, so all of your files are combined and compressed, and you’ve never seen the CSS and JavaScript download so quickly. How can it possibly go any faster? Quite simple – by preventing the browser from having to download the files at all. Modern browsers include a lot of caching technology to prevent them from downloading unnecessary data from the server. The problem is that many web servers do not communicate properly the states of the files, and the browsers can thus not determine if a file has changed or not; and therefore they download the file just to make sure. So, what should you do?

First of all, you need to tell your web server to send out as much data as possible about your file. This especially applies to dynamic files such as those created by PHP. Have a look here for a more thorough discussion of this topic.

Second, files that are GZipped by Apache don’t always get an expiration date, causing the browser to re-download the file on every page load. To overcome this problem, have a look at the first answer on this page

Final thoughts

In the course of this post, I hope I have given you an overview of what can be done to speed up the loading time of web pages, and enough pointers to keep you going in your quest for the best speed your website can achieve. This is an ever-expanding topic, and new techniques are always appearing, so you should attempt as best you can to keep up to speed (pun intended) on the newest advances in the field.

Happy speeding!

We all use the web differently, and all have the pages we cannot live without. Following is a list of the top websites I cannot live without. What are yours?

Google Apps

Google Apps is a service that allows you to manage Google services for your own domains. For instance – I own several domains (thesquareplanet.com, casa-rioja.com, awknard.com, nerd-geek.com …), and host them all myself. I also used to host my own mail and calendar, but when I discovered Google Apps, I stopped doing that. Not only would I lose all mail that was sent to me if my server or internet connection went down, but I also had to constantly maintain security and other enhancements. With Google Apps, you can add all of your domains into a central system, and manage e-mail addresses, users, calendars and all sorts of other Google hosted services. Best thing is – it’s free! (as long as you’re non-profit that is)

I now have the GMail and GCalendar interfaces on my own domains, and can use all of their features as if I was using my Google Account.

Facebook

Although Facebook receives quite a bit of critique with regards to privacy issues, and many abandon it entirely because of this, I feel that it is truly an amazing tool on the web that should be taken advantage of. As with everything else one does online, one has to be conscious that everything that is put out can, and probably will, be used against you, but as long as this is clear, Facebook brings a lot to the table. It allows you to organize your social life, and easily stay in touch with everyone you know (and some you don’t). By using Facebook, you also create an online presence which is extremely important in this increasingly online world.

Google Reader

If you follow anything online, be it a blog, a forum, a news site, a video feed or acoolname.com, you should get a feed reader. It allows you to merge feeds from all your article sources, and view them in one grand, unified list; sorted and grouped to heart’s desires. Google Reader is an online feed reader which provides a clean online interface for you to check updates in your online world from anywhere, at any time. All you need is a Google account, and you’re on your way!

What else do I use?

• Vimeo – Upload & watch videos
• YouTube – Upload & watch videos
• GrooveShark – Listen to music online for free
• Songza – Listen to music online for free
• XMarks – Synchronize your bookmarks and passwords across browsers and computers, and access them from anywhere
• StumbleUpon – Find new, interesting pages on the web

So, you want to make websites, do you? Becoming a web developer is both very easy, and very hard at the same time. Mocking up a simple page online with some text and images is easy. Not only are there several WYSIWYG website editors (What You See Is What You Get) out there, but there are also several websites that allow you to create your page online directly through point and click. This is not web development.

Furthermore, if all you want to do is make a design for a web page in Photoshop, you are not a web developer, you are a web designer. Although many web developers tend to be web designers and vice-versa, this is certainly not a matter of implication. A web designer creates a site design, a web developer implements that design – there is nothing more to it. If you create your designs and implement them, you are a web developer as well as a web designer.

There are, of course, several degrees of web development; from basic HTML and CSS to fully-fledged PHP/ASP/<insert programming language here> web applications, but this is not the topic of this post. In this post I intend to give you, as an aspiring web developer, a couple of shortcuts, strategies, tricks and gotchas that I have found during my six years of development experience at the time of writing. This is by no means a complete guide to becoming a web developer, but more of a reference document to get you past the various obstacles browser developers, web standards, faulty documentation and operating systems have put in place to make our life a bit more interesting.

So, without further ado:

The only 5 tags you’ll ever need

HTML and XHTML both contain large amounts of tags. Too many, in fact, for them all to be useful in most cases. Remember, XHTML (and HTML to a large degree) aims to describe the structure of the content, and that is what we have all the tags for. To make the content readily available to screen-readers, text-to-speech engines, search engines and the likes. When prototyping a design, however, you should rarely make use of all of, for instance, the subtle difference between an <strong> and a <b> tag. In fact, you probably shouldn’t even care about the difference between an <em> and a <strong>; you would probably substitute them both with a <span> anyway. Although you should put in the appropriate tags when you begin to develop larger websites, or when you begin to polish the smaller ones, you will find that there are only 5 tags you really need when building an initial design.

• <span> – The fundamental inline element
• <div> – The fundamental block element
• <a> – The link
• <img> – The image
• <ul> – For making lists

Although coding your site using only these tags may be considered bad practice for the reasons explained above, they will in fact get you started quickly and substantially reduce the amount of tags you have to keep in your head when you’ve just started making web pages.

Due note that these are only content tags, and not the additional meta tags such as <link>, <style>, <script> and <meta> that you will also need to style and animate your site.

Reset your styles

The #1 reason why your designs do not work when you try to open them in a different browser from the one you initially developed and tested in is because of default margins and paddings. Every browser has its own definition for what padding and margin every element should have if you don’t specify any, and consequently, when you move to another browser, all your elements become slightly smaller or larger, and your design collapses into an unrecognizable heap of divs for no apparent reason.

Although reset stylesheets (google it) have recently become quite popular, I often find them unnecessary as they tend to reset too much – giving you more work. Instead, I just a good old rule which simply resets the margin and padding, and nothing more:

* { margin: 0; padding: 0 }

Try putting this in your document before you begin, and you’ll find cross-browser design becomes a whole lot easier!

Understand the box model

Way too many web developers don’t understand what the difference between margin and padding is, and how these are rendered together with the border of the element. Much less how to calculate the total dimensions of the element. The fact of the matter is that this is essential to being able to create potent web sites. It is also the alpha-omega of many of the CSS hacks you will encounter through your web development career.

A simple Google search reveals several images and sites trying to explain it, and one of the first results explains it quite simply:

Learn it by heart – it will save you much hassle and confusion!

Face the truth – learn to program

If you’re going to make any decent web site, you will have to learn how to program. And I’m not talking about plain ol’ HTML, I’m talking of at least Javascript, and preferably a proper server-side language such as PHP or ASP. Javascript allows you to manipulate your site dynamically to make your site a lot more interactive. For instance, you can use it to show a date-picker for an input field, validate form input without going through the server (though for security reasons you should ALWAYS check the data on the server as well), update the page behind the scenes without the user having to refresh the page (AJAX) or make elements on your page fly all over the place. All this power, however, becomes nothing but a fun topping when you consider a server-side language.

Where Javascript deals with the page the user is seeing, the server-side languages allow you to store data the user submits, add dynamic content to your site (This can be anything from a simple “Quote of the day” to allowing users to add articles, list recently added articles and show all articles using the same HTML, allowing the scripting language to fill in the content) and track user sessions (i.e. login and have their own preferences and personalized pages).

Personally, I prefer PHP to ASP, but this is completely up to you – Just learn one because you will have to!

Use a JavaScript library

Javascript is a wonderful thing, but it also quite awkward. In order to do even simple animations, you need to write a lot of code. In addition, Javascript lacks the broad set of tools that often comes with larger, self-contained languages. There are many libraries out there that attempt to “fix” Javascript in one way or another. Some simply make it easier to manipulate the DOM (Document Object Model) and do animations (jQuery is a typical, and extremely popular library that aims to do this), whilst others go all the way and manipulate Javascript’s native methods and objects to make them more powerful, more usable and more flexible. A good example of the latter is the Javascript framework/library MooTools. For larger, more complex Javascript-enabled projects, such a framework is often preferred over the lightweight jQuery equivalents. Pick the tool for the task at hand.

Know your clients

When developing a website, you need to know what browsers and resolutions you are developing for. If you are making a site for a design bureau, you can usually assume that they will have high resolution screens of at least 1280×1024 or 1600×1400, and you should design your site accordingly. In such cases a fluid design layout might be worth considering to allow your users to utilize the full resolution of their screens. More commonly though, you will be designing for the majority of users, and the majority do not have resolutions of that scale. Too many users still use a 1024×768 resolution, or even smaller, and consequently we have to take this into consideration. Usually, making a page between 900px and 950px wide makes the site viewable for most users, and at the same forces you to avoid extraneous information.

Also, determine whether you should support IE6 or not. This is a major issue as supporting IE6 requires a lot more work than not supporting it because of its blatant disregard for standards and ridiculous implementations of it at times.

A word of caution: If a designer hands you a design with a page-width wider than 950px, don’t just scale it down and get on your way! This will not only distort the design, but it will also make all the text smaller which you should avoid at all costs. Make sure all text on your site is easily readable, and try to keep to a maximum of 15 words per line. Anything smaller makes it hard to read, especially on high resolutions!

A second word of caution: Not everyone has Javascript enabled! Make sure you either provided a usable scaled-down version of your site without JS, or that you warn the user that your site requires Javascript. Don’t just leave it up to them to find that nothing works..

Prototype, then validate

If you’re told to make a design, make a quick and dirty mock-up. That is not to say it should not look like the design, but that you shouldn’t care too much whether your prototype validates or uses the correct tags. Those kinds of things can be changed later if the design is approved. Designers change their mind all the time, and you don’t want to spend a lot of time on something they are going to change or remove entirely at the next signpost.

When you’re prototyping, however, do try to make the site look similar in all target browsers. The reason for this is mainly because if you make it work in one browser and OK the design, and then, when told to make the site to the design you OK’d, you might find that what you did in that one browser cannot be done in another due to different implementations of the standard. Then what are you going to do? You OK’d the design, remember?

Follow standards, but not blindly

Standards are a good thing – no question about it. The problem is that at times, it can be a bit too strict. Especially when trying to make your pages render correctly cross-browser.

For instance, IE6 only supports the CSS :hover attribute on <a> elements, and as such, you may be force to put divs, or other block-level elements, inside an <a> tag which will not validate. It will seldom create any problem in any other browser, so theoretically you could just ignore the validation warnings. The problem is that all too many people are concerned about whether their code validates or not. Bottom line is, if it works, and you’re confident that it works, and will continue to work cross-browser, the warning is really not that important.

That said, if you can follow the standard and validate your site, do so. It is a form of guarantee that your site design should not be broken by future browsers that might have different rendering engines or follow newer standards. Often, when you’re code doesn’t validate, it is quite simple to fix the issue. The fix also tends to come at the expense of a degraded experience to IE6 users…. Oh well, that’s sad isn’t it?

Plan for security, but delay the implementation

When developing web applications, security should be a major concern. Unfortunately, it is often completely overlooked, or applied seemingly haphazardly; “Oh, I think I’ll just put in a striptags here and we’ll be good to go!” Not thinking about, and planning, for security can cost you very dearly in the end.

On the other hand, security takes a lot of time and effort to implement, and in the initial stages of web development – when prototyping a new concept or design – you don’t really want to bother with that kind of thing. The danger is that once the prototype is complete, you decide to skip the extra work and simply continue work on the prototype with all its hacks, shortcuts and security holes. Don’t do it! Instead, plan you security measures thoroughly from the beginning. Do not simply say: “We’ll run striptags on all output and addslashes on all input”, but set out an abstraction layer which allows you to secure all input and output, not matter where it’s going. Decide on security policies and content rules beforehand, instead of patching your old code when someone breaks your system.

This might seem like a lot of work, and it is! It is, however, also very necessary to prevent all sorts of nasty security breaches. The upside of course is that you do not have to implement these security measures when prototyping. In fact, you SHOULDN’T implement them at this stage. Prototyping is about coming up with something workable quickly to see if it works as intended and according to plan. This does not require excessive security precautions. Just remember to put them in when you begin developing the real system.

Use as few images as possible and merge those you can

Everyone does not have a fast internet connection. In fact, there are still those out there browsing the web on a 56.6kbit modem, and although they are a minority, it should tell us that the excessive use of images and other external media is not exactly taking care of your users. Rather the contrary. That is not to say that you should never use images on a site, in fact images are usually essential to a visually appealing website. The danger is excessive use of media.

A common misconception about external media on web pages is that as long as one uses small images, everything is fine. The truth is that the fewer files the browser has to fetch, the better. Every new request to the server comes with overhead and delay waiting for the server to send its response. Where you can, merge together your images and use CSS sprites to display only the part you wish to show. This saves you from the overhead and increases the overall compression rate of your images.

Don’t make a mess – modulate

When you develop websites, it is all too easy to put everything in one file. When prototyping this makes your development speed much higher, but as your system grows it will soon become slow and unmanageable. Instead, try to split your system in to logically separated units. A good place to start is to implement a MVC oriented framework for your site.

Avoid the lazy fallbacks

If you can’t understand how to do something the first time around, don’t fall back to the easy solutions.

• Tables are for tabular data
• Absolute positioning is for overlay windows
• Frames are an absolute no-no

Instead, try to learn something new, and ask someone who knows more about the problem than you do to help you out.

When asking for help

If you are not already a programmer, do not make the mistake, as many do, of demanding answers or asking for a piece of code instead of advice. There are plenty of skillful people out there willing to help, but they will not help you if:

• You do not ask nicely
• You do not provide source code and other relevant material for them to examine for problems
• You ask for the complete solution
  • Rather than saying “I need a script that does x”, try to make one that does it yourself, and then ask: “I’m trying to do x, and have come up with y. I seem to be stuck because of z. Has anyone got a suggestion about how I might accomplish this?” where y is your source code and z is your problem.

Essential hacks and gotchas:

In the world of web development, there are some gotchas that are very very common, but are not too often explained. I will try to bring up some of those here:

The “overflow: hidden” fix:

Say you have the following code:

<div>
<div style="float: left; height:200px;"></div>
<div style="float:right; height: 300px;"></div>
</div>

How tall would you guess the parent div to be? 200? 300?

It will in fact be 0px tall. The reason is that the browser does not count floated elements when considering the height of the element. This “bug” becomes very apparent if you, for instance, try to position something absolutely a certain distance from the bottom within the parent div, because that element would really be position from the bottom of the tallest NON-FLOAT element within the parent.

There are several fixes for this problem, but the most common (and cleanest) method is simply to put the style “overflow: hidden” or “overflow: auto” on the parent. For some reason the browser then takes the floating children into account and correctly sets the height of the parent.

Absolute positioning

Consider the following code:

<div style="margin: 50px;">
<div style="position: absolute; top:50px; left: 50px;"></div>
</div>

Where would you say the child div would be positioned in relation to the browser viewport? At (100, 100)? You would be wrong. The correct answer is at (50,50). The reason for this is that absolute positioning is considered relative to the first parent element that has been given a “non-flow” position. That is, any element that has its “position” style set to something else than “static”. If no such parent exists, it is positioned in relation to the viewport. A quick fix is simply to set the style declaration “position: relative” on the parent you want to absolutely position the child in relation to. Because relative positioning does in fact not move the element at all unless left and top are specified this means the rest of the page is not affected, and we can get on with our work.

Due note however, that doing this means that ALL absolutely positioned children of the element that receives “position: relative” will be positioned relative to that element. Thus, you cannot have two children elements where one should be relative to the viewport and one relative to the parent.

Full height column backgrounds

A common design layout is one which has multiple vertical columns (often two or three) next to each other. In the designs, these columns almost always are the same length, with the background color extending to the very bottom of the elements. The problem is that whilst this is very much possible in Photoshop, in HTML an element is only as large as its content or as large as you specify it to be. And unfortunately, CSS does not allow you to specify an element to be as tall as another element. Therefore, if you have multiple columns, and at least one of them has content that will vary in height, you will inevitably end up with the columns being different height. If you then set a background color on each of them, you will notice that the background color is only drawn on the element, thereby making it very obvious that the columns are not the same height.

So, how do you make the columns appear to both be as high as the tallest column? Faux columns. In essence, this approach depends on the parent being as tall as the tallest of the children – which it will be unless we’re floating the columns; in which case we can apply the “overflow: hidden” trick. A background image containing all column backgrounds is then applied to the parent and repeated down its entire height. Read the link for full implementation details.

Working with IE

Let’s face it – IE makes our life terrible. With IE8, Microsoft is starting to get it right, but IE7 and particularly IE6 gets a lot of things wrong. Luckily, the fact that they don’t follow the standards also gives us several methods of giving specifically tailored instructions to IE. There are two ways of approaching IE specific hacks – the quick, simple way, and the longer but better way:

The quick fix

Targeting IE6 in CSS: Prefix your selector with “* html “

Targeting IE7 and 6 in CSS: Prefix the attribute with a star (no space between the star and the attribute) – Note that this hack will invalidate your CSS!

The good fix

Use the Internet Explorer only Conditional Comments

Jumping pages

Okay, so you’ve made you site perfect. It is centered, looks beautiful and navigation is smooth as a kitten’s hair. Just one more page to test – the “About us” page with lots of text… As you click the link, your entire page jumps slightly to the left. You try to figure out why, and notice that you now have a scrollbar which means the center of the page has moved, and so, dutifully following your CSS, the browser has moved you page to the new center.

The solution to this annoying problem is to make sure that the scrollbars are always present, but are grayed out when there is nothing below the fold. And how do you do that? Like this:

html {
	height: 100%;
}
body {
	min-height: 100.1%;
	overflow-y: auto;
}

Centering

CSS has many ways of centering, to the confusion and irritation of most developers:

text-align: center – This makes inline elements center horizontally in its parent element

vertical-align: middle – Should be applied to inline element and sometimes centers and element vertically if you’re lucky (press “sometimes” for more details)

margin: 0 auto – Centers a block-level element by setting its left and right margins to the same value. Here, IE of course has to mess up the beauty, and requires the parent to have “text-align: center” in order for it to work. Remember to reset the text-align to left inside the element!

position: absolute – This one requires a bit more explanation. The idea here is that you position an element in the center of another by moving it 50% from the top and 50% from the left, and then move it back by half its dimensions using a negative margin. For example – if you wanted to center an element that is 400x200px within its parent, you would first of all set “position: relative” or similar on the parent (see the Absolute positioning headline further up in this post), and then you would set the following styles on the element itself:

position: absolute;
/* Center vertically */
top: 50%;
margin-top: -200px;
/* Center horizontally */
left: 50%;
margin-left: -100px;

If this does not make sense, read it again.

display: table-cell – This quite new method relies on using CSSs ability to render any element as if it was a table cell, and then using the vertical alignment property of a table cell to center the content. It is especially good for centering text! View it here.

Use a Doctype

A Doctype is not something one puts in simply to make the page validate, it does actually have an effect as well. In the case of some browsers, the determine whether to render the page according to the standard or not based on the presence of a Doctype. Therefore, put it in!

Final words

This non-exhaustive list contains my experiences from web development so far, and will probably be expanded upon by both me and hopefully some of you (use the comment field below!). Use it for what its worth, and avoid the pitfalls that are all too easy to fall into when one is new in the field. If you have any questions or comments regarding these notes, feel free to post your comment below, and I’ll do my best to give you a proper response!

Happy coding!

Ever wanted to grab some data from a webpage, or perhaps several, manipulate it in one way or another, and then format access the data through, for instance, JavaScript? Earlier, you would have to write such a script in PHP, ASP, Perl or another scripting language on your web server, and then get the data from that script through JavaScript. This meant you had to run a web server in the first place, and that it would have to support not only fetching data remotely, but also the scripting language you chose to use. And if you didn’t know any scripting languages apart from JavaScript, you would be out of luck.

The engineers over at Yahoo came up with a solution: Yahoo! Pipes. Pipes allows you to set up a series of, well, pipes, for your data to go through, each manipulating it slightly, until you have processed your raw data into what you wanted to extract in the format you want it.

Pipes is an essential tool in today’s world of mash-ups ( that is, sites that do not have any content themselves, but just collect data from other sources, combine and manipulate them, and then publish the processed data ), because it allows you to do most of the work on a remote server ( Yahoo! ), through a simple drag-and-drop GUI ( Pipes ), and fetch the end result using nothing but JavaScript. You can of course access the data using whatever technology you want, since you determine the output format.

Just to give a simple example of what Pipes can do: Say I wanted to create a feed for my users that contained all my posts, except those that contain the word “task”. In order to do this before Pipes, I would have to write up a new PHP script that would fetch all my articles from the database, filter out those with “task” in them, and create a whole new RSS document. Replicating large portions of the code that renders my current feed. Or, I could change my current RSS feed generator so that it accepted URL arguments that allowed filtering by words. All of this would require me both to edit files on my server and make my scripts more complex.

With Yahoo! Pipes, I can simply set my feed as the input source using feed auto-discovery, set up a filter to block items containing “task”, and output the resulting elements as RSS XML. Done!

You can see my pipe for doing this at: http://pipes.yahoo.com/jonhoo/nontaskblog

Oh, and did I mention you can publish pipes…? ;)

PS: If you want to fetch the feed form a WordPress blog, notice that you should not input the feed URL directly, but rather select the source to be “Site Feed”, and then input the URL for the front page of your blog.