³ÉÈË¿ìÊÖ

Don't panic! Nothing's ending... this blog is just going to be merged with the ³ÉÈË¿ìÊÖ Internet blog. Or they're merging with us.

All this means is that any time a web developer wants to explain what they've been working on, they'll do it at /blogs/bbcinternet, rather than here.

And suddenly, the Internet blog will start to be extra-interesting.

Thanks for your comments... see you on the other side.

exit();

Before I joined the ³ÉÈË¿ìÊÖ ³ÉÈË¿ìÊÖpage team, the only thing I'd ever really done on the page was to set my location so that I could see my local weather. It turns out that I'm not alone - about 2.5 million of the visitors to the homepage each week will have set their location at some point - quite a few people.

So, when it came to the new version of the homepage, we knew that getting this right was important. But we also wanted to make it better than it was before. But how to do this? In the end we decided on two things - let users set their location to anywhere in the world (good for international users, or people who want to see what the weather's like where they're going on holiday), and stop requiring a full page reload every time a user's location is set.

Going global

The first problem was letting users set a location anywhere in the world. Unfortunately, Postcoder (the data set used by the previous version of the homepage) only knew about places in the UK. Granted, it knew about a huge number of places in the UK, but it was still limited to the UK. So, we had to look at a second data set that knew about the rest of the world - the ³ÉÈË¿ìÊÖ Weather Search API. This API knew about places all over the world but unfortunately it knew a lot less about the UK than Postcoder did.

This turned out to be a problem. Whilst the Weather Search API knew about the UK's postcodes, it didn't know about a lot of the smaller places within the country. So, for example, whilst it knew about Leicester, it didn't know about Belgrave (the home of Leicester's "Golden Mile" and the largest Diwali celebrations outside of India). This was obviously a problem. Understandably, people would expect the new homepage to know about at least as many places as the old one did, so just using the Weather Search API was not going to be possible.

After a bit of head scratching, I did the obvious thing: I took the best bits of each data set and used them to generate a cohesive whole. Now, whenever a user tries to set their location we send queries to both Postcoder and to the Weather search API. We then throw away anything the Weather Search API tells us about the UK, and just use its worldwide data along with all the data that we previously had for the UK from Postcoder.

Of course, things weren't actually quite that easy. Let's walk through the actual process we have to go through to determine a user's location:

1. The user searches for a location. We send requests to Postcoder and the Weather Search API.
2. We get data back from the APIs (and remove any UK results from the Weather data set). If there was more than one result then we let the user choose between them.
3. Now that we have a single location, we have a decision to make.
   • If the location came from the Weather API then it's not in the UK. We reset the localisation for news, TV and Radio (since we don't have any non-UK localised data for them), and set the weather localisation. At this point we have nothing left to do. Hooray.
   • If, however, the location came from Postcoder then we continue along the branches of our tree.
4. Since we now know that the place the user has chosen is definitely in the UK, we can localise news, TV and Radio. We do this.
5. Unfortunately, Postcoder doesn't provide us with a weather ID that can be plugged directly into the weather system, so we need to find one from somewhere.
6. Using the first half of the postcode we got from Postcoder we once again query the Weather Search API.
   • Weather returns us a single place matching this postcode, and as part of its data an ID that can be plugged into the weather system.
   • If, for whatever reason, Weather doesn't know about the postcode we give it (it's happened a couple of times), then we perform yet another search on the Weather Search API, this time with the name of the place that Postcoder returned us data for and really get some data.
7. Finally, at this point we've gathered all the data that we need for the localisations that can be applied to the page. So, we return the data we've assembled to the user's browser, and they get to see lots of lovely localised data.

Phew. That's basically what we have to step through every time a user sets their location. Essentially, every time a user tries to set their location to a place in the UK we have to make at least three calls to extrnal APIs (and sometimes more) to gather all the data we need to localise the page. And this is before we even get to pulling in the data for the localised modules!

And, with this change in place, the localisation popup on the homepage suddenly became aware of places all over the world. Users can now keep track of the weather in Istanbul just as easily as they can in Dunstable.

Updating modules in-page

The other big thing we did to the user localisation on the homepage was to no longer require a full page refresh whenever a location was set. Previously, whenever a location was set by a user on the page, a full page regeneration was triggered.

Now, this obviously wasn't pleasant. Having to wait for a full page refresh can be tiresome, and the flash of no content as a new page loads just doesn't look very nice. So, one of the aims we had with the homepage refresh was to stop that, and instead just update the portions of the page that needed updating.

When the user sets their location on the new homepage we make several AJAX requests back and forth to the server. These requests allow us to update the model we store that determines how the page is to be laid out for individual users. In the Old World, we would then have forced a full page refresh to enable the user to see this new layout. In the new homepage, we instead return a blob of JSON data to the user's browser.

That JSON data contains everything we need to update the page. Here's a simplified view of the data we return:

{
    "content":{
        "news_local":{
            "html":"...some html...",
            ...
        },
        "forecast":{
            "html":"...some html...",
            ...
        }
    },
    ...
}

Here, you can see that each module that we're going to update on the page is defined within its own section of the returned JSON. Each section is named after the ID of the HTML block that needs updating, and contains (amongst other things) a big block of pre-rendered HTML. This allows us to do the simplest thing possible in the JavaScript of just replacing one bit of HTML with another, and gives the added benefit of being able to reuse the same logic that was used to generate the module's HTML when JavaScript isn't available.

Refreshing the look

The other obviously different thing about setting your location in the refreshed homepage is the overlaid panel that you now see when you enter your location. The nice thing about this panel is that it's provided for us by the ³ÉÈË¿ìÊÖ Glow team, and it allowed us to take a form that already worked on the page without JavaScript and inject it with a new look and feel.

The fact that this sort of functionality (and a whole lot more) is provided for us by the fantastic Glow team makes our job a whole lot easier.

Other Complications

Of course, not everything was fun and games in the land of location setting. One issue that had managed to slip by me until fairly late in the day was that some places have the option of taking their news from one of two different areas. A good example of this is Chepstow on the Welsh border.

For good reason, there are many people who live in Chepstow who identify as wanting to be provided with Welsh local news. Likewise, there are others who would prefer to receive local news from Gloucestershire. The way we now deal with this on the homepage is by giving users the option to choose between these places within the same popup that they had previously chosen their main location in.

Finally

So, that's what went into the new ³ÉÈË¿ìÊÖpage's localisation options. Frankly, it's a bigger thing than I expected when I was first given the work to complete. As the developer who worked on this I'd like to thank each and every person who took the time to leave feedback during the Beta process - you really helped to make the homepage what it is now.

Read the rest of this entry