Census 2016, not quite as smooth as hoped

Census 2016

This is what greeted me after half an hour of plugging away, entering data on the population of my household:

Census Session expired

I was somewhat disappointed. Not that I was interrupted, but that “All unsaved information was lost.” Was it? Really?

On the online census form there was no ‘Save’ button. There was only a ‘Save and Exit’ button, and who wants to exit only to then log back in? So I assumed (and I still hope) that there was some kind of autosave mechanism for the data I’d already entered.

So, had it been lost? Couldn’t tell of course. The Australian Bureau of Statistics (ABS) website was completely unresponsive. Over the next few hours the main web site came up, but the pop-up invoked by the ‘Complete my Census’ button failed to connect. Then that started to connect, after a lengthy delay, to:

Come back in 15 minutes

After a while the lengthy delay diminished, presumably as people gave up, so by around 10pm the popup snapped into place with the same ‘wait 15 minutes’ message.

Incidentally, as I write this the next morning, some 13 hours after the major problems started, the same message appears. But we now know that ‘the system is [NOT!] very busy’ because no one is using it.

So what happened? The ABS says that there were four ‘Denial of Service’ attacks on census night. Despite the first three, two million households successfully lodged their census forms online. With the fourth, the ABS says it switched off the system ‘to ensure the integrity of the data.’

David Kalisch, the Australian Statistician, says, “Steps have been taken during the night to remedy these issues, and I can reassure Australians that their data are secure at the ABS.” As I write an hour has passed since a promised 9am further update, with no update forthcoming.

The 10am news reports that the Census web site is back up. That depends on what you mean by back up. Here’s what the site says:

Census site unavailable

A Denial of Service attack is where access to a web site is blocked by the attackers’ computers sending tens, hundreds, thousands or even millions of access requests to the site. Any web site can be overloaded if enough people try to get on at once. Even if many of those ‘people’ are really just hackers’ computers.

But perhaps this attack merely pushed things over the edge. The ABS had promised plenty of capacity earlier in the day, indicating that it could handle a million ‘form submissions’ per hour, which was double what it expected. But back-of-the-envelope calculations suggested that Tuesday night after dinner would see several million people on the site at once.

Then there’s the question of what a ‘form submission’ is. In normal parlance, ‘form submission’ means handing in the completed thing. But in Web design a form may be submitted for each piece of information.

The online Census form had built in processing. The form changed on the fly depending on what was entered in each individual box. For example, if you ticked ‘self employed’, it instantly removed the questions about employers and replaced them with questions about the business. That kind of responsiveness could be programmed into the web code itself to some extent (which means it would be processed on the local computer), but there’s a chance that it was submitting the information to the ABS site for validation.

More likely a ‘form submission’ was on a per page basis. Conditional processing within a page makes sense locally, but it would be poor design indeed if each pages wasn’t sent back to the ABS’ servers as it was completed. Would each page submission count as a ‘form submission’?

I submitted about 30-40 pages before it crashed out on me. If each of those was a ‘form submission’ and there were several million people doing the same thing …