I've found a solution: A CD case! I noticed that pushing the GPU up a little bit prevents it from failing at startup. So I just jammed something in there that fits perfectly between the floor and the underside of the graphics card. Hey presto, computer works again.

Three cheers for MacGuyver!

Er... Your GPU sits flush on the bottom of the case? I'm surprised it doesn't overheat.

No it's in the middle. I am using a CD case (standing up on it's side) to apply continous pressure to the GPU's underside so it is now firmly in the slot without any 'drooping'...

That's... Odd. The GPU should be just fine free standing. Just to be sure, you did screw it into the back of the case, right?

Doesn't matter how tight it's screwed in there - it doesn't start at all without applying pressure.

It works perfectly now.