Floating points: Make Gaussian elimination go wrong
Taking this as a question about numerical inaccuracy in calculations rather than in conversion from numbers which can't be exactly represented in 64 bits to 64-bit representations, we're interested in nearly singular matrices.

Let's take the simplest possible case:

[a b] [x] = [p]
[c d] [y]   [q]


has solution

[x] = [(dp - bq) / (ad - bc)]
[y]   [(aq - cp) / (ad - bc)]


so that if ad - bc is very small we find that errors in dp - bq and aq - cp are magnified.

We also find that scaling p and q scales the absolute error linearly, which points out a massive flaw in the scoring system of this question.

How can we get a nearly singular matrix? In order of ill-conditionedness:


a=1+e, b=c=d=1 gives ad - bc = e. This allows us to get a determinant of 1 ulp (which for IEEE 64-bit is on the order of 10^-16).
a=1+e, d=1-e, b=c=1 gives ad - bc = -e^2. This allows us to get a determinant on the order of 10^-32.

2
1.000000000000001 1
1 0.999999999999999

1 1


gives output:

1.0 1.0 | 1.0   
1.0 1.0 | 1.0   

Result: -9.0    10.0


rather than x = 100000000000000, y = -100000000000000

Similarly,

2
1.000000000000001 1
1 0.999999999999999

1E307 1E307


gives

1.0 1.0 | 1e+307    
1.0 1.0 | 1e+307    

Result: -9.1120238836e+307  1.01120238836e+308  


rather than x = 1E321, y = -1E321. This seems to roundly beat Johannes' solution (it gives a score on the order of 10^642 vs 10^588), and moreover it would continue to generate about the same score even if the input numbers were tweaked to be exactly representable in IEEE 64-bit, whereas his would then score 0.
a=d=e, bc=0 gives ad - bc = e^2 but this time we can choose e to be a very small number rather than 1 ULP. In principle this allows us to get a determinant on the order of 10^-600 (i.e. underflowing IEEE 64-bit). However, so far I haven't managed to get a decent score without an infinity in the answer, and I gather from your chat with Johannes that even though the question doesn't rule out infinite scores, in practice you do.