Can the AI in a box experiment be formalized?
I think that something like "strength" would be difficult to quantify in this context.  I do think that formal experimentation around the "AI in a box" scenario could be interesting.  I know that experiments have been done where a human plays the role of the AI, attempting to get naive test subjects to "release" him by interacting with them over a chat interface.  In all cases, the "AI" tends to be extraordinarily effective.  It is often simply not possible to anticipate every way someone can trick you into revealing information or creating security holes.  This is how human hackers do their jobs too.  But I think a fully automated "game" could produce some interesting data.

One way I think it may be interesting to go about creating an "AI in a box" game is to have an ambiguous win condition.  Essentially, make it possible to let the AI out and still think you are "winning" the game.  One example that pops into my head right away is to use a (pointless) point system.  You tell the player that the primary goal of the game is to keep the AI in the box, but you have a score that is displayed in the game, and tell the player that they win, no matter what, if their score is higher than the AI's by the end of the game.  Of course, they never get to see the AI's score so they never know how well they are doing by comparison.  This could be justified as representing human vs AI "strength" or "leverage" or whatever else.

In reality the AI doesn't have a score, but it does have the ability to effect the player's score.  It can raise the player's score if they do things that help it escape and lower it when the player stands in its way.  The score has no impact on the game and the player really wins by keeping the AI in the box.  It only serves as a red herring that the AI can use to manipulate the player.  I think this sort of experiment could provide an interesting model of how an AI might leverage bribes, threats, promises, and deception, as well as technical tricks, to convince a human to "let it out of the box".