What would happen if you caught a PokeBall with another PokeBall?
While it's not an ordinary Pokéball, the Mewtwo ball is capable of capturing another Pokéball inside it.



This is demonstrated in the first Pokemon movie and quoted in the synopsis section of the movie's wiki page. A gif of the event is below too:


  "Ash tries returning Bulbasaur and Squirtle (who have been fighting the balls bravely), but it's no use. Two of Mewtwo's Poké Balls fly over and capture Bulbasaur and Squirtle's Poké Balls."




Granted, this is a special kind of ball developed by Mewtwo, so I can't say for sure in this answer whether a standard Pokéball would behave similarly.

Additionally, the catch rate for the Mewtwo ball is listed as 100% on the wiki page. Therefore it's impossible to say whether it would increase the catch chance of a wild pokémon through stacking, as it's a guaranteed capture with just one.