How is the environment designed for testing a reinforcement learning algorithm?
In reinforcement learning, the "environment" is typically a set of states the "agent" is attempting to influence via its choice of "actions". 

For example, in "Reinforcement learning design for cancer clinical trials" by Zhao, Kosorok, and Zeng (2009),


  ..."states" may represent individual patient covariates and "actions" can
  be denoted by various treatments or dose levels.


More abstractly, in "A brief introduction to reinforcement learning" by Murphy (1998),


  The environment is a modeled as a stochastic finite state machine with inputs (actions sent from the agent) and outputs (observations and rewards sent to the agent).


So, the "structure of the environment," what signals are relevant, and how they interact is highly specific to the problem you're trying to solve, as is the choice of where to draw the boundary between the agent and the environment.

In "Reinforcement Learning: an Overview", Sutton and Barto (1998), give an overview of the Agent-Environment interface and note:


  The agent-environment boundary can be located at different places for
  different purposes. ... In practice, the agent-environment boundary is
  determined once one has selected particular states, actions, and
  rewards, and thus has identified a specific decision-making task of
  interest.


"Reinforcement Learning: A Survey" by Kaelbling, Littman, and Moore (1996) also provides an overview of the problem.